Pytorch doesn't support one-hot vector? : <p>I am very confused by how Pytorch deals with one-hot vectors. In this <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" rel="noreferrer">tutorial</a>, the neural network will generate a one-hot vector as its output. As far as I understand, the schematic structure of the neural network in the tutorial should be like:</p>

<p><a href="https://i.stack.imgur.com/1v35k.png" rel="noreferrer"><img src="https://i.stack.imgur.com/1v35k.png" alt="enter image description here"></a></p>

<p>However, the <code>labels</code> are not in one-hot vector format. I get the following <code>size</code></p>

<pre><code>print(labels.size())
print(outputs.size())

output&gt;&gt;&gt; torch.Size([4]) 
output&gt;&gt;&gt; torch.Size([4, 10])
</code></pre>

<p>Miraculously, I they pass the <code>outputs</code> and <code>labels</code> to <code>criterion=CrossEntropyLoss()</code>, there's no error at all.</p>

<pre><code>loss = criterion(outputs, labels) # How come it has no error?
</code></pre>

<h2>My hypothesis:</h2>

<p>Maybe pytorch automatically convert the <code>labels</code> to one-hot vector form. So, I try to convert labels to one-hot vector before passing it to the loss function.</p>

<pre><code>def to_one_hot_vector(num_class, label):
    b = np.zeros((label.shape[0], num_class))
    b[np.arange(label.shape[0]), label] = 1

    return b

labels_one_hot = to_one_hot_vector(10,labels)
labels_one_hot = torch.Tensor(labels_one_hot)
labels_one_hot = labels_one_hot.type(torch.LongTensor)

loss = criterion(outputs, labels_one_hot) # Now it gives me error
</code></pre>

<p>However, I got the following error</p>

<blockquote>
  <p>RuntimeError: multi-target not supported at
  /opt/pytorch/pytorch/aten/src/THCUNN/generic/ClassNLLCriterion.cu:15</p>
</blockquote>

<p>So, one-hot vectors are not supported in <code>Pytorch</code>? How does <code>Pytorch</code> calculates the <code>cross entropy</code> for the two tensor <code>outputs = [1,0,0],[0,0,1]</code> and <code>labels = [0,2]</code> ? It doesn't make sense to me at all at the moment.</p>
