PyTorch Binary Classification - same network structure, 'simpler' data, but worse performance? : <p>To get to grips with PyTorch (and deep learning in general) I started by working through some basic classification examples. One such example was classifying a non-linear dataset created using sklearn (full code available as notebook <a href="https://github.com/philipobrien/colab-notebooks/blob/master/Non_Linear_Data_Classification.ipynb" rel="noreferrer">here</a>)</p>

<pre><code>n_pts = 500
X, y = datasets.make_circles(n_samples=n_pts, random_state=123, noise=0.1, factor=0.2)
x_data = torch.FloatTensor(X)
y_data = torch.FloatTensor(y.reshape(500, 1))
</code></pre>

<p><a href="https://i.stack.imgur.com/w20Tb.png" rel="noreferrer"><img src="https://i.stack.imgur.com/w20Tb.png" alt="enter image description here"></a></p>

<p>This is then accurately classified using a pretty basic neural net</p>

<pre><code>class Model(nn.Module):
    def __init__(self, input_size, H1, output_size):
        super().__init__()
        self.linear = nn.Linear(input_size, H1)
        self.linear2 = nn.Linear(H1, output_size)

    def forward(self, x):
        x = torch.sigmoid(self.linear(x))
        x = torch.sigmoid(self.linear2(x))
        return x

    def predict(self, x):
        pred = self.forward(x)
        if pred &gt;= 0.5:
            return 1
        else:
            return 0
</code></pre>

<p>As I have an interest in health data I then decided to try and use the same network structure to classify some a basic real-world dataset. I took heart rate data for one patient from <a href="https://physionet.org/physiobank/database/bidmc/" rel="noreferrer">here</a>, and altered it so all values > 91 would be labelled as anomalies (e.g. a <code>1</code> and everything &lt;= 91 labelled a <code>0</code>). This is completely arbitrary, but I just wanted to see how the classification would work. The complete notebook for this example is <a href="https://github.com/philipobrien/colab-notebooks/blob/master/Basic_Heart_rate_Classification.ipynb" rel="noreferrer">here</a>.</p>

<p><a href="https://i.stack.imgur.com/6izzI.png" rel="noreferrer"><img src="https://i.stack.imgur.com/6izzI.png" alt="enter image description here"></a></p>

<p>What is not intuitive to me is why the first example reaches <strong>a loss of 0.0016 after 1,000 epochs</strong>, whereas the second example only reaches <strong>a loss of 0.4296 after 10,000 epochs</strong></p>

<p><a href="https://i.stack.imgur.com/rDag4.png" rel="noreferrer"><img src="https://i.stack.imgur.com/rDag4.png" alt="Training Loss for Example 1"></a></p>

<p><a href="https://i.stack.imgur.com/PMubT.png" rel="noreferrer"><img src="https://i.stack.imgur.com/PMubT.png" alt="Training Loss for Heart Rate Example"></a></p>

<p>Perhaps I am being naive in thinking that the heart rate example would be much easier to classify. Any insights to help me understand why this is not what I am seeing would be great!</p>
