How does choosing between pre and post zero padding of sequences impact results : <p>I'm working on an NLP sequence labelling problem. My data consists of variable length sequences <code>(w_1, w_2, ..., w_k)</code> with corresponding labels <code>(l_1, l_2, ..., l_k)</code> (in this case the task is named entity extraction).</p>

<p>I intend to solve the problem using Recurrent Neural Networks. As the sequences are of variable length I need to pad them (I want batch size >1). I have the option of either pre zero padding them, or post zero padding them. I.e. either I make every sequence <code>(0, 0, ..., w_1, w_2, ..., w_k)</code> or <code>(w_1, w_2, ..., w_k, 0, 0, ..., 0)</code> such that the lenght of each sequence is the same. </p>

<p><strong>How does the choice between pre- and post padding impact results?</strong> </p>

<p>It seems like pre padding is more common, but I can't find an explanation of why it would be better. Due to the nature of RNNs it feels like an arbitrary choice for me, since they share weights across time steps.</p>
