How to iterate over files in an S3 bucket? : <p>I have a large number of files (>1,000) stored in an S3 bucket, and I would like to iterate over them (e.g. in a <code>for</code> loop) to extract data from them using <code>boto3</code>.</p>

<p>However, I notice that in accordance with <a href="http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.list_objects" rel="noreferrer">http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.list_objects</a>, the <code>list_objects()</code> method of the <code>Client</code> class only lists up to 1,000 objects:</p>

<pre><code>In [1]: import boto3

In [2]: client = boto3.client('s3')

In [11]: apks = client.list_objects(Bucket='iper-apks')

In [16]: type(apks['Contents'])
Out[16]: list

In [17]: len(apks['Contents'])
Out[17]: 1000
</code></pre>

<p>However, I would like to list <strong>all</strong> the objects, even if there are more than 1,000. How could I achieve this?</p>
