How to destroy Python objects and free up memory : <p>I am trying to iterate over 100,000 images and capture some image features and store the resulting dataFrame on disk as a pickle file. </p>

<p>Unfortunately due to RAM constraints, i am forced to split the images into chunks of 20,000 and perform operations on them before saving the results onto disk.</p>

<p>The code written below is supposed to save the dataframe of results for 20,000 images before starting the loop to process the next 20,000 images. </p>

<p>However - This does not seem to be solving my problem as the memory is not getting released from RAM at the end of the first for loop</p>

<p>So somewhere while processing the 50,000th record, the program crashes due to Out of Memory Error.</p>

<p>I tried deleting the objects after saving them to disk and invoking the garbage collector, however the RAM usage does not seem to be going down.</p>

<p>What am i missing? </p>

<pre><code>#file_list_1 contains 100,000 images
file_list_chunks = list(divide_chunks(file_list_1,20000))
for count,f in enumerate(file_list_chunks):
    # make the Pool of workers
    pool = ThreadPool(64) 
    results = pool.map(get_image_features,f)
    # close the pool and wait for the work to finish 
    list_a, list_b = zip(*results)
    df = pd.DataFrame({'filename':list_a,'image_features':list_b})
    df.to_pickle("PATH_TO_FILE"+str(count)+".pickle")
    del list_a
    del list_b
    del df
    gc.collect()
    pool.close() 
    pool.join()
    print("pool closed")
</code></pre>
