How to improve performance when having a huge list of bytes to be written to file in python? : In python, I have a huge list of floating point values(nearly 30 million values). I have to convert each of them as 4 byte values in little endian format and write all those to a binary file in order.

For a list with some thousands or even lakhs of data, my code is working fine. But if the data increases, it is taking time to process and write to file. What optimization techniques can I use to write to file more efficiently?

As suggested in this blog, http://dabeaz.blogspot.in/2010/01/few-useful-bytearray-tricks.html?m=1 , I am replacing all the small writes to a file by the use of bytearray. But still, the performance is not satisfiable.

Also I have tried multiprocessing (concurrent.futures.ProcessPoolExecutor()) to utilize all the cores in the system instead of using a single CPU core. But still it is taking more time to complete the execution.

Can anyone give me more suggestions on how to improve the performance(in terms of time and memory) for this problem.