Best way to read million line txt file while using multi processing to improve speed in python : So I'm running multiple functions, each function takes a section out of the million line txt file. Each function has a for loop that runs through every line in that section of million line file. It takes info from those lines to see if it matches info in 2 other files, one about 50,000-100,000 lines long, the other about 500-1000 lines long. I checked to see if the lines match by running for loops through the other 2 files. Once the info matches I write the output to a new file, all functions write to the same file. The program will produce about 2,500 lines a minute, but will slow down the longer it runs. Also when i run 1 of the functions it does about 500 a minute, but when I do it with 23 other processes it only makes 2500 a minute, why is that? Does anyone know why that would happen? Any way i could import something to make the program run/read through files faster, im already using the 'with '' as file1:' method? Can the multi-processes be redone to run faster? thanks in advance for your input