Multithread Processing with Python

I did fair amount of multithreading programming with C# in the past but never tried it with Python. Let’s imagine a function that takes 3 seconds. You want to execute it 3 times. The total execution time should be 9 seconds like the following.

import threading
import time


class MyThread(threading.Thread):
    def run():
        # Long running process
        time.sleep(3)
        print('Done')


if __name__ == '__main__':
    start_time = time.time()
    MyThread.run()
    MyThread.run()
    MyThread.run()
    elapsed_time = time.time() - start_time
    print(f"Took {elapsed_time}")

Output:

Done
Done
Done
Took 9.006911993026733

If you can run the functions in 3 different threads at the same time, you could save time. Here is the multithread sample in Python.

import threading
from multiprocessing import Pool
import time


class MyThread(threading.Thread):
    def run():
        time.sleep(3)
        print('hoge')


if __name__ == '__main__':
    start_time = time.time()
    pool = Pool()
    pool.apply_async(MyThread.run)
    pool.apply_async(MyThread.run)
    pool.apply_async(MyThread.run)
    pool.close()
    pool.join()
    elapsed_time = time.time() - start_time
    print(f"Took {elapsed_time}")

Here is the output:

hoge
hoge
hoge
Took 3.179738998413086

It should take 9 seconds if those processes run sequentially but it only took 3.17 seconds. It’s because the 3 threads run at the same time like the following image.

It’s not too difficult to do multithreading in Python. There is one more thing I am thinking about. What if you need to run 100 processes but you want to limit the number of threads to 5 at a time? I will write about it after this blog article.

Author: admin

A software engineer in greater Seattle area

Leave a Reply

Your email address will not be published. Required fields are marked *