Python Multiprocessing Tutorial: Run Code in Parallel Using the Multiprocessing Module

///Python Multiprocessing Tutorial: Run Code in Parallel Using the Multiprocessing Module

Python Multiprocessing Tutorial: Run Code in Parallel Using the Multiprocessing Module

FavoriteLoadingAdd to favorites

In this video, we will be learning how to use multiprocessing in Python.

This video is sponsored by Brilliant. Go to to sign up for free. Be one of the first 200 people to sign up with this link and get 20% off your premium subscription.

In this Python Programming video, we will be learning how to run code in parallel using the multiprocessing module. We will also look at how to process multiple high-resolution images at the same time using a ProcessPoolExecutor from the concurrent.futures module. Let’s get started…

The code from this video can be found at:

Threading Tutorial –
List Comprehensions Tutorial –
Exception Handling Tutorial –
Pillow (PIL) Tutorial –

✅ Support My Channel Through Patreon:

✅ Become a Channel Member:

✅ One-Time Contribution Through PayPal:

✅ Cryptocurrency Donations:
Bitcoin Wallet – 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3
Ethereum Wallet – 0x151649418616068fB46C3598083817101d3bCD33
Litecoin Wallet – MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot

✅ Corey’s Public Amazon Wishlist

✅ Equipment I Use and Books I Recommend:

▶️ You Can Find Me On:
My Website –
My Second Channel –
Facebook –
Twitter –
Instagram –

#Python

source

By |2020-10-22T10:13:26+00:00October 22nd, 2020|Python Video Tutorials|48 Comments

48 Comments

  1. Corey Schafer October 22, 2020 at 10:13 am - Reply

    Hey everybody! I hope you find this video helpful. I'm finally moved into my new home, so I should be back on a schedule of regular releases very soon. Hope everyone has a great weekend!

  2. Jorge Martínez Zapico October 22, 2020 at 10:13 am - Reply

    Hello, what if I have a subprocess.call which runs a specific command in the cmd. How could I do this using multiprocessing?

  3. Richard Wang October 22, 2020 at 10:13 am - Reply

    multi….do you enjoy drinking it? it's a famous brand of chinese baijiu

  4. Andres Vodopivec October 22, 2020 at 10:13 am - Reply

    Hi Corey. As usual, you make the best tutorial for python. Thanks to you I was able to do multiprocessing for my research. I gave you all the credit on my Github for the multiprocessing code. (https://github.com/ananvodo/Cluster-Analysis-for-molecules). I appreciate a lot for sharing your knowledge.

  5. Anthony Silva October 22, 2020 at 10:13 am - Reply

    you are the man

  6. Akash Chauhan October 22, 2020 at 10:13 am - Reply

    Amazing. Thanks Corey!

  7. Jude Kanjamala October 22, 2020 at 10:13 am - Reply

    Thank you Corey !!!

  8. karthik basavarajappa October 22, 2020 at 10:13 am - Reply

    Would you please make video for Queues, Deques, Joinable queues with multiprocessing?

  9. Ali Pardhan October 22, 2020 at 10:13 am - Reply

    threads are for I/O bound. processes are for CPU bound.

    unfortunately this videos wont help if you have n unique tasks to do. let alone the topic of not copying a 100MB data structure

  10. Fernando PRIETO October 22, 2020 at 10:13 am - Reply

    Dear Corey thank you for this excellent video. It’s a good mid-point between understanding what is behind and not too complicated. I tried to implement it with a function returning a dataframe and I need to consolidate the results but it is very slow… An advice?

  11. Applied statistics October 22, 2020 at 10:13 am - Reply

    So cool ! !

  12. João Caldeira October 22, 2020 at 10:13 am - Reply

    I do have to thank you @Corey Schafer your work in these videos. I learn a lot from them and in the end I use them on my work to automate some tasks. Some love from Portugal

  13. varshini ram October 22, 2020 at 10:13 am - Reply

    I'm using a mac but I still get the freeze_support() error. But it got resolved when I used the if _name_ == _main_ . Even in the python documentation, it is stated that the error is for windows only.

  14. Shweta Suhas Joshi October 22, 2020 at 10:13 am - Reply

    How can you pass multiple arguments to the do_something() functions?

  15. YEM October 22, 2020 at 10:13 am - Reply

    Is concurrent.futures better than multiprocessing.Pool().map ?

  16. Deep Grewal October 22, 2020 at 10:13 am - Reply

    You are the best teacher I have ever had. I wish I could give this video a million likes.

  17. raghav October 22, 2020 at 10:13 am - Reply

    I am not in a position right now to donate through patreon. All I can do is watch your videos without skipping ads and give it a like.

  18. Daniel A October 22, 2020 at 10:13 am - Reply

    yea… this doesn't work. When printing (f1.result()) you get a brokenProcessPool exception

  19. Trump News October 22, 2020 at 10:13 am - Reply

    my p1 and p2 are loops on an imported .py file so i keeps getting stuck on p1 loop.

  20. 琳家生活Livingwithlinlin October 22, 2020 at 10:13 am - Reply

    Has anyone run the code in jupyter notebook? I ran both code snippets (using multiprocessing.Process and concurrent.futures) in my jupyter notebook and did not get the print out from do_something(). I ran the same code snippets in google collaborator and got the proper print out. My local/jupyter notebook python version is 3.7.6 and google collaborator python version is 3.6.9. Any input or suggestion is appreciated.

  21. Poncho Salazar October 22, 2020 at 10:13 am - Reply

    This video is a jewel

  22. Schlumpfpirat October 22, 2020 at 10:13 am - Reply

    What I like about threading and multiprocessing is how easy it is to use. I can simply fire a single task that is irrelevant to the rest of my execution, but would otherwise block it.
    What bothers me about concurrent.futures is that – aside from being much more complex in code – it's much more blown-up and seems to be geared towards actually having a multi-processing task, without having the ability to simply and simply fire a single instance.

  23. Kritan Bhandari October 22, 2020 at 10:13 am - Reply

    When loop through the processes list and join a process ( process.join() ) aren't we still joining each process once at a time?

  24. Adam Akmal October 22, 2020 at 10:13 am - Reply

    What is your sublime text theme use??? I like the color

  25. Azizan Haniff Adenan October 22, 2020 at 10:13 am - Reply

    My first time watching a video from you. It's very good and truly helpful. Thank you for your hard work.

  26. Dipayan Maji October 22, 2020 at 10:13 am - Reply

    a question.. can we have a setup in multiprocessing so that, if i am searching simultaneously in parallel then after first hit all other process supposed to be stopped?

  27. Cary Hutchinson October 22, 2020 at 10:13 am - Reply

    I am getting an error with this code. File "C:UsersxxxxxxxxxxxPythonPython37libconcurrentfuturesprocess.py", line 102, in _python_exit thread_wakeup.wakeup() Windows 10 Python 3.7. It executes fine until the end then this error.

  28. Afrid Mondal October 22, 2020 at 10:13 am - Reply

    Thanks Corey…. Amazing video

  29. Hend abd el-aziz October 22, 2020 at 10:13 am - Reply

    Really thanks a lot
    For your tutorials, so helpful
    But can you please make a series about gpu using numba and Pytorch

  30. Gokulnath October 22, 2020 at 10:13 am - Reply

    Can we use multi processing to open several videos at the same time and run inference on a ml model?

  31. Jarbait October 22, 2020 at 10:13 am - Reply

    Hi, when I execute the code on Windows (using Spyder and the if name==main statement), the code is completed within a second and the statements did not print. Is this a known Spyder issue or is there something else going on?

    Thank you for your help in advance!

  32. John Paul Spinelli October 22, 2020 at 10:13 am - Reply

    Great video! If you have a sec, could you look at my issue? It sort of pertains to this video: https://stackoverflow.com/questions/63378389/scope-issue-with-false-statement thanks!

  33. Zhao Xu October 22, 2020 at 10:13 am - Reply

    1000x better than the official manual

  34. shawnmccori October 22, 2020 at 10:13 am - Reply

    Here is how I made parallel processing during my minecraft engine days in 1998. I decided to make it a packet and used modem code to produce it and the read it as a packet to preload information. The most elaborate way to make parallel processing to work in code. But JBuilder I could find no other way. Windows kicks modem and server info to second processor any way.

  35. Bhanuday Sharma October 22, 2020 at 10:13 am - Reply

    at 13.11 What does this multiprocessing.process() returns? I mean, what is the value of 'p'? Is it like some Process ID?

  36. Bhanuday Sharma October 22, 2020 at 10:13 am - Reply

    Man… You have explained so good…. Fall in love with your teaching style.

  37. dillmon1 October 22, 2020 at 10:13 am - Reply

    Sir, this video is a fucking treasure. It Saved me a bunch of time reading shitty tutorials on google.

  38. John Tamplin October 22, 2020 at 10:13 am - Reply

    When faced with a python challenge, I ask myself "What would Corey Schafer do?" I then, somewhat ironically, ask myself "Why am I wasting my time asking myself rhetorical questions, when I could be watching one of his outstanding videos."

  39. John, the Revelator October 22, 2020 at 10:13 am - Reply

    If you're on Windows 10 and using IDLE, you will need to start the IDLE through the Command Prompt using 'py -m idlelib'. When you run the code, on IDLE, it will correctly output to the Command Prompt, while you will not see any output of the Process on IDLE.

  40. nihat nihat October 22, 2020 at 10:13 am - Reply

    how can I pass multiple attributes in the submit method of ProcessPoolExecutor?

  41. Gokulnath October 22, 2020 at 10:13 am - Reply

    Then when ayncio is used?Could you put a video on that?

  42. H.venkatesh developer October 22, 2020 at 10:13 am - Reply

    Great content brother, thank you .

  43. Apoqlite October 22, 2020 at 10:13 am - Reply

    Unrelated Question but do you know what theme you are using for sublime?

  44. john valton October 22, 2020 at 10:13 am - Reply

    Just started using Arc's and Rwlocks in Rust

  45. Sai Praneeth October 22, 2020 at 10:13 am - Reply

    The print statements in the do_something program are not printing out ?? But the program is giving the time taken at the end correct

  46. English Rain October 22, 2020 at 10:13 am - Reply

    God bless you Corey!!!

  47. Marcelo October 22, 2020 at 10:13 am - Reply

    Great tutorials! Definitely deserves a contribution

Leave A Comment

*