Add to favorites
In this video, we will be learning how to use multiprocessing in Python.
This video is sponsored by Brilliant. Go to to sign up for free. Be one of the first 200 people to sign up with this link and get 20% off your premium subscription.
In this Python Programming video, we will be learning how to run code in parallel using the multiprocessing module. We will also look at how to process multiple high-resolution images at the same time using a ProcessPoolExecutor from the concurrent.futures module. Let’s get started…
The code from this video can be found at:
Threading Tutorial –
List Comprehensions Tutorial –
Exception Handling Tutorial –
Pillow (PIL) Tutorial –
✅ Support My Channel Through Patreon:
✅ Become a Channel Member:
✅ One-Time Contribution Through PayPal:
✅ Cryptocurrency Donations:
Bitcoin Wallet – 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3
Ethereum Wallet – 0x151649418616068fB46C3598083817101d3bCD33
Litecoin Wallet – MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot
✅ Corey’s Public Amazon Wishlist
✅ Equipment I Use and Books I Recommend:
▶️ You Can Find Me On:
My Website –
My Second Channel –
Facebook –
Twitter –
Instagram –
#Python
source
Hey everybody! I hope you find this video helpful. I'm finally moved into my new home, so I should be back on a schedule of regular releases very soon. Hope everyone has a great weekend!
Hello, what if I have a subprocess.call which runs a specific command in the cmd. How could I do this using multiprocessing?
multi….do you enjoy drinking it? it's a famous brand of chinese baijiu
Hi Corey. As usual, you make the best tutorial for python. Thanks to you I was able to do multiprocessing for my research. I gave you all the credit on my Github for the multiprocessing code. (https://github.com/ananvodo/Cluster-Analysis-for-molecules). I appreciate a lot for sharing your knowledge.
you are the man
Amazing. Thanks Corey!
Thank you Corey !!!
Would you please make video for Queues, Deques, Joinable queues with multiprocessing?
threads are for I/O bound. processes are for CPU bound.
unfortunately this videos wont help if you have n unique tasks to do. let alone the topic of not copying a 100MB data structure
Dear Corey thank you for this excellent video. It’s a good mid-point between understanding what is behind and not too complicated. I tried to implement it with a function returning a dataframe and I need to consolidate the results but it is very slow… An advice?
So cool ! !
I do have to thank you @Corey Schafer your work in these videos. I learn a lot from them and in the end I use them on my work to automate some tasks. Some love from Portugal
I'm using a mac but I still get the freeze_support() error. But it got resolved when I used the if _name_ == _main_ . Even in the python documentation, it is stated that the error is for windows only.
How can you pass multiple arguments to the do_something() functions?
Is concurrent.futures better than multiprocessing.Pool().map ?
You are the best teacher I have ever had. I wish I could give this video a million likes.
I am not in a position right now to donate through patreon. All I can do is watch your videos without skipping ads and give it a like.
yea… this doesn't work. When printing (f1.result()) you get a brokenProcessPool exception
my p1 and p2 are loops on an imported .py file so i keeps getting stuck on p1 loop.
Has anyone run the code in jupyter notebook? I ran both code snippets (using multiprocessing.Process and concurrent.futures) in my jupyter notebook and did not get the print out from do_something(). I ran the same code snippets in google collaborator and got the proper print out. My local/jupyter notebook python version is 3.7.6 and google collaborator python version is 3.6.9. Any input or suggestion is appreciated.
This video is a jewel
What I like about threading and multiprocessing is how easy it is to use. I can simply fire a single task that is irrelevant to the rest of my execution, but would otherwise block it.
What bothers me about concurrent.futures is that – aside from being much more complex in code – it's much more blown-up and seems to be geared towards actually having a multi-processing task, without having the ability to simply and simply fire a single instance.
When loop through the processes list and join a process ( process.join() ) aren't we still joining each process once at a time?
What is your sublime text theme use??? I like the color
My first time watching a video from you. It's very good and truly helpful. Thank you for your hard work.
a question.. can we have a setup in multiprocessing so that, if i am searching simultaneously in parallel then after first hit all other process supposed to be stopped?
anybody? https://stackoverflow.com/q/63720847/14214029
I am getting an error with this code. File "C:UsersxxxxxxxxxxxPythonPython37libconcurrentfuturesprocess.py", line 102, in _python_exit thread_wakeup.wakeup() Windows 10 Python 3.7. It executes fine until the end then this error.
Thanks Corey…. Amazing video
Really thanks a lot
For your tutorials, so helpful
But can you please make a series about gpu using numba and Pytorch
Can we use multi processing to open several videos at the same time and run inference on a ml model?
Hi, when I execute the code on Windows (using Spyder and the if name==main statement), the code is completed within a second and the statements did not print. Is this a known Spyder issue or is there something else going on?
Thank you for your help in advance!
Great video! If you have a sec, could you look at my issue? It sort of pertains to this video: https://stackoverflow.com/questions/63378389/scope-issue-with-false-statement thanks!
1000x better than the official manual
Here is how I made parallel processing during my minecraft engine days in 1998. I decided to make it a packet and used modem code to produce it and the read it as a packet to preload information. The most elaborate way to make parallel processing to work in code. But JBuilder I could find no other way. Windows kicks modem and server info to second processor any way.
at 13.11 What does this multiprocessing.process() returns? I mean, what is the value of 'p'? Is it like some Process ID?
Man… You have explained so good…. Fall in love with your teaching style.
Sir, this video is a fucking treasure. It Saved me a bunch of time reading shitty tutorials on google.
When faced with a python challenge, I ask myself "What would Corey Schafer do?" I then, somewhat ironically, ask myself "Why am I wasting my time asking myself rhetorical questions, when I could be watching one of his outstanding videos."
If you're on Windows 10 and using IDLE, you will need to start the IDLE through the Command Prompt using 'py -m idlelib'. When you run the code, on IDLE, it will correctly output to the Command Prompt, while you will not see any output of the Process on IDLE.
how can I pass multiple attributes in the submit method of ProcessPoolExecutor?
Then when ayncio is used?Could you put a video on that?
Great content brother, thank you .
Unrelated Question but do you know what theme you are using for sublime?
Just started using Arc's and Rwlocks in Rust
The print statements in the do_something program are not printing out ?? But the program is giving the time taken at the end correct
God bless you Corey!!!
Great tutorials! Definitely deserves a contribution