Python Tutorial – 29. Sharing Data Between Processes Using Queue

///Python Tutorial – 29. Sharing Data Between Processes Using Queue

Python Tutorial – 29. Sharing Data Between Processes Using Queue

FavoriteLoadingAdd to favorites

The topics incorporated in the video are how to share data between processes using multiprocessing queue in python with appropriate examples, what is queue, how to create a queue, method of queue and classes of queue, how to communicate between two process in python, how to use “queue” to store results of square function, how to create queue variable in Multiprocessing Queue class, what is queue class method put(), what is Queue data structure FIFO, what is the difference between multiprocessing queue and queue module and the use of queue class method empty() as well as get().

Topics that are covered in this Python Video:
0:03 How to share data between processes using multiprocessing queue in python
0:15 Example
1:40 How to communicate between two processes in python
2:06 Use “queue” to store results of the square function
2:17 create queue variable in Multiprocessing Queue class
2:55 Queue class method put()
2:58 What is Queue data structure FIFO(First In First Out)
3:30 Queue class method empty(), get()
5:11 Difference between Multiprocessing queue vs queue module

Code used in this tutorial:

Next Video:
Python Tutorial – 30. Multiprocessing Lock:

Website:
Facebook:
Twitter:
Patreon:

source

By |2020-03-17T02:24:14+00:00March 17th, 2020|Python Video Tutorials|30 Comments

30 Comments

  1. nicholas rogers March 17, 2020 at 2:24 am - Reply

    What if I'm not trying to process to a queue, but rather from a queue. I want the arguments my process sends to the function to be gotten from my queue

  2. ALI M March 17, 2020 at 2:24 am - Reply

    Not getting result in Visual Studio

  3. Maple Foong March 17, 2020 at 2:24 am - Reply

    Hi, very good video, would like to know about data sharing using Manager. Hope you can make a video on that… for some reason I I have to use manager.list() to share data between processes, but it seems the data cannot shared to a function that inside a class…

  4. Sahil Chachere March 17, 2020 at 2:24 am - Reply

    fabulous.

  5. Adnan Qureshi March 17, 2020 at 2:24 am - Reply

    Thank you so much this helped me a lot.

  6. Cristian Florescu March 17, 2020 at 2:24 am - Reply

    can not run multiprocessing in pycharm, but in IDLE it's works just fine.
    PermissionError: [WinError 5] Access is denied ( I already ckecked to run it as administrator )

  7. maximiliano ogaz March 17, 2020 at 2:24 am - Reply

    ( n * n )… that is a fat cat covering eyes with its paws

  8. Asala Mahajna March 17, 2020 at 2:24 am - Reply

    Thank you so much for the clear illustration! 🙂
    Here is my question:
    I am running a simulation using multiprocessing, and if I want to extract the results, output, from one simulation run (process) and have it input for the other process, how can I do that?

  9. Parzival The Knight March 17, 2020 at 2:24 am - Reply

    When you also add the memory ref in print statement: f'outside process: {result}. Memory ref: {id(result)}', then in your example `result` has the same memory address inside of the method and in the __main__, meaning they are pointing to the same thing. So I do not understand how they are not sharing the memory if they are pointing to the same variable in memory?

  10. ahmed sayari March 17, 2020 at 2:24 am - Reply

    Thanks so much

  11. MeditationMan March 17, 2020 at 2:24 am - Reply

    Sorry that i cant say this on all your vids but. They are fucking good and get the message right. Very very understandable and one of the only ones that is on this level. Keep posting haha

  12. Siraj Khan March 17, 2020 at 2:24 am - Reply

    At time 1:36, if executing in python 3.6 i am getting result outside the child process as well

  13. Richard March 17, 2020 at 2:24 am - Reply

    Thank you for all of your excellent videos. Much appreciated. Quick question (but maybe long answer): I wrote a program and made an interface with PyQt5. While the program is running, the interface is not responsive, until the program finishes. Would the solution be multiprocessing?

  14. ebramful March 17, 2020 at 2:24 am - Reply

    Great work!

  15. Jorge L March 17, 2020 at 2:24 am - Reply

    you say is shared memory, but isn't queue a pipe?

  16. Zoltar_mmo March 17, 2020 at 2:24 am - Reply

    If I put time.sleep(1) in my process, nothing gets printed for several seconds, then the numbers seem to be printed almost simultaneously. So it seems like the program is essentially waiting until the process finishes before going on to the while statement. I'm confused about what exactly is going on. I thought the whole point was to process the numbers and print them as they come out.

  17. sabarish amaravadi March 17, 2020 at 2:24 am - Reply

    can we use any one of those sharing methods to share images(with opencv) between two processes?

  18. Yuan Gao March 17, 2020 at 2:24 am - Reply

    bro, is there a way to make Queue first in first out? Stack?

  19. Solana Foo March 17, 2020 at 2:24 am - Reply

    Thanks so much for all your multiprocessing tutorials!

  20. Lukaz Nejakej March 17, 2020 at 2:24 am - Reply

    Excellent tutorial. Thank you very much!

  21. Ayush Bajirao March 17, 2020 at 2:24 am - Reply

    Is it possible to initiate a variable inside a function(process) and then give it back to main, or do we have to initialize it in main only?

  22. xtremeHackerzPro March 17, 2020 at 2:24 am - Reply

    while not q.empty() is more pythonic

  23. Nitesh Rijal March 17, 2020 at 2:24 am - Reply

    while not q.empty() should be easier to read, nice tutorial

  24. jean paul yong chia March 17, 2020 at 2:24 am - Reply

    Hi I am trying to share a complex dictionary nested object carrying inputs into all my processes. how can you address that.

  25. Anurag Bisht March 17, 2020 at 2:24 am - Reply

    Nice tutorial thanks for it !
    On a side note, as a substitute to "if queue.empty(): is False" could be "if not queue.empty(): "

  26. Akhilesh Gotmare March 17, 2020 at 2:24 am - Reply

    Thanks for the tutorials! They are very well designed and moduled. I had a question on this one – Is there any specific advantage of using shared queues instead of shared arrays?

  27. Simon Bouchard March 17, 2020 at 2:24 am - Reply

    use .join() to join string :p

  28. dio din March 17, 2020 at 2:24 am - Reply

    Not using q.join()?

  29. sekmani med amine March 17, 2020 at 2:24 am - Reply

    sir queue and multiprocessing work in idle 2 or exactly python 2.7.9 because multithreading work fine and i don't know about the multiprocessing if work or not

Leave A Comment

*