Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Must we use a Queue? #112

Open
ughstudios opened this issue Jul 31, 2022 · 3 comments
Open

Must we use a Queue? #112

ughstudios opened this issue Jul 31, 2022 · 3 comments
Labels

Comments

@ughstudios
Copy link

ughstudios commented Jul 31, 2022

Is the only way to share stuff between subprocesses is to use the Queue object? Can we use any other types? It seems a bit overkill to use a queue if you only have two string values for example...

The example I have and want is this:

  1. Two processes
  2. Both of which use an infinite loop and do some work
  3. The infinite loop is tied to a synchronized boolean that can be changed from the main application
  4. When the main application sets the bool to false, the processes gracefully quit (the infinite loop ends)
  5. The other string values which are passed to the sub processes are absolute filepaths to the location of where they saved their work.
  6. The main application can now obtain the two filepaths (they are synchronoized/passed to the processes)
  7. Use those paths to build my final data file
@ughstudios
Copy link
Author

https://github.com/uqfoundation/multiprocess/blob/master/py3.10/doc/sharedctypes.html

The example here:

Array('c', 'hello world', lock=lock)

Results in:


Exception has occurred: RemoteError

---------------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\pbssu\Desktop\LeagueTools\LeagueHelper\venv\lib\site-packages\multiprocess\managers.py", line 212, in handle_request
    result = func(c, *args, **kwds)
  File "c:\Users\pbssu\Desktop\LeagueTools\LeagueHelper\venv\lib\site-packages\multiprocess\managers.py", line 379, in create
    obj = callable(*args, **kwds)
  File "c:\Users\pbssu\Desktop\LeagueTools\LeagueHelper\venv\lib\site-packages\multiprocess\managers.py", line 1016, in Array
    return array.array(typecode, sequence)
TypeError: cannot use a str to initialize an array with typecode 'c'

Could you update the example?

@ughstudios
Copy link
Author

image

PyLint also does not like this package

@mmckerns
Copy link
Member

So, no, there are a number of different ways, the most common is Pool. I'm not sure what example you are referring to with Array, as the doc you refer to only has an interface specification on it and no examples. Here's a doc for multiprocessing that has a ton of information and examples on sharing objects between processes: https://docs.python.org/3/library/multiprocessing.html. If you replace multiprocessing with multiprocess, all of the examples apply.

Also, can you be more specific with regard to pylint? If you post some code/commands that I can run to reproduce what you are seeing, then I might be able to help you with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants