New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Celery #41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
- Even supports the setting context manager - Added FORKED_BY_MULTIPROCESSING in hopes of Windows support? - Added TERRA_IS_CELERY_WORKER so I can know if I'm a worker - The main logger is now setup by the Executor, so in the future ProcessPool and celery can support their own logging schemes Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Set terra.zone for tasks universally
- Now executor settings are auto translated from runner to exectutor, instead of from master controller to executor. This clears the path to more consistent behavior when it comes to passing in arguments. - Task args and kwargs that match the same suffix patterns settings so, will now be auto translated from runner to task. This should make for a more seemless transaction to task, as long as the variable names are right. - Executor and compute both have a volume map store in the settings now.
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Auto translate arguments to task functions
Fix resetting zone on exception [skipci]
- Temporary directors have a meaningful suffix - TERRA_KEEP_TEMP_DIR added for debuging - TERRA_CELERY_MAIN_NAME is automatic now - config.json.orig removed, as no longer needed - Fixed a duplicate volume_map bug Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
- Once celery task logs are received, sucess will just be a spam message - The rest of the message are updated to reflect their severity Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Return values in celery get volume translations
create a simple __main__.py
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Celery worker limit
update password mask
Add the -T flag to `docker-compose` which disables its tty allocation [1] (the inverse of docker's -t flag). This prevents the staircase formatting we were observing when printing (not just logging) messages received by the master controler's Listner (which is running in a background thread). Because this container runs in the forground, it can change the tty's settings. This is a problem because it looks like the container disables opost (which postprocess output) in the tty, which screws up the formatting done with onlcr [2]. Also, we don't need a tty because logging to stdout/stderr directly in these processes is not threadsafe. A simple example of this error can be seen with the 'Sending and receiving logging events across a network' example in the python logging cookbook, which sets up a handler on the server to listen for messages and a handler on the client to send them. Start the server with $ python3 server.py & and the client with: $ docker run -i --rm -v /home/sgrichar/terra/terra_dsm/external/terra:/src python:3 python3 /src/client.py which looks as expected; compared to this, which doesn't: $ docker run -it --rm -v /home/sgrichar/terra/terra_dsm/external/terra:/src python:3 python3 /src/client.py [1] moby/moby#8513 [2] https://unix.stackexchange.com/a/242814/332869 [3] https://docs.python.org/3/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network
…ontroller. Currently the process hangs and does not exit once the tasks are done
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
scott-vsi
reviewed
Jun 8, 2020
scott-vsi
reviewed
Jun 8, 2020
scott-vsi
reviewed
Jun 8, 2020
scott-vsi
reviewed
Jun 8, 2020
scott-vsi
reviewed
Jun 8, 2020
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Multithread safe settings
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Signed-off-by: Andy Neff <andy@visionsystemsinc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Finalize working support for celery executor and multiprocess logging
Breaks
terra.settings.terra.zone
can be misrepresented in ThreadPoolExecutor, resulting in log messages appearing to come from the wrong zone.Added
parent
settings.logging.server.hostname and settings.logging.server.port
log messages
Changes
set if the main App CLI is a built-in (aka compiled python module) which is
not typical.
controller and runners zones
for task zones
which run is which easier.
terra.executor.base.BaseExecutor
be called
controller will output the runner's and task's messages (received via TCP)
itself, to stderr.
formatters or "extra" variables, it is a more universal entrypoint for adding
values
Bug Fixes
the config file all of a sudden
variable to be used
prevent a race condition
Testing
handled by Test Cases classes in terra.tests.utils
unconfigured
configures it using an empty dictionary.
function so that all temp files are created in the test dir, preventing /tmp
from being spammed with temp terra log files. Also mocks the TCP server so the
TCP server thread is not actually started.
unit tests.
enabled, but TCP Server is still disabled.