Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add log when worker is "rebooted" with --max-tasks-per-child or --max-memory-per-child #8916

Open
4 tasks done
laurent-brisbois opened this issue Mar 19, 2024 · 0 comments
Open
4 tasks done

Comments

@laurent-brisbois
Copy link

laurent-brisbois commented Mar 19, 2024

Checklist

  • I have checked the issues list
    for similar or identical feature requests.
  • I have checked the pull requests list
    for existing proposed implementations of this feature.
  • I have checked the commit log
    to find out if the same feature was already implemented in the
    main branch.
  • I have included all related issues and possible duplicate issues
    in this issue (If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

  • None

Possible Duplicates

  • None

Brief Summary

When I started using the parameters --max-tasks-per-child and --max-memory-per-child, I first tried with --max-tasks-per-child=3 to see if it was working.
I was starring at the logs and I had no clue if it was working or not, except by checking the INFO/ForkPoolWorker-... worker number to see it was changing every 3 tasks. It's ok for three but could be annoying to check with greater limit.

I didn't test --max-memory-per-child yet to see if it was in the logs but I guess not.

Would it be possible to add a log for that ?

Something like :

  • logger.info("Stopped worker-<number_of_the_worker>: reached max tasks limit
  • logger.info("Stopped worker-<number_of_the_worker>: reached max memory limit

Small note/context :
I'm currently using celery==5.2.7 and django-celery-beat==2.5.0 and I want to to start using these options because once every a few days, I have to restart my application as my celery workers seem to reach some kind of memory limit. And I'd like to have these worker events (drop worker,...) in my logs.

I'm open to make the contribution myself if I can find the place where such check is done (in billiard apparently).

Design

Architectural Considerations

None

Proposed Behavior

Something like :

  • logger.info("Stopped worker-<number_of_the_worker>: reached max tasks limit
  • logger.info("Stopped worker-<number_of_the_worker>: reached max memory limit

Proposed UI/UX

N/A

Diagrams

N/A

Alternatives

None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant