Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Why TF Serving using one CUDA Compute Stream #2221

Closed
ndeep27 opened this issue May 6, 2024 · 4 comments
Closed

Why TF Serving using one CUDA Compute Stream #2221

ndeep27 opened this issue May 6, 2024 · 4 comments
Assignees
@singhniraj08
Labels
stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response type:support

Comments

@ndeep27
Copy link

ndeep27 commented May 6, 2024

Trying to understand why TF uses one CUDA compute stream? Is there a metric which shows if ops are waiting to be scheduled on that one compute stream? I want to understand if the ops are waiting in high QPS scenarios
@singhniraj08 singhniraj08 self-assigned this May 7, 2024
@singhniraj08
Copy link

@ndeep27,
Looks like this is not an issue from Tensorflow Serving side. This question is better asked on TensorFlow Forum since it is not a bug or feature request. There is also a larger community that reads questions there. Thank you!

@github-actions GitHub Actions
Copy link

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale This label marks the issue/pr stale - to be closed automatically if no activity label May 16, 2024
@github-actions GitHub Actions
Copy link

This issue was closed due to lack of activity after being marked stale for past 7 days.

@google-ml-butler Google-ML-Butler
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@singhniraj08 singhniraj08

Labels
stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response type:support
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@singhniraj08 @ndeep27