[DOCS] Update the document of HETERO pipeline parallelism #24470
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WeldonWangwang
requested review from
peterchen-intel,
wangleis,
songbell and
yangwang201911
github-actions
bot
added
category: docs
zhaixuejun1993
approved these changes
May 13, 2024
...cles_en/openvino-workflow/running-inference/inference-devices-and-modes/hetero-execution.rst
Show resolved
Hide resolved
peterchen-intel
approved these changes
May 14, 2024
kblaszczak-intel
approved these changes
May 15, 2024
songbell
reviewed
May 16, 2024
| Pipeline parallelism | ||
| ------------------------ | ||
|
|
||
| The pipeline parallelism is set via ``ov::hint::model_distribution_policy``, This mode is an efficient technique to infer large models on multiple devices. The model is split into multiple stages, and each stage is assigned to a different device (``dGPU``, ``iGPU``, ``CPU``, etc.). This mode assign operations to different devices as reasonably as possible, ensuring that different stages can be executed in sequence and minimizing the amount of data transfer between different devices. |
There was a problem hiding this comment.
Suggested change
| The pipeline parallelism is set via ``ov::hint::model_distribution_policy``, This mode is an efficient technique to infer large models on multiple devices. The model is split into multiple stages, and each stage is assigned to a different device (``dGPU``, ``iGPU``, ``CPU``, etc.). This mode assign operations to different devices as reasonably as possible, ensuring that different stages can be executed in sequence and minimizing the amount of data transfer between different devices. | |
| The pipeline parallelism is set via ``ov::hint::model_distribution_policy``. This mode is an efficient technique to infer large models on multiple devices. The model is split into multiple stages, and each stage is assigned to a different device (``dGPU``, ``iGPU``, ``CPU``, etc.). This mode assign operations to different devices as reasonably as possible, ensuring that different stages can be executed in sequence and minimizing the amount of data transfer between different devices. |
There was a problem hiding this comment.
Updated, thanks.
| The pipeline parallelism is set via ``ov::hint::model_distribution_policy``, This mode is an efficient technique to infer large models on multiple devices. The model is split into multiple stages, and each stage is assigned to a different device (``dGPU``, ``iGPU``, ``CPU``, etc.). This mode assign operations to different devices as reasonably as possible, ensuring that different stages can be executed in sequence and minimizing the amount of data transfer between different devices. | ||
|
|
||
| For large models which don’t fit on a single first priority device, model pipeline parallelism is employed where certain parts of the model are placed on different devices to ensure that the device has enough memory to infer these operations, and assign other operations to next priority device. | ||
|
|
There was a problem hiding this comment.
maybe can re-organize this part? kind of confusing of different devices, and next priority device
There was a problem hiding this comment.
Yes, the description that may cause confusion has been removed.
github-actions
bot
removed
the
category: docs_snippets
songbell
approved these changes
May 17, 2024
wangleis
approved these changes
May 17, 2024
WeldonWangwang
added a commit
to WeldonWangwang/openvino
that referenced
this pull request
May 23, 2024
…envinotoolkit#24470)" This reverts commit 03aad66.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details:
Tickets: