Skip to content

Issues: NVIDIA/Megatron-LM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

307 Open 230 Closed
307 Open 230 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Fonte facilitada em fractal 2030
#843 opened May 28, 2024 by felipeliliti
[BUG]
#842 opened May 28, 2024 by felipeliliti
Configuring datasets using train-data-path, valid-data-path, and test-data-path results in training errors
#841 opened May 27, 2024 by Eisenhower
[BUG] GroupedMLP calculation problem.
#839 opened May 27, 2024 by Baibaifan
[BUG] Can't continue training from GPT-345M checkpoint with TransformerEngine - RuntimeError: Error(s) in loading state_dict for ParallelTransformer
#838 opened May 24, 2024 by arktoswb
1
[QUESTION] Is FP32 supported in MultiNode Training
#837 opened May 24, 2024 by JiwenJ
2
[BUG] The problems with bucket and shared_embedding.
#835 opened May 23, 2024 by Baibaifan
2
[BUG] Checkpoint saving is slow for zarr backend + distributed optimizer
#834 opened May 22, 2024 by chotzen
4
[QUESTION] Why enable non_blocking=True when doing synchronous D2H?
#833 opened May 22, 2024 by raywan-110
1
[QUESTION] How to Obtain Computation Model Graphs in Megatron-LM?
#832 opened May 19, 2024 by fwyc0573
[BUG] Modify FLOPs in MFU calculation for casual mask when using FlashAttention.
#831 opened May 17, 2024 by Yuxin-CV
Question with forward_backward_pipelining_without_interleaving in Megatron-LM Pipeline
#830 opened May 17, 2024 by Hongjie1Chu
[QUESTION] Why not use tensor parallel APIs of pytorch
#829 opened May 16, 2024 by GuWei007
1
[QUESTION] how to profile bubble time in pipeline parallelism?
#828 opened May 15, 2024 by starstream
1
[BUG]
#827 opened May 14, 2024 by chrisgao7
[BUG] The argument --no-position-embedding should be fixed
#826 opened May 14, 2024 by Hoonly
[BUG]:there is a small chance that it will get stuck, If i repeat runing test_serialization.py many times,
#825 opened May 14, 2024 by starkhu
Does Megatron has plan to support llama pre-train？
#824 opened May 13, 2024 by wen020
Projeto liliti stk 3.6.9 inteligência artificial 🤖
#822 opened May 11, 2024 by felipeliliti
Projeto liliti stk 3.6.9 inteligência artificial
#821 opened May 11, 2024 by felipeliliti
Megatron-LM for LLaMa3
#818 opened May 10, 2024 by SDsly
7
[QUESTION] How does tensor_parallel coop with q/k_layernorm
#816 opened May 10, 2024 by cryoco
1
[BUG] Typo in drop_policy options in moe_utils.py
#815 opened May 9, 2024 by Malikeh97
2
[QUESTION] Why is expert parallelism not supported during fp16 training?
#810 opened May 7, 2024 by yutian-mt
1
[core dataset compilation error]
#807 opened May 6, 2024 by shamanez
ProTip! Follow long discussions with comments:>50.