mat1 and mat2 shapes cannot be multiplied (1000x768 and 4096x256)

[PandaDalin]

RuntimeError: mat1 and mat2 shapes cannot be multiplied (1000x768 and 4096x256)

[PandaDalin]

未修改llm_dim的维数

[kwuking]

具体是哪里报错呢，可以有有详细的运行日志嘛

[PandaDalin]

具体是哪里报错呢，可以有有详细的运行日志嘛

是因为大模型的底座没修改，llama的最大长度是768，参考主函数的llm_dim维度参数

[kwuking]

好的 所以现在还有问题嘛？

[1767773043]

运行时错误：mat1 和 mat2 形状无法相乘（1000x768 和 4096x256），
请问应该怎么改呢

[1767773043]

0it [00:00, ?it/s]
Traceback (most recent call last):
File "/home/gf-shu/wsb/Time-LLM-main/run_main.py", line 260, in
outputs = model(batch_x, batch_x_mark, dec_inp, batch_y_mark)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1852, in forward
loss = self.module(*inputs, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/gf-shu/wsb/Time-LLM-main/models/TimeLLM.py", line 197, in forward
dec_out = self.forecast(x_enc, x_mark_enc, x_dec, x_mark_dec)
File "/home/gf-shu/wsb/Time-LLM-main/models/TimeLLM.py", line 242, in forecast
enc_out = self.reprogramming_layer(enc_out, source_embeddings, source_embeddings)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/gf-shu/wsb/Time-LLM-main/models/TimeLLM.py", line 287, in forward
source_embedding = self.key_projection(source_embedding).view(S, H, -1)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/gf-shu/anaconda3/envs/time_llm/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 116, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1000x768 and 4096x1024)

Process finished with exit code 1