Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mamba or Jamba models #61

Open
DewEfresh opened this issue Apr 11, 2024 · 1 comment
Open

Mamba or Jamba models #61

DewEfresh opened this issue Apr 11, 2024 · 1 comment

Comments

@DewEfresh
Copy link

Has any work been done with state space models. I'd be curious how they would perform with this framework applied.

@kwuking
Copy link
Collaborator

kwuking commented Apr 16, 2024

Clearly, this is a very interesting question. The use of state-space models as a base for large models has recently become a very popular topic. For example, papers like "Is Mamba Effective for Time Series Forecasting?" and "TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting" have made numerous attempts. In fact, we have also been studying the use of state-space models for time series forecasting for a long time and have followed the precursor to the Mamba model, the S4 model. However, we have not achieved good results in time series forecasting, which is why we decided to "let the bullets fly" for a while. We also look forward to seeing breakthroughs in time series forecasting using SSM. Finally, I would like to mention that as a general reprogramming framework, we are obviously also capable of adapting Mamba or Jamba and look forward to more like-minded friends joining us to contribute code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants