+0
Daily
19,005
stars
1,843
forks
123
watchers
288
prs
781
issues
33
releases
README
Mamba
!Mamba
Mamba: Linear-Time Sequence Modeling with Selective State Spaces\
Albert Gu, Tri Dao\
Paper: https://arxiv.org/abs/2312.00752
!Mamba-2
Transformers are SSMs: Generalized Models and Efficient Algorithms\
Through Structured State Space Duality\
Tri Dao, Albert Gu\
Paper: https://arxiv.org/abs/2405.21060
!Mamba-3
Mamba-3: Improved Sequence Modeling using State Space Principles\
Through Structured State Space Duality\
Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J. Zico Kolter, Tri Dao†, Albert Gu†\
Paper: https://arxiv.org/abs/2603.15569
About
Mamba is a new state space model architecture showing promising performance on information-dense data such as language modeling, where previous subquadratic models fall short of Transformers.
It is based on the line of progress on structured state space models,
with an efficient hardware-aware design and implementation in the spirit of FlashAttention.
Installation
Install PyTorch first. By default, mamba-ssm installs the core package without compiling the
selective_scan_cuda extension and without downloading cached CUDA-enabled wheels.
| Install mode | Command | Behavior |
| --- | --- | --- |
| Core package | pip install mamba-ssm --no-build-isolation | Installs mamba-ssm without selective_scan_cuda and does not compile CUDA extensions. |
| Core package plus causal-conv1d | pip install "mamba-ssm[causal-conv1d]" --no-build-isolation | Installs the core package and the causal-conv1d extra, still without selective_scan_cuda. |
| Force local core package build | MAMBA_FORCE_BUILD=TRUE pip install mamba-ssm --no-build-isolation | Builds the default pure Python wheel locally, still without selective_scan_cuda. |
| CUDA selective scan opt-in | MAMBA_KEEP_CUDA_BUILD=TRUE pip install mamba-ssm --no-build-isolation | Installs selective_scan_cuda; pip first tries a matching prebuilt CUDA/HIP wheel, then compiles locally if no wheel is available. |
| Force local CUDA selective scan build | MAMBA_FORCE_BUILD=TRUE MAMBA_KEEP_CUDA_BUILD=TRUE pip install mamba-ssm --no-build-isolation | Skips cached wheels and compiles selective_scan_cuda locally. |
--no-build-isolation is required for CUDA builds so that pip uses your existing CUDA-enabled
PyTorch instead of installing torch-cpu in an isolated build environment.
Source installs use the same defaults and opt-in flags:
| Install mode | Command |
| --- | --- |
| Source default, no selective_scan_cuda | pip install . --no-build-isolation |
| Source default from GitHub, no selective_scan_cuda | pip install git+https://github.com/state-spaces/mamba.git --no-build-isolation |
| Source forced local core build, no selective_scan_cuda | MAMBA_FORCE_BUILD=TRUE pip install . --no-build-isolation |
| Source CUDA selective scan opt-in | MAMBA_KEEP_CUDA_BUILD=TRUE pip install . --no-build-isolation |
| Source forced local CUDA selective scan build | MAMBA_FORCE_BUILD=TRUE MAMBA_KEEP_CUDA_BUILD=TRUE pip install . --no-build-Top issues
#55causal-conv1d installation error49 comments · 2023-12-14#169Import Error41 comments · 2024-02-07#97selective_scan_cuda29 comments · 2024-01-09#7About max token length28 comments · 2023-12-05#556Can't train mamba2 from scratch with HF Trainer26 comments · 2024-09-06#345Error when trying to use Mamba225 comments · 2024-06-03#72loss nan after several epochs25 comments · 2023-12-20#607Installation of `mamba_ssm` Package Fails on Google Colab24 comments · 2024-10-25
Discussions
#1067Pooling and padding for frozen Mamba features with closed-form heads0 comments · 2026-10-25#1051Exploring a new approach to make Mamba even stronger0 comments · 2026-10-01#970AI ChatBots1 comments · 2026-06-11#865The first implementation of Mamba3 in Flax on GitHub (to my knowledge)!0 comments · 2026-03-20#844Routed Attention: Learning when to use O(N) conv vs O(N²) attention (75-99% compute savings)0 comments · 2026-02-07#755What does causal-conv1d do?1 comments · 2025-07-10#616Pytorch geometric temporal mamba installation1 comments · 2024-11-07#553你好方便加个微信吗0 comments · 2024-09-05
Latest release
v2.3.2.post1 v2.3.2.post1 2026-05-09What's Changed Bump version to 2.3.2 by @ko3n1g in https://github.com/state-spaces/mamba/pull/944 Bump to 2.3.2.post1 by @ko3n1g in https://github.com/state-spaces/mamba/pull/945 Full Changelog: https://github.com/state-spaces/mamba/compare/v2.3.2...v2.3.2.post1