https://www.solosoft.dev/tags/state-space-model/Recent content in State Space Model on SoloSoftHugoen-usFri, 01 May 2026 00:00:00 +0000https://www.solosoft.dev/post/causal-conv1d-cuda-2026/Fri, 01 May 2026 00:00:00 +0000https://www.solosoft.dev/post/causal-conv1d-cuda-2026/<p>The Transformer architecture has dominated deep learning for years, but a new challenger has emerged: state space models (SSMs). At the heart of one of the most influential SSM architectures, <strong>Mamba</strong>, lies a surprisingly modest CUDA kernel library called <strong>Causal-Conv1d</strong>. Developed by Tri Dao (known for FlashAttention) and Albert Gu (the creator of Mamba), this library provides the computational backbone for the causal depthwise 1D convolutions that make Mamba&rsquo;s selective state space mechanism possible.</p> <p>Causal-Conv1d is not a flashy project with a web UI or chat interface. It is infrastructure &ndash; the kind of low-level optimization that makes new architectures feasible. Its purpose is singular: compute causal 1D convolutions as fast as humanly possible on NVIDIA GPUs, providing a PyTorch-compatible interface that can be dropped into any model implementation.</p>