MAMBA and State Space Models explained | SSM explained
Вставка
- Опубліковано 7 чер 2024
- We simply explain and illustrate Mamba, State Space Models (SSMs) and Selective SSMs.
SSMs match performance of transformers, but are faster and more memory-efficient than them. This is crucial for long sequences!
AI Coffee Break Merch! 🛍️ aicoffeebreak.creator-spring.... Celebrating our merch launch, here is a limited time offer! 👉 Get 25% discount on AI Coffee Break Merch with the code MAMBABEAN.
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael
Outline:
00:00 Mamba to replace Transformers!?
02:04 State Space Models (SSMs) - high level
03:09 State Space Models (SSMs) - more detail
05:45 Discretization step in SSMs
08:14 SSMs are fast! Here is why.
09:55 SSM training: Convolution trick
12:01 Selective SSMs
15:44 MAMBA Architecture
17:57 Mamba results
20:15 Building on Mamba
21:00 Do RNNs have a comeback?
21:42 AICoffeeBreak Merch
📄 Gu, Albert, and Tri Dao. "Mamba: Linear-time sequence modeling with selective state spaces." arXiv preprint arXiv:2312.00752 (2023). arxiv.org/abs/2312.00752
📄 MoE-Mamba arxiv.org/abs/2401.04081
📄 Vision Mamba arxiv.org/abs/2401.09417
📄 MambaByte arxiv.org/abs/2401.13660
🕊️ Mamba rejected from ICLR: / 1750526956452577486
📖 Prefix sum (Scan) with Cuda: developer.nvidia.com/gpugems/...
📺 Transformer explained: • The Transformer explai...
Great resources to learn about Mamba:
📙 Mamba: jameschen.io/jekyll/update/20...
📕 The Annotated S4: srush.github.io/annotated-s4/
📘 Mamba The Easy Way: jackcook.com/2024/02/23/mamba...
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: / aicoffeebreak
Ko-fi: ko-fi.com/aicoffeebreak
Join this channel to get access to perks:
/ @aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: / aicoffeebreak
Twitter: / aicoffeebreak
Reddit: / aicoffeebreak
UA-cam: / aicoffeebreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research
Scientific advising by Mara Popescu
Video editing: Nils Trost
Music 🎵 : Sunny Days - Anno Domini Beats - Наука та технологія
I've a question. Given that SSMs are entirely linear, how do they conform with universal approximation theorem? I mean a lack of non-linear activation should imply that they should be particularly bad at approximate functions, but they are not.
Am i missing something?
Also really loved the video!
Exactly, SSMs are linear. 2:05 This is why they are part of larger neural nets, that bring in the nonlinearities needed to fulfil the universal approximation theorem.
You can think of SSMs like of attention in Transformers: they are just a module (the module that aggregate information from the whole sequence, such that the current token is informed of its context).
Great question, thanks!
Thank you!
One last question. Since the output is realized rather through a convolution than a recurrent function, do we use Backprop through time for updating A B and C? Or just a normal backprop
@@drummatick You ask so many great questions! (I'll pin your original comment, as I think others might have the same question)
As the state of token t depends on the state of token t-1, we still need to do backpropagation through time (otherwise we would not know what to backpropagate at t-1 if we did not resolve t first). But because of the linearity, backprop through time is reportedly more stable for Mamba than for e.g., LSTMs.
@@AICoffeeBreak thank you! Makes sense. That might also explain why mamba doesn’t suffer from
Vanishing gradient problem as the gradients propagated vanish linearly? Correct me if I’m wrong
@@drummatick Yes, no activation function to squash signals to zero.
I have to give a presentation on Mamba next week and I've been waiting for this video so I could finally learn what the hell I need to talk about
Don't feel dumb. The first time I heard about transformers used for LLMs, I thought it was a new movie prequel. Oh, well, I guess I'm dumber. 😂
Us
Thanks! Looking forward to a Hyena video :)
Thank you so much! It seems like you paid for the coffee I'm drinking right now!
only way I managed to understand up to 40% about how mamba is implemented was watching this video
Give these things some time to crystallize. Your understanding will grow even bigger with time. Also, better explainer resources will keep coming.
I understand 10% of that stuff, but the presentation is lovely.
Thank you so much!
This is exactly the level of detail I needed right now. Thank you so much!
Thanks! This was also the level I was looking for when trying to understand Mamba. So I decided to explain what I understood.
I can't wait for neural networks that are tokenless and takes bits directly. Where words are learned through images and sound rather than their own modality. And richer input with simultaneous multimodal vectors.
Did you see MambaByte? 20:45
But exactly, I agree with you. This would be tremendous for both multilinguality and multimodality. Tokenizers are very tedious.
nitpick: with FlashAttention, transformers have linear memory requirements, but the compute complexity is still quadratic
Really cool video! Thank youuuu
Thanks, nice to see you here, Dominik, aahm I mean Outlier!
i barely grasped the transformers and now they have this new thing.....
well you had 6 years to look into transformers, so..
Don't feel bad. NLP and Linguistic Models had been around for decades. It really takes a long time because processing power was so expensive. I did NLP AI demos in year 2000 and they told me that the tech was too advanced for its time and they didn't know what to do with it! 😅
A big thanks for a comprehensive explanation of the Mamba Architecture & computations, @AICoffeeBreak!
Thank you for your visit and for leaving this wonderful comment!
@AICoffeBreak, thank you for the awesome video. Very small pet peeve which had me re-check all the math. At 11:20 it would make the explanation much easier to understand if you kept x 0 indexed as that is the notation you were going for since the beginning. Also, maybe making it explicit that you're taking t = L, although this is kind of obvious. This was an awesome lecture, thank you again.
Hats off to you for this amazing video! Best explanation of Mamba I have seen.
Great video. Speaking of VMamba.
For some reasons it seems people in medical images field are more excited for mamba than in LLM.
So many, they are competing between themselves. "Swin-UMamba demonstrates superior performance with a large margin compared to CNNs, ViTs, and latest Mamba-based models. Notably, on AbdomenMRI, Encoscopy, and Microscopy datasets, Swin-UMamba outperforms its closest counterpart U-Mamba by an average score of 3.58%".
Like if you pick a random mamba paper from arxiv (there are dozens of them already), it probably will be related to medical image segmentation.
Yes, this was exactly my impression when I talked to some friends working on medical imaging yesterday! 🤗 Unfortunately, I couldn't mention this insight in the video as it was already almost done with editing.
Thanks for the MAMBA video!
I always appreciate your insight on these new, influential papers! Your thoughts always pair well with a good cup of coffee. 😁☕️
Thank you so much!
I was waiting for exactly this topic! Thanks so much!
Thank you for the shoutout to me repo!
I later realized it was an application of a known idea "heisen sequence", which is a pretty cool way to do certain associative scan operations via cumsum
Thanks for the video! I’ve been confused between RNN and MAMBA for several days😂
Happy to help!
Excellent summary! Note BTW that the B discretization has been simplified to B * \Delta, because it made little difference (see mamba-minimal comments).
Thank you for the great Mamba explanation
Thank You for your visit and for leaving this comment!
Great.
There are a lot of failed explanation or completely wrong approach about SSM and Mamba on the internet, but finally I found the exact what I want.
Thank you for the video.
Thank you!
Nice video, good overview, which is what I was searching for
Glad you like it!
Awesome video! I especially like the simple explanation and the visuals.
That was our aim, thanks!
Nice T-shirt! So excited to listen about new models!
this was such a well summarized explanation, thank you! ❤
Glad you like it!!
thanks a lot for this (love your sweater)
Thank You for your kind words!
Amazing explanation! Please create more content!
Awesome explanation Letitia!! ☕️
Thank you, Tim!
I knew you would have covered this hot topic! I bet now you are working on RWKV ! They are discovering so many alternative architectures these days!
am I? 😆
My favourite kind of animation of Ms Coffee Bean is on 19:47
🥂
Amazing !!
Neat summary! Thank you ^^
Thanks Hannes!
The ASMR at the end is nice. I always sit and wait for it. 🙂
😂I should start a channel reading ML papers in ASMR style.
OK, Miss Coffee bean, you really made my head spin this time! Keep up the good work. We love you!
I wish I could find a TensorFlow implementation with all the optimizations! I need to bite the bullet and switch to Torch soon.
Good luck with switching your faith.
Move asap bro. Pytorch > TensorFlow
The pytorch version has a bunch of specialized CUDA kernels so I hope you know CUDA too
this explanation was excellent. Thank you very much :)
I am in general happy that we are trying to leave transformers behind. RWKV and SSMs look promising. Transformers are inelegant, and by their very formulation a resignation from looking for better methods.
Thank you so much! Super helpful.
great video! thanks
my pleasure
Celebrating our merch launch, here is a limited time offer!
👉 Get 25% discount on AI Coffee Break Merch with the code MAMBABEAN: 🛍 aicoffeebreak.creator-spring.com/
Isn't it cannibalism when Coffee Bean drinks coffee?
Have you checked the cup? 🤔 It's hot water. Beans drinking hot water is how coffee is made! ☕
Looking forward to see a video of vision mamba to know the points I missed while reviewing other content related to it
Can tou share some info about the material you used to learn mamba?
Sure! You can find most of these things in the video description.
1. Foremost, the paper and its references
2. 📕The Annotated S4: srush.github.io/annotated-s4/
I also found this after publishing the video: jameschen.io/jekyll/update/2024/02/12/mamba.html
Yannic Kilcher has already done great video about Mamba and I wholeheartedly recommend it! My only problem with it, is that he sticks too much to the paper and the Mamba paper (while being well written), is scattering the architecture all over the paper. It also assumes prior knowledge about previous papers, such that I honestly did not understand how it all hangs together and what Mamba actually is and how tokens flow through it.
@@AICoffeeBreak wow, those are great. Thanks!
Incredible work. Thank you !
I hope I'll remember everything, so I can present this paper tomorrow in the reading group. 😅
min 11:30 x = (x_0,...x_{L-1}) Amazing video
Yea that was good
Awesome video.
So for NLP we only have discrete inputs, hence we don’t need delta, right?
And then we also cannot learn different deltas for different layers to be resolution independent as we could for continuous signals? No?
Thank you! About your question: We always need the delta for discrete outputs. The delta-less formulation is for writing the differential equation which assume continuous transitions with infinitesimal steps. When we go to matrices and vectors and we are doing numerical approximations (all of deep learning), we need to discretize and choose delta steps.
@@AICoffeeBreak thanks for your answer and your patience Ms Coffee Bean. What I am stuck with is, that we use delta to discretize continuous inputs, but in the NLP applications of Mamba we already have discrete inputs, the tokens. What would we then apply delta too?
Also, it would be conceivable that we could learn to fuse together for example every three consecutive tokens using delta, but then delta would be discrete (1, 2, 3, 4 tokens per window) and could not be differentiated/learned?
@@mkamp Thanks for the follow-up, now I better understand the confusion. It all has to do with terminology and what word mean in different ML contexts.
I think you talk about **the inputs**, whether they are discrete/continuous. Language is discrete and symbolic, but with our neural networks, we pretend like it would be continuous : we map words into tokens and tokens into vectors that can take continuous values.
What I am talking about with SSMs is about the continuous / discrete nature of **the model**, in the way it updates the hidden state h. When we model sequences and want to compute hidden states for tokens in sequences, we are jumping in large (discrete) steps from token to token (word to word, as in "The big fox sat on the mat ...").
* In contrast, a differential equation is continuous: how h changes in time (h dot at 06:13) is an infinitesimal small step from the last state of h. So we are modelling the example sinusoid curve with the smallest step imaginable (the assumption of calculus that steps can be infinitely small - think of the integral vs. the discrete sum). I hope you can see that we cannot jump continuously from "The" to "big", as there is not much semantics in between those words (there are some semantic steps in between, but not infinitely many small steps. I mean, there is "cat", there is "woman", we have a "catwoman", a "woman cat" but not much else). So we cannot use differential equations to model jumps from tokens to tokens, so we use SSMs instead: 👇
* The SSM formulation is the discrete version of the continuous equation: we are modelling that sinus curve with larger steps (not infinitesimal steps), given by delta.
Which means that we are jumping from token to token in our language / sequence applications. One h is "The", the next h has to model "big", the next "fox" ... So we are jumping in large, discrete steps from word to word (token to token). This is why we cannot use differential equations, but discretized versions of them: SSMs.
Thanks for the answer and for taking the time!@@AICoffeeBreak
@@mkamp Videos are these static, uneditable things (in YT). I could have made this point much clearer in the video if I would have known about your question beforehand. And of course, it is all also limited by how much time I can invest into doing one video.
I'd love to see your take on Hasani, Lechner, et al's Closed-form Continuous Time Neural Nets, Liquid Structural State-Space Models, Forward Invariance of Neural ODEs, ... and maybe compare with MAMBA SSSM
Looks more like a RNN model
It is a linear RNN with a lot of Schnickschnack around it. 😅
@@AICoffeeBreakto use the scientific terminology
@@pooroldnostradamus 🤣🤣🤣
I'm not entirely sure on how SSM differ from RNNs, especially regarding how attention is being used. Theres still the bottleneck of h_t to h_{t+1} between time steps, which was one of the motivations for the attention layer -- so that information in one part of the sequence doesn't have to be squeezed before computation with information from another part of the sequence.
Is the main innovation from RNN to SSM the fixed delta, A,B,C formulation such that the training can be done in parallel for all time steps?
SSMs are linear RNNs. But linear is not powerful enough (universal approx. theorem). 2:05 This is why they are part of larger neural nets, that bring in the nonlinearities needed to fulfil the universal approximation theorem. You can think of SSMs like of attention in Transformers: they are just a module (the module that aggregate information from the whole sequence, such that the current token is informed of its context).
But unlike attention, this information aggregation module works token after token, while attention has access to all tokens at once. So once a token is wrongly forgotten, it is gone.
Therefore, the state of research for now cannot really leave attention alone, because it is so useful for retrieval and finding needles in a haystack: arxiv.org/abs/2402.04248
or arxiv.org/abs/2402.19427
Thanks for the amazing question.
And yes, coming to the last question: SSMs are linear RNNs which makes them computable in parallel at training time, exactly.
Thanks for the great explanation! I suppose things will become more clear when I try to work through the math myself.
When I first read about SSM, I saw a big connection between it and kalman filtering. But now I see the connection is probably just inspirational
Wasn't your voice enough? now I have a crush on you!!!
Does matrix A change during inference or only during training?
Only during training. :)
Good job as far as possible, but I fear this topic may require more like a semester course than a single video.
As a side note, I find it striking that mathematics is at the heart of the new (and miraculous) world of AI. If this rate of math penetration continues, kids will soon need to go directly from third grade to graduate school.
Thank you a lot for your explain! may i know which software you using for making animation in your video? thanks!
My editor uses Adobe Premiere Pro for video editing (this is also when Ms. Coffee Bean comes in).
I use PowerPoint for all other visualisations.
@@AICoffeeBreak really beautiful!
I did not understand why do we need to perform a discretization step? Which are the advantages ? I understood what happens from math point of view, But why we need to change from Real numbers to discrete numbers ? Is it something related to Backpropagation ?
Great question, and I do not know exactly the answer myself. I assume you could do without it and hope for SGD to find the right parameter settings. But I assume the theoretical soundness helps to stabilize training. I would be curious to find out more about this.
@@AICoffeeBreakisn't it because if we have continuous function then we have to solve ODEs with proper solvers, but if we discretize it and apply the Euler method then we get recurrence that can be optimized with a backprop?
Exactly. That's a great way to put it with the right background knowledge.
pls do xLSTM too
It's already on my todo-list! 😅
so whats next? kalman filter with learned dynamic?
The explanation of the SSM sure sounds similar to a Kalman filter to me!
You are absolutely right, sir. That's because Kalman Filter is indeed a State Space Model
You are absolutely right, sir. That's because Kalman Filter is indeed a State Space Model
解释得很好！爱来自中国💓
how can i buy your tshirt?
On the AICoffeeBreak merch store. 🤭 aicoffeebreak.creator-spring.com/
@@AICoffeeBreak Oh I see. At the time I asked I hadn't watched to the end. 🤣
Am I the only one that finds mamba much harder to grasp than transformers?
Let's see: what was the first architecture you learned? Was it transformers or LSTMs?
merch lol
Mamba jamba samma
This spawns music in my head. 🎶😅
14:06 so they reinvented memoization and gave it a different fancy name?
Hey Ai Coffe Break create video on cited papers from Gemini 1.5 Pro paper 😮❤
Kindly make a tutorial to fine tune an open source LLM model on many pdfs data. The fine tuned LLM must be able to answer the questions from the pdfs accurately.
its kinda obv that you dont rlly understand this shit yet good enough to explain it better. Its like trying to reverse analyzing a circuit board instead of going from the intended use case, to the technical rules to the abstraction to the design to the analogies. You just explained me what a mosfet is for and how it works inside of a circuit board... yeah well nice. This isnt a test. Its a lesson. Try to explain it in human language, feel into it. THEN you can down to why all that stuff achieves that and which analogies could be used for it.
This video right now just shows one thing... you are smart. But it doesnt show me that you understood teaching.
Hi skullteria, with my videos I always have to make the difficult decision of what will be the target audience of the video. I have a general guideline that I set myself when I started this channel, which was to broadly aim the videos at computer science students somewhere in their bachelors. With a topic like mamba, I had to gloss over many complicated topics (for some of which I have already made some more in-depth explanations) to not make the video over an hour and still get into the level of detail that many of my regular viewers expect. You will find in the comments many people for whom this was exactly the right balance. But this of course means that there will also be people who will not find the video super useful, and I'm very sorry that you're one of them. If there's a specific topic for which you'd like a more high-level, more intuitive explanation, absolutely put it in the comments and I'll add it to my to-do list :)
আমার কী পছোনদো তোকে বোলবো কেনোরে