Steering vectors: tailor LLMs without training. Part I: Theory (Interpretability Series)

State-of-the-art foundation models are often seen as black boxes: we send a prompt in and we get out our - often useful - answer. But what happens inside the system as the prompt gets processed remains a bit of a mystery & our ability to control or steer the processing into specific directions is limited.
Enter steering vectors!
By computing a vector that represents a particular feature or concept, we can use this to steer the model to include any property in the output we want: add more love into the answers, ensure it always answers your prompts (even if harmful!), or make the model such that it cannot stop talking about the Golden Gate Bridge. In this video we discuss how to compute such steering vectors, what makes it such simple steering possible (somehow the network's hidden representations decompose into simple-ish linear structures), and look at a couple of examples. In Part II ( • Steering vectors: tail... ) we code up our steering vectors.
Disclaimer: finding these steering vectors is an active area of research; right now making it work includes a lot of trial-and-error and clarity on when steering works vs when it's not possible to find a useful direction remains unclear. Work on sparse autoencoders (a current hot topic in interpretability research) aims to automate the finding of useful directions.
Further reading & references I used:
Activation addition: arxiv.org/abs/...
Refusal directions: www.alignmentf... and huggingface.co...
Golden Gate Claude: www.anthropic....
Superposition: transformer-ci...
Sparse autoencoders: arxiv.org/pdf/...

[@TarunGupta360]

Very helpful video! Please keep the good work coming :)

[@GAURAVKAUL84]

Wonderful explanation Anastasia!

[@anastasiaborovykh120]

Thank youuuu :)

[@swairshah]

Oh wow. Great to have non-slop ML channel like this. I think steering vectors, SAEs some of other MechInt papers would make a good series. I'd also like to know why something like KSVD isn't used (these days its faster too?) instead of SAEs.

[@anastasiaborovykh120]

oh interesting! i wasn't aware of KSVD, but i think it could be valuable in this setup. will look into it more & get back to you.

[@RahulKumar-m1j2q]

better w/out music

[@anastasiaborovykh120]

Thanks for the feedback!