This is a rundown of the best NLP, Transformers and LLM papers of 2024. The papers in this video cover a wide range of disciplines - from core deep learning research, novel tokenization schemes, resurrecting RNNs, building on the Mamba architecture, to the latest prompting techniques for efficient retrieval, continual learning, and maximizing code intelligence.
To support the channel financially, and get access to the material used in all channel videos - consider joining our Patreon: www.patreon.com/NeuralBreakdownwithAVB
Links to papers:
ICL: arxiv.org/abs/2412.15563
BLT: arxiv.org/abs/2412.09871
Were RNNs All We Needed? - arxiv.org/abs/2410.01201
xLSTM: Extended Long Short-Term Memory - arxiv.org/abs/2405.04517
The Era of 1-bit LLMs - arxiv.org/abs/2402.17764
LLM2Vec - arxiv.org/abs/2404.05961
Moshi - arxiv.org/abs/2410.00037
Mixtral of Experts - arxiv.org/abs/2401.04088
Transformers are SSMs - arxiv.org/abs/2405.21060
SliceGPT: Compress LLMs - arxiv.org/abs/2401.15024
The AI Scientist - arxiv.org/abs/2408.06292
RAGs vs Long Context LLMs - arxiv.org/abs/2407.16833
CriticGPT - arxiv.org/pdf/2407.00215
DocLLM - arxiv.org/abs/2401.00908
SWE-Agent - arxiv.org/abs/2405.15793
Self-Rewarding Language Models - arxiv.org/abs/2401.10020
DeepSeek Coder - arxiv.org/abs/2401.14196
Videos you may consider watching:
Attention to Transformers playlist: ua-cam.com/play/PLGXWtN1HUjPfq0MSqD5dX8V7Gx5ow4QYW.html
50 concepts to know NLP: ua-cam.com/video/uocYQH0cWTs/v-deo.html
Guide to fine-tuning open source LLMs: ua-cam.com/video/bZcKYiwtw1I/v-deo.html
Generative Language Modeling from scratch: ua-cam.com/video/s3OUzmUDdg8/v-deo.html
Overview of LLM Training Framework (Apple Intelligence Case Study): ua-cam.com/video/Sah0dnu8Hxo/v-deo.html
0:00 - Intro
1:00 - Deep Learning papers
17:00 - Agentic AI papers
#nlp #deeplearning #transformers