[UPDATED] ViViT & NaViT papers: How Sora encoded space-time patches | Shawn's ML Notes

Update on April 8th, 2023:
- Fixed missing narration on slide 25
- Added explanation for accuracy increase from upsampling (thanks to @ryuku4966!)
- Amplified audio track
Original video (archived): • [ARCHIVED] ViViT & NaV...
--
Thank you for checking out my video notes on ViViT & NaViT papers: how Sora encoded space-time patches! I would love to share my ML learning journey with you.
Paper information:
- Arnab, Anurag, et al. "Vivit: A video vision transformer." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
- Dehghani, Mostafa, et al. "Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution." Advances in Neural Information Processing Systems 36 (2024).
Please let me know in the comment section regarding any questions, points of discussion, or anything you would like see next. See you in the next video!

[@xiaojinyusaudiobookswebnov4951]

I learned a lot from your videos. Please keep them coming. They are worth all the time and effort it takes to produce them.

[@yuxiangwang9624]

Thank you, will do!

[@matin2021]

Hi
I am very happy that I was able to find your channel on UA-cam
I hope you will make more videos about computer vision
keep Going ✌

[@yuxiangwang9624]

Thank you for your support!

[@abhranilchandra2775]

This is such a great explanation, do you plan to cover the "DiT: Scalable Diffusion Models with Transformers" paper sometime soon?
Thanks a lot for such wonderful and insightful explanations...

[@yuxiangwang9624]

Thank you for the kind words! That's a good idea and let me look into it. :)

[@chriso8285]

Great voice. For fun, audition for a voice actor gig. Would look great on resume. Or on a date or at a conference. Lol

[@yuxiangwang9624]

Lol thanks for the compliment!