Відео

very interesting of him to call deep learning a trade :)

good video but his and the camera placements are subobtimal

I’m based in Milan, any chances any events will take place in Milan as well? I working on computational environment for agents. Already in production to some degree.

Great work guys! ❤🎉🎉

I just wish he hadn't stood right in front of what he was trying to show people, but I love his passion for explaining what he's talking about.

Tarea 6

i notice that Yasmin Moslem is the only person without a photo, is being photographed for public display against her religious beliefs, does she also not appear in public in person, or is she simply a virtual mentor who doesn't exist in the real world?

what these systems don't do i suppose is recognize text from images/scans which translators work a lot with so accurate text extraction from source is often required before the actual translation could be performed unless this software is able to not only extract text but also correct all the errors which are usually associated with OCR, and the more exotic the writing system the more challenging is the extraction task

ChatGPT gave me link to this video)

Strange thing, he mention "attention" term before explaining what it is. What was EXACT meaning of this Query Key Value magic ??? I suspect speakers just copy thoughts of another people mechanically, not understaning real meaning of operations !

Thank you !

Great describon. Love from Pakistan 🇵🇰

does embedding gets trained or key or query or value gets trained i am confused. please help

the video image is too poor, you need to fix it more

where can i find the implementation

...comes from Google - Check. ...TensorFlow T-shirt - Check. Most viewers therefore rate this lecture highly - Check. This is very hand-wavy throughout with relatively no rigor shown. There are many lectures/presentations online which actually explain the nuts and bolts and wider use cases of Attention mechanisms. Maybe the title of this video should be something else, like "Our group's success with one use case (language translation) of Attention." Frankly, the drive-by treatment of the technical details of language translation case was almost terrible and should have probably been omitted.

Tokio

7y30 8

Gg v cejtqnhmmt i io

Thanks buddy

Fantastic talk!

I love how excited he is.

where is the library he talks about to get the details of training the DL "right"?

The passion that he transmits is priceless

Great content in this video. Would love if you had the multi-headed devil covered! Though, great video to get the overall view..

Hey! I just found your channel and subscribed, love what you're doing! I appreciate how clear and detailed your explanations are as well as the depth of knowledge you have surrounding the topic! Since I run a tech education channel as well, I love to see fellow Content Creators sharing, educating, and inspiring a large global audience. I wish you the best of luck on your UA-cam Journey, can't wait to see you succeed! Your content really stands out and you've put so much thought into your videos! Cheers, happy holidays, and keep up the great work!

Very nice 😍💋 💝💖♥️❤️

Love you 💋💋😘😘❤️💯

Hi nice project. If that’s okay can you share the GitHub link?

Most I gather from this talk is that "attention" is a pretty terrible term. Something like "fuzzy lookup" or "matching" or "mapping" would have been much more descriptive, but oh well, which researcher needs to think about terminology before unleashing it on the world.

Transformers learned translation without language dependency O_o

math majors/ graduate math students skip to 15:36

in two months i am starting a farm here in nepal, looks useful for me aswell how can i know more about smartcrop.

They should invent a device that can always tell the time of day when the user wants.

47:55 "We tried it on images it didn't work so well". 2020, Visual Transformer: am I a joke to you?

Interessante, grazie

great explanation !

Starting @ 15:45, in well under 2 minutes, attention explained! Only a true master can do it. Love.

Thanks for sharing! It'd be great if the video could pay more attention to the slides though.

Best expaination!!

This is the best explanation of attention I have seen so far! And I have been looking :)

Great talk. Something very useful.

Is there any way i could get the ppts for reference?

Hey can you please send the model to me for this project

nice

Great session Pi School 📚and Łukasz 😀. Here are a few key concepts I "attended" to and found interesting: "There is always another head that whatever word it is in, it looks at the (head) noun of the sentence or the (head) verb, just wants to know what are we talking about here 🤔" - Łukasz Kaiser ua-cam.com/video/rBCqOTEfxvg/v-deo.html "n^2 * d seems worse than n * d^2 (on attention vs recurrent algorithmic/operation complexity). Luckily at Google there is guy name Noam (Shazeer), he never got his bachelor but wrote most of these papers..." - Łukasz Kaiser ua-cam.com/video/rBCqOTEfxvg/v-deo.html "We trained a model to translate from English to French and vice versa, and from English to German and vice versa. Then if you give it French and ask for a German translation it will do a reasonable job. Multitask helps with deep learning tasks where you have little data." - Łukasz Kaiser ua-cam.com/video/rBCqOTEfxvg/v-deo.html

Can i see the code?

Your book on Go is the best

I have a Synthetic Text Generator working on simple Ngram propagation technique that handles "long term memory" very crudely by keeping a Python set of items already present in the SynTexExtension. Where 'items set' represents the Topic....globally rare Tokens that are 'featured' to appear much more frequently in the Extension than in the Training Data....thus simulating a Topic and also a semblance of "long term coherency". A Bag-of-Words excluding the Nmost frequent Tokens (maybe N=500 or =1000). In practice: everything except Stopwords and HiFreqs.... A Parasitic Context is obtained by choosing a random stretch of EuroParl Training Text, then extracting a Bag-of-Words as described above... Then STG generates Extensions favoring that Context... Works OK for short passages....a paragraph or two... but to 'progress' a narrative the Context needs to be a Moving Window with new favorites gradually replacing tired old ones... The CONVERSE problem is handled in a similar way. When a few Topicals repeat TOO OFTEN. The Extensions often end up OVER-featuring items so that their proliferation detracts from plausibility... I'm only training on 10 million Tokens of EuroParl English, so "Topic" is already naturally limited to English Eurocratese anyway... so STG typically harps on "the Commission", "the European Parliament", "the rapporteur"...

Slides?