How to train a model to generate image embeddings from scratch

Embeddings are one of the fundamental building blocks behind Large Language Models.
I built a simple model to generate image embeddings. This video will help you understand embeddings from first principles. I don’t use transformers or anything fancy. Instead, I build a simple Siamese Network step by step, and train it using contrastive loss.
Link to the code in the video: github.com/svpino/contrastive...
I teach a live, interactive program that'll help you build production-ready Machine Learning systems from the ground up. Check it out here:
www.ml.school
To keep up with my content:
• Twitter/X: / svpino
• LinkedIn: / svpino
🔔 Subscribe for more stories: / @underfitted

[@emrahe468]

I had been working on a similar problem for a few weeks and had already implemented most of the code you mentioned (after many trial and errors) . But after watching your video, I realized that I had missed a few crucial details like the dense layer and the loss function. Your clear instructions and fantastic tutorial really saved me tons of of time. I wish you had released this video earlier, but regardless, thank you very much! 🙏

[@underfitted]

Thank you!

[@LuisAlvarado-hm3br]

Great, insightful video with an original approach to explaining embeddings. Most explanations focus on text, so it's refreshing to see image embeddings for a change. It's also fantastic to see such an influential paper used as a reference for the implementation. Thank you!

[@sachinmohanty4577]

Beautiful explanation ❤ loved the tutorial 😊

[@kalinduSekara]

Clear and great explanation 💯

[@ThetaPhiPsi]

Contrastive explained nicely! It's a shame nobody uses it.
I've some improvements to add:
1. you can use the model itself to compare pairs and take the loss to discriminate results (but the embedding is fine too for a class of downstream tasks)
2. you can further take ROC AUC and optimize your threshold on the given training data (I used sigmoid to squish the loss between 0 and 1)
Works nicely!

[@mgreek31]

cool explanation, i always wondered how embeddings worked at the lower level

[@user-ez6ti9vh6q]

I sincerely would like to see how you'd go about it using 3d images while implementing triplet loss

[@toddroloff93]

Great video. I like your enthusiasm, and passion you display in your videos. The way you break things down and explain it is great. Thank you

[@underfitted]

Thanks

[@ian-haggerty]

[@Aclodius]

You're doing the Lord's work

[@raheemnasirudeen6394]

A great explanation

[@KoenYskout]

I experimented with modifying the embedding size to 2, and visualize that on a 2d plot (colored by label). Easy to see how all (or most) numbers with the same label are clustered together by the embedding, and numbers with a different label are moved apart.

[@yaseromar1539]

What a magnificent explanation, every time I watch one of your videos I feel enjoyment and excitement and I can see the same in your way of talking about machine learning 🤩🤩🤩🤩🤩🤩🤩🤩🤩🤩

[@underfitted]

Thanks!

[@dcrasto]

Thanks!

[@underfitted]

Thanks!

[@user-yw9us2qo6g]

amazing

[@gemini_537]

Gemini 1.5 Pro: This video is about creating image embeddings from scratch using a neural network.
The speaker starts by explaining what embeddings are and why they are important. Embeddings are a way of representing data points as vectors in a high-dimensional space. Similar data points will have similar embeddings, while dissimilar data points will have dissimilar embeddings. This makes embeddings useful for tasks such as finding similar documents or images.
The speaker then introduces the concept of a Siamese network. A Siamese network is a type of neural network that takes two inputs and outputs a measure of similarity between the inputs. The speaker explains how to use a Siamese network to train a model to generate image embeddings.
The speaker then shows how to train the model on a dataset of handwritten digits. The model learns to generate embeddings for the digits such that similar digits (e.g., two different images of the digit 3) have similar embeddings, while dissimilar digits (e.g., an image of 3 and an image of 7) have dissimilar embeddings.
Finally, the speaker shows how to use the trained model to generate embeddings for new images. The speaker concludes by discussing some of the applications of image embeddings.

[@user-ez6ti9vh6q]

@Underfitted , Thank you for this amazing video. How would you ideally do the same using 3d images?

[@chuanana]

Thank you for the video! Is it expected to have the distance of image embeddings of different labels (3 vs. 7) to be greater than 1? I got (1.0468788, 1.087123). Since we normalized the inputs, I had expected the embedding distance to be normalized as well. Is there an expected range for the distance?

[@thevoyager7675]

Thanks for the nice explanation!
Could we use these image embeddings for classification tasks? if so, how?

[@underfitted]

You could. You can create 10 template embeddings, representing each digit. To classify a new image, compare it to all 10 embeddings and select the closest one.

[@KoenYskout]

I would say: transform the input into its embedding, and classify based on the embedding coordinates. I guess a simple KNN classifier will already do well, because similar numbers are moved closer together, and different numbers further apart, in the embedding.

[@user-wm8xr4bz3b]

Thanks for the video! so am i right to say that the process is the supervised learning?

[@underfitted]

This one is supervised, yes

[@it_is_random]

9000+ power

[@ian-haggerty]

Funny, it wasn't too long ago that MNIST wasn't a "toy" problem. The history of computer vision is rather short. Are we writing the beginning of it?

[@underfitted]

Probably

[@ddemmkkimm]

1:51 Image is not 2D data. It is # of pixels dimensional data, i. e. width x height.

[@underfitted]

I meant you need 2 dimensions to represent one image: 1 dimension to represent height and 1 to represent width.