Don’t Embed Wrong!

Поділитися
Вставка
  • Опубліковано 25 лис 2024

КОМЕНТАРІ • 79

  • @madytyoo
    @madytyoo 25 днів тому +22

    This is the best channel about Ollama!

  • @aurielklasovsky1435
    @aurielklasovsky1435 6 днів тому

    Wow. I am very surprised that this actually works. It is so bizarre that this technology (LLMs) actually performs better when you tell it what it should be doing. Thank you for the tip!
    And thank you for telling me not to use llama for embeddings. I absolutely thought that it works better because it is bigger without ever testing anything else. Cheers!

  • @rundeks
    @rundeks 24 дні тому +2

    I never heard of this before. Thank you so much for sharing it!

  • @conneyk
    @conneyk 23 дні тому +3

    Thanks for the video!
    I‘m working on my own RAGs for some time now. Maybe prefixing would help.
    What I learned so far is, that RAG is very individual for each use case. Like if you are dealing with code Docs or with large Texts or with multi Line PDFs. Also if your docs aren’t in english embedding models like nomic or other open source are really weak. You first have to translate the docs before embedding them. Than we even haven’t talked about reranking queries, corrective rag to enhance your query with web search results or other docs, hybrid query search based on metadata and docs content, and so on. Also the vector store you using is playing a difference.
    All this is making it very complex to implement all the combinations and benchmark and test them.
    I really would love to find some RAG KISS Principles and best practices

    • @aurielklasovsky1435
      @aurielklasovsky1435 6 днів тому

      I am running into the same problem working with none English documents. I think there needs to be a ton of investment done for each language, there is probably no real way around it

  • @ahasani2008
    @ahasani2008 25 днів тому

    can't help to notice your Batik shirt, nice one. And the content is excellent as always Matt, thanks

  • @raymond_luxury_yacht
    @raymond_luxury_yacht 22 дні тому

    When I created embeddings before I sent to the embedder I got llm to analyse the text and add 10 questions about it added it to the chunk and sent it. Search accuracy was very good

  • @karlfranz2pl
    @karlfranz2pl 25 днів тому +2

    I havent used any embed models but a while ago I tried to give PDF to llama 3.1 7b and results were between nothing to horrible. Then I tied the same document with llama 3.1 70b and results were actually pretty good. I could not test it really in depth because my PC runs 70b model at almost negative speed :) (please keep in mind I actually don't know what I am doing with thees LLMs :) )

  • @deucebigs9860
    @deucebigs9860 24 дні тому

    Liking and subscribed to tell you you're definitely on the right path of what I want to learn!

  • @Ad434443
    @Ad434443 16 днів тому

    Interesting! I didn't know how big difference the embedding models meant. I have used nomic for RAG. And maybe i have been a bit sloppy, as prefixes I just cursively read about before and didn't implement them in the RAGs. If you have time, Matt, then a video on how to use n8n to do the prefixing ingestion of external docs. I used tagging for filtering and so on, but prefixes does seem very powerful.

  • @Leo_ai75
    @Leo_ai75 23 дні тому +1

    I've just added this to my project folder in an .md file, for an embedding project I was working on, for my Coding assistant to use as context:
    Embedding Best Practices from Ollama Founding Team:
    When doing embeddings with Ollama models, you were doing it wrong until now. Adding prefixes to content can make results twice as accurate.
    Three of the five embedding models in official Ollama Library support prefixes:
    1. Nomic embed-text:
    - Documents: "search_document:"
    - Queries: "search_query:"
    - Classification: "classification:"
    - Clustering: "clustering:"
    2. Snowflake & Arctic:
    - Queries: "represent the sentence for searching relevant passages:"
    - Documents: No prefix needed
    3. Mixed Bread:
    - Same as Snowflake/Arctic format
    Implementation:
    - For vector stores: Add prefix before chunk of text
    - For similarity search: Add prefix before query
    - For hosted Nomic API: Use API option
    - For Ollama: Simply prepend prefix text
    Testing shows prefixes deliver:
    - More complete answers
    - Better document matching
    - 2x accuracy improvement in many cases
    This comes directly from Matt Williams, founding Ollama team member.
    I hope it helps!

    • @technovangelist
      @technovangelist  22 дні тому

      I can’t see why you would want to do that

    • @Leo_ai75
      @Leo_ai75 22 дні тому

      @@technovangelist it’s for people who use coding assistants Matt, like copilot, Cursor, Cody etc.

    • @technovangelist
      @technovangelist  22 дні тому

      But that’s not something that goes into a prompt. It doesn’t make any sense

    • @Leo_ai75
      @Leo_ai75 22 дні тому

      @@technovangelist What I meant was I used it as an .md file in my project folder for the AI assistant to use as context. My apologies, I realise I said system prompt previously, that was in error.

    • @technovangelist
      @technovangelist  22 дні тому

      I still don’t get that. That’s how you need to interact with an embedding model. It’s not something a model would benefit knowing.

  • @AlekseyRubtsov
    @AlekseyRubtsov 25 днів тому +1

    Thanks!

  • @blackswann9555
    @blackswann9555 21 день тому

    Thank you for sharing ❤

  • @proterotype
    @proterotype 25 днів тому

    So awesome man. I really appreciate this kind of information

  • @paulomtts
    @paulomtts 25 днів тому

    Right on time, I'm just implementing a RAG pipeline!

  • @serikazero128
    @serikazero128 25 днів тому

    @10:49 could've been the perfect time for "Stop, Get some help!" meme :)

  • @kasomoru6
    @kasomoru6 23 дні тому +1

    Don't know why, but I feel like I just watched a really convincing A.I.. Great info though.👍

  • @wnicora
    @wnicora 22 дні тому

    This video opens new perspectives on Rag, tx
    Could you share links to articles explaining the design and use of prefixes?

  • @fabriai
    @fabriai 25 днів тому +1

    Excellent stuff, Matt. Thanks for this! Why do you prefer typescript for coding the test over python? Do you run it in node? Have you tried dejó for these tasks?

    • @technovangelist
      @technovangelist  23 дні тому +1

      It doesn’t have all the installation baggage that comes with Python. Python is so brittle and easy to screw up your setup. I use deno to run it usually. I don’t know what dejo is.

    • @fabriai
      @fabriai 21 день тому

      @@technovangelist Thanks for the answer. It makes sense. Sorry for "dejó", I intended to write "deno" but the Spanish autocorrect in the phone changed it and I didn't notice until your reply.
      I'm more a nodist by trade than a pythonist, so Deno comes more naturally to me. Is good to know an expert like you uses Typescript on Deno. Will follow you lread.
      Thank you so much!

  • @sebastianpodesta
    @sebastianpodesta 25 днів тому +2

    Thanks a lot!!! Great stuff! Quick question, what would you recomend for multilingual data, what happens if the rag data and the user prompts are in Spanish, should I do all system prompts and instructions in Spanish? Or tell it to translate the just answer?

    • @sebastianpodesta
      @sebastianpodesta 25 днів тому

      I’m trying to do RAG on n8n using ollama with llama 3.1 chat model and nomic embedding model with mixed results, I get answers some times in English, others in Spanish and some times the model tells me that it didn’t understand the question

  • @basterman13
    @basterman13 25 днів тому +2

    Thank you for the video, I learned a lot! Could you please advise on best RAG implementation and document splitter for Python? I’ve tried several methods, but I often get mixed results, around 50/50 accuracy. The main issue is with chunking: sometimes, chunks split in a way that separates the beginning of a class or method from its continuation. Is there a way to ensure that chunks belonging to the same file can be grouped or kept together more effectively?
    Thank you in advance.

    • @technovangelist
      @technovangelist  25 днів тому +1

      That’s what the metadata in most vector databases is for. Describe the source. Then use that in your code to keep similar things together.

    • @basterman13
      @basterman13 24 дні тому

      @@technovangelist Thank you. Yesterday, after I left my comment, I came to the same conclusion. I just need to get distracted sometimes. The answer was always on the surface. I thought maybe there were some more specific approaches, but in this case, the simplest way is the best.
      Have you heard anything about LightRAG(HKUDS developer)? I'd be interested to hear your thoughts on it.

  • @hitmusicworldwide
    @hitmusicworldwide 24 дні тому

    I see the Thanka on your wall on the viewer left hand side.

    • @technovangelist
      @technovangelist  24 дні тому +1

      Good eye. From one of my two visits to Nepal. My sister used to run a health care clinic in a town called Jiri for about 20 years.

  • @marcelgeers3263
    @marcelgeers3263 21 годину тому

    How would you prefix code snippets or unit functions?

  • @muchainganga9563
    @muchainganga9563 25 днів тому +1

    Love this!

  • @YuryGurevich
    @YuryGurevich 25 днів тому

    Thanks, Matt!

  • @NLPprompter
    @NLPprompter 20 днів тому

    hi Matt, do you happen to know about contextual RAG by anthropic? does this some how similar with it?

  • @PanayotPanayotov-x6p
    @PanayotPanayotov-x6p 19 днів тому

    Can you provide articles or links to the documentation?

    • @technovangelist
      @technovangelist  17 днів тому

      The docs are in the github repo. You can get to it from ollama.com

  • @davidtapang6917
    @davidtapang6917 25 днів тому

    Hey bro! Subscribed!

    • @technovangelist
      @technovangelist  25 днів тому +1

      It’s been 20 years. Wonderful to have you

  • @agsvk-com
    @agsvk-com 23 дні тому

    Thank you for sharing. I'm just wondering how would we be able to select one of the prefixed nomic or prefixed snowflake arctic using one of the vector databases. Is this possible or do we need to do this via typescript or python? All the videos I see doesn't seem to have embeddings using any prefixed models? I'm still learning. It would be really great to have more step by step tutorials on this. 😊 God bless

  • @ToddWBucy-lf8yz
    @ToddWBucy-lf8yz 25 днів тому

    Great I'm refactoring for prefixes now, I'm sure now I need to update training data as well for prefixes Any pre trained models already capable.of using prefixes?

    • @technovangelist
      @technovangelist  25 днів тому +1

      Perhaps you should watch the video. It shows 3 models that use the prefixes.

    • @ToddWBucy-lf8yz
      @ToddWBucy-lf8yz 25 днів тому

      ​​@@technovangelistnomic isn't useful when your trying to integrate cypher queries and vector store queries in the same model. I'm try to avoid multiple models for my particular RAG setup.

    • @technovangelist
      @technovangelist  25 днів тому

      Avoiding multiple models is asking for lower quality results

    • @ToddWBucy-lf8yz
      @ToddWBucy-lf8yz 24 дні тому

      @@technovangelist yeah you are probably right...at least nomic is small and fast. Someone really needs to create a MoE just for RAG and databases.

    • @technovangelist
      @technovangelist  24 дні тому +1

      embedding models arent something you ask questions to. its just for the embedding to stick into the vector db and find similar results. you still have to use a regular model to get insights into your data.

  • @k1chko
    @k1chko 23 дні тому

    Seems similar to contextual embedding.

    • @technovangelist
      @technovangelist  23 дні тому

      Different topics. This was about how to get the embedding model to function correctly.

  • @tomwawer5714
    @tomwawer5714 25 днів тому

    Prefix yay

  • @FrankenLab
    @FrankenLab 25 днів тому

    The wave of the future doesn't include MORE work to get models to digest our content, it involves models that perform better on their own without coaxing them to give us a marginal improvement in the results. Also, only having 2 models with prefixing doesn't give many options. Great content though, appreciate the effort it takes to research, edit, and produce videos!

    • @technovangelist
      @technovangelist  25 днів тому +2

      Eventually maybe, but not for a long while. It’s still early days for this tech. There are more than 2. 3 were in this video and there are others that can be imported. And 2x in some cases is hardly marginal

    • @rv7591
      @rv7591 24 дні тому

      Well yeah but the future is discovered through experiments.

    • @technovangelist
      @technovangelist  24 дні тому

      Yeah but wishing for things doesn’t make them happen

  • @remmask
    @remmask 25 днів тому

    Hi Matt. Thank you for these videos. Can we get the source in python?

  • @ISK_VAGR
    @ISK_VAGR 25 днів тому

    Ok, Matt. All what u just said i knew. However, the question of the million dollars is why bigger models perform bad in embedding?

    • @technovangelist
      @technovangelist  25 днів тому +1

      They aren’t embedding models. Embedding models do embeddings. Regular LLMs don’t do it.

    • @jparkerweb
      @jparkerweb 24 дні тому

      @@technovangelist in other words, just because something "can" do it doesn't mean it "should" 🤣

    • @technovangelist
      @technovangelist  24 дні тому +2

      But I don’t think that language is strong enough. An embedding model might take 30 seconds when an llm can take 45 minutes and is 10% as effective. It’s bad enough when folks insist on using a 70b model for an answer that is maybe 10% better than an 8b model and wait 3 minutes instead of 30 seconds. That’s not worth it in most cases but there is a debatable benefit. Embedding with an llm make zero sense.

    • @jparkerweb
      @jparkerweb 24 дні тому

      @@technovangelist oh, I 100% agree! Choose the right tool for the right job

    • @ISK_VAGR
      @ISK_VAGR 24 дні тому

      @@technovangelist Matt u may have misunderstood my question. I was interested in why mathematically, a good LLM is not a good embedder. When I started to use RAG I believed that perhaps embedding models were LLMs delivering the output of hidden layers as embeddings. I still wonder why if LLMs can find patterns are not good in providing embeddings for RAG. Cheers..

  • @smhanov
    @smhanov 24 дні тому

    I have 200000 images of things described by llava. But if the user is searching for a single word, like "pants" then the search is too broad. It comes up with people wearing pants, shoes, etc. I'm hoping this prefix method helps a little.

  • @ShaunyTravels.
    @ShaunyTravels. 24 дні тому

    Wish there was more videos about running ollama on a mobile app I made a chat app using ollama running on a server on my phone with flutter dart but we need more videos to do that 😂

  • @grahaml6072
    @grahaml6072 25 днів тому +2

    I had to stop watching unfortunately with that fuzzy text flashing across the screen. Maybe I will just try and read the transcript

    • @technovangelist
      @technovangelist  25 днів тому

      I don’t have any fuzzy text on this one. If it’s fuzzy don’t watch at all low rez

    • @grahaml6072
      @grahaml6072 25 днів тому

      @@technovangelistI am not watching at low resolution. I watched on a 65” OLED. An iPad 12.9” a Samsung 49” widescreen and a 4K UST projector on 120” screen just to check it wasn’t me. It starts at 6:10 when you scroll through your outputs.

    • @technovangelist
      @technovangelist  24 дні тому +1

      oh, you were making a joke...got it....you aren't supposed to read that, which is why i said I am speeding forward.

    • @NLPprompter
      @NLPprompter 20 днів тому

      ironically we all do AI with fuzzy input output too... better get used to fuzzy mate 😁