Open AI Embeddings in Azure Vector Database of Cognitive Search

Поділитися
Вставка
  • Опубліковано 22 січ 2025

КОМЕНТАРІ • 34

  • @kevindibb6534
    @kevindibb6534 Рік тому +2

    Excellent cog search tutorial as usual, but honestly I needed to hear the last minute and half of this video more than anything else. Thank you.

  • @Yanikikudon
    @Yanikikudon Рік тому +1

    🎯 Key Takeaways for quick navigation:
    00:06 🤷‍♂️ Struggling with deciding between using Azure Cognitive Search and creating a vector database
    - Discusses the struggle of deciding between using Azure Cognitive Search for indexing words retrieval or creating a vector database with word embeddings.
    - Mentions the possibility of taking a hybrid approach.
    01:16 🤝 Introduction to the hybrid approach
    - Discusses that Azure Cognitive Search now has the capability to store word embeddings as a vector database and index them, enabling both semantic search and word embedding based retrieval.
    - Suggests using this approach for chat with your data scenarios.
    03:44 🔎 Deep Dive into Vector Search
    - Highlights the details and the best practices of using Vector Search.
    - Explains that Vector search uses Azure Cognitive Search as a vector database to store generated word embeddings for text, images, and videos.
    07:16 🌐 Multilingual Search Capability with Vector Search
    - Talks about the ability to perform searches using Vector Search regardless of language, thanks to the vector representation of the context.
    - Briefly mentions the use of vector search to support semantic search and word embedding based search.
    08:54 💾 Creating an Index for Word Embeddings in Azure Cognitive Search
    - Demonstrates a Python code showing how you can create an index for word embeddings using Azure Cognitive Search.
    - Specifies required credentials and configurations to connect to Azure Open AI and Azure Cognitive search services.
    14:25 🗣️ Conversation with Data using Lang Chain and Cognitive Search
    - Uses LangChain for managing the conversation with the data, using Cognitive Search for retrieving the information based on word embeddings.
    - Shows how questions are answered based on the closest word embeddings.
    16:34 🔍 Digging into the Backend of Cognitive Search
    - Looks into the backend of Cognitive Search, showing how word embeddings and data are stored and indexed in Azure Cognitive Search.
    - Asserts the value and efficiency of using Vector search in the retrieval process.
    18:48 🌟 Highlighting the Value of Vector Search
    - Emphasizes the value of Vector search and its advantages in making retrieval of information simpler and more efficient.
    - Encourages followers to use the hybrid approach leveraging semantic search and word embeddings based search.
    21:06 💡 Final thoughts and Philosophical ending
    - Discusses the concept of imperfection and the power of forgiveness.
    - Encourages viewers to dream big, believe in themselves, and take action.
    Made with HARPA AI

  • @dbiswas
    @dbiswas Рік тому +3

    One of the best content, and the end spiritual teaching was awesome too. Thanks 🙏

  • @shamaldesilva9533
    @shamaldesilva9533 Рік тому +6

    Was wondering what the azure equivalent was to vector databases like pinecone , thank you so much MG 🎉 appreciate your hard work and dedication 🥳

  • @learnwithengineer66
    @learnwithengineer66 8 місяців тому

    Hello I have a doubt if I am using RBAC in my azure AI search how can I Create a connection? below code we have to pass Key but if I am using RBAC how can I create Connection
    acs = AzureSearch(
    azure_search_endpoint=endpoint,
    azure_search_key=azureaikey,
    index_name=index_name,
    embedding_function=embeddings.embed_query,
    )

  • @bramjanssen8865
    @bramjanssen8865 Рік тому

    keep up the good work! great tutorials!

  • @pylanookesh8227
    @pylanookesh8227 Рік тому

    Thank you MG, for the clear explanation.

  • @bobetko71
    @bobetko71 Рік тому +1

    What about data storage? Where is original data and data embeddings stored?

    • @ttjordan81
      @ttjordan81 9 місяців тому

      I found where the documents are stored, which are stored in a blob container, but I cannot figure out where the actual embeddings are stored. Any luck? This UI sucks!

  • @nishantb80
    @nishantb80 Рік тому +1

    Interesting MG. Thanks 👍

  • @vijayakannanr2543
    @vijayakannanr2543 7 місяців тому

    Excellent Video.
    Say, if our document repository grow in size on daily basis. Creating the embedding of the newly added documents is not a problem but the time taken to index the whole vectors again is time consuming. How can we reduce the time of indexing?

  • @flutemelody8503
    @flutemelody8503 4 місяці тому

    how to add custom metadata to azure ai search

  • @vikassalaria24
    @vikassalaria24 Рік тому

    I am getting errror in Azure Search while connecting with cognitive search,it says resource not found,although i am grabbing correct resource as cognitive search url and key.i have models and cognitive search in same region,East US

    • @adityakoul6290
      @adityakoul6290 Рік тому

      Hi
      I am getting the same error. Were you able to resolve the issue?

    • @sayan3023
      @sayan3023 Рік тому

      I am facing the same issue as well

    • @a4amitava
      @a4amitava Рік тому

      @@sayan3023 facing the same, if you are able to solve please do share

    • @vikassalaria24
      @vikassalaria24 Рік тому

      I will be sharing a code base shortly.i have resolved the issue.

    • @kalyan909
      @kalyan909 Рік тому

      Even i got same resource not found error, its due to OpenAIEmbeddings.
      embeddings = OpenAIEmbeddings(model="model name",
      deployment="deployment name",
      openai_api_key = OPENAI_API_KEY,
      openai_api_base = OPENAI_API_BASE,
      openai_api_version = OPENAI_API_VERSION,
      chunk_size=1,
      openai_api_type="azure",)
      use this for embeddings it should work

  • @mihaelamironescu9736
    @mihaelamironescu9736 Рік тому

    How and where do i push this code to azure and make it run remotely?

  • @student8080
    @student8080 Рік тому

    Does small chunk size matters ? What happen if my question is between 2 chunk ?

  • @Gamezone-kq5sx
    @Gamezone-kq5sx 4 місяці тому

    Excellent tutorial.Please make One video for Azure open AI and azure AI with practical explanation .Also need tutorial work on google cloud for Generative AI

  • @JohnLee-wv4wq
    @JohnLee-wv4wq 11 місяців тому

    Can you point me where is the code you are using?

  • @sandeeppatidar5450
    @sandeeppatidar5450 Рік тому

    Thank you so much for this wonderful video. Do we have any trade off for hybrid search? Let’s say we have 1000 of pdfs ingested in azure congnitive index and also use same index for embedding vectors and then we do a hybrid search, so seach will take more time in hybrid search compare to only vector or semantic seach ?

  • @AnimeshTibrewal
    @AnimeshTibrewal Рік тому

    do you have this done in nodejs by any chance?

  • @NonnoSgrenf
    @NonnoSgrenf Рік тому +1

    Thank youuu

  • @AmirSharifianasanet
    @AmirSharifianasanet Рік тому

    Really Interesting MG, Thank you, Your accent is transforming from persian English into Indian English, maybe because your colleagues in MS are mostly Indian :) kidding boro :D.

  • @parkerrex
    @parkerrex Рік тому +2

    This intro 😂😂😂😂

  • @michaelwindeyer6278
    @michaelwindeyer6278 Рік тому +1

    I have a similar question @shamaldesilva9533 as to if it is similar to pinecone and also, as a first time viewer of yours, I wanted to say the spiritual message at the end was unexpected but AWESOME!