Timestamps 00:05 - Most students fail to master GenAI fundamentals through quick tutorials. 02:02 - Learn to build production-grade systems in GenAI with a focus on NLP foundations. 06:09 - Understanding text processing for computer efficiency is crucial. 08:31 - Text processing fundamentals for NLP and machine learning. 12:47 - Preprocessing text by removing punctuation, emojis, and stop words for NLP. 14:55 - Overview of text processing techniques, including tokenization. 19:17 - Understanding tokenization and embeddings for NLP models. 21:28 - Embeddings convert words into vectors to reveal their relationships. 25:44 - Understanding word embeddings and their relationships. 27:44 - Understanding tokenization, embeddings, and attention mechanisms in language processing. 31:39 - The attention mechanism helps models focus on key words in sentences. 33:43 - Attention mechanisms highlight key words for better model performance. 37:42 - Attention models prioritize words by retention scores for better comprehension. 39:48 - Attention mechanisms focus on key information in data for effective NLP. 43:51 - Transformers improve language processing by analyzing entire sentences at once. 45:54 - Transformers use attention mechanisms for effective understanding of language. 50:05 - Transformer models refine understanding through layered processing. 52:02 - Understanding Transformers and text similarity measurement. 55:57 - Explains various similarity measures for comparing text meaning. 57:50 - Understanding mathematical similarity and information retrieval for text analysis. 1:01:52 - Representing and ranking documents for effective information retrieval. 1:03:45 - Indexing enhances information retrieval efficiency in systems like Google Search. 1:07:46 - Retrieval models act as expert librarians for finding specific information. 1:09:44 - Retrieval models are crucial for accurate answers in generative AI systems. 1:13:47 - Dense retrieval models improve information search by focusing on meaning instead of exact word matches. 1:15:53 - Retrieval models enhance generative AI by connecting relevant data. 1:19:34 - Understanding TF-IDF for keyword relevance in search queries. 1:21:39 - Understanding Inverse Document Frequency in TF-IDF for word importance. 1:25:37 - BM25 improves retrieval with frequency saturation and document length normalization. 1:27:45 - Document length normalization ensures fair importance across varying document lengths. 1:32:09 - Dense retrieval models enhance relevance beyond keyword matching. 1:34:05 - Dense retrieval models enhance search relevance by understanding meaning over exact word matches. 1:38:13 - Embeddings in Dual Encoder Architecture enhance understanding of meaning in document retrieval. 1:40:24 - DPR training uses positive and negative document-query pairs for relevance. 1:44:21 - Generative models address limitations of retrieval models in processing queries. 1:46:23 - Importance of clear and coherent responses in generative models. 1:50:26 - Generative models build coherent responses word by word. 1:52:26 - Generative models adapt responses based on user knowledge levels. 1:56:26 - Understanding retrieval and generative steps in AI models. 1:58:21 - Understanding photosynthesis through vector similarity matching. 2:02:18 - Understanding the retrieval and generation process in GenAI. 2:04:19 - Understanding RAG architecture enhances AI's response accuracy and clarity. 2:08:11 - Understanding Retrieval-Augmented Generative Systems for accurate AI responses. 2:10:09 - Understanding the RAG pipeline for effective information retrieval and generation. 2:13:52 - Text chunks are converted into 1536-dimensional embeddings for better understanding. 2:15:53 - Overview of the retrieval process in RAG systems. 2:19:53 - Efficient operations for embedding and similarity search using PG Vector. 2:22:39 - Introduction to essential libraries for GenAI integration. 2:26:30 - Creating and storing embeddings using OpenAI and PostgreSQL. 2:28:17 - Utilizing PG Vector for efficient question retrieval and embedding similarity. 2:32:01 - Using prompts to enhance AI chat responses effectively. 2:33:53 - Understanding randomness in model outputs through temperature adjustment. 2:38:02 - Setting up the database is crucial for project functionality. 2:40:00 - Setting up a database for document storage and categorization. 2:44:04 - Overview of document and tax metadata storage in the database. 2:45:48 - Establishing a many-to-many relationship between documents and tags. 2:49:49 - Understanding document chunking and embeddings for efficient data storage. 2:51:44 - Overview of generative models and database integration using SQL. 2:55:31 - Overview of document deletion and chunking processes in AI. 2:57:37 - Using AI to extract key facts from larger text chunks. 3:01:21 - Handling retries and processing PDF chunks with AI. 3:03:11 - Role of the system in generating and validating facts from user input. 3:07:05 - Function extracts and matches semantic tags from documents asynchronously. 3:09:04 - The process of tag matching and asynchronous API calls for document handling. 3:12:58 - Extract and validate PDF text using API for generating matching tags. 3:14:59 - Integrating document upload with tagging and chunking processes. 3:19:02 - Splits long text into smaller, manageable chunks for processing. 3:21:01 - PDF chunk processing involves creating asynchronous tasks for data extraction. 3:25:18 - Understanding asynchronous task execution for efficient data processing. 3:27:11 - Utilizing asynchronous methods for efficient data processing in GenAI. 3:31:36 - Overview of document chunking and embedding in the RAG system. 3:33:23 - Create an interactive Streamlit interface for PDF management. 3:37:10 - Document management system allows uploads and deletions with user prompts. 3:38:49 - Understanding the message class definition for chat applications. 3:42:24 - Utilizing vector similarity for efficient document retrieval. 3:44:26 - Document management for enhanced conversation input. 3:49:19 - Managing message handling in a conversational AI system. 3:51:20 - Managing references and messages in a chat application workflow. 3:54:56 - Enhancing document analysis with multi-file support and machine learning. 3:56:42 - Transform Lexi chat into a faster, smarter, production-ready system. 4:00:13 - Automating PGI vectorizer for efficient database interaction and retrieval. 4:01:55 - Setting up Python environment and Docker for GenAI project. 4:05:47 - Install and set up PG Vector for enhanced embedding generation. 4:07:39 - Challenges of previous document chunking and embedding processes. 4:11:34 - Automated embedding creation enhances efficiency using OpenAI with PostgreSQL. 4:13:25 - Automating document chunking and embedding with scheduled checks. 4:17:02 - Utilizing PGI vectorizer for efficient data handling and storage. 4:19:09 - Understanding optimization challenges in large datasets. 4:22:47 - Utilizing disk-based storage enhances data organization and retrieval speed. 4:24:36 - Efficient chunk retrieval enhances speed and accuracy in the RAG system. 4:28:06 - PG Vector Scale optimizes SQL queries for faster data retrieval. 4:30:00 - Optimizing database interactions reduces latency and errors in retrieval systems. 4:33:43 - Using PGI for efficient data retrieval and response generation. 4:35:37 - Understanding automatic chat completions and embedding creation.
It's true, from last 2 weeks I working to train llm model for conversation chat bot, but the training model is simple 10 minutes code, but it won't work as expected
Hi, I’m 25 years old and currently working as a Java backend developer with nearly 3 years of experience. I’m looking to transition into the field of AI development, specifically focusing on generative AI technologies. My goal is to build generative AI applications that solve real-world problems and create impactful solutions. That’s how I came across your UA-cam channel! I really admire your work, and I was hoping you could provide some guidance. From a career perspective, how should I begin this transition into AI and generative AI development? Your advice would mean a lot to me, and I’d be truly grateful for your help!
Firstly Learn the Basic Mathematics(Linear algebra, calcus, statistics & probability ), Python(Your Java knowledge will help to transistion into different programming) because it has AI library(numpy, pandas and sckitlearn) to easy your hectic work, ML concept and understand the math behind in each of the concepts, DL algorithms and undertand the usecase of each algorithms, error reducing and validation. This is basic concept to move into GEN AI. then you can explore different LLM models, NLP, transformer etc.... Statistical and Mathematical Foundations Probability and Statistics - Descriptive and inferential statistics - Probability distributions (Normal, Poisson, Binomial) - Hypothesis testing - Confidence intervals - Central Limit Theorem - Bayesian vs. Frequentist approach - A/B testing methodology Mathematics - Linear Algebra - Matrix operations - Eigenvectors and eigenvalues - Vector spaces - Calculus - Derivatives and gradients - Optimization techniques - Gradient Descent - Regularization techniques Machine Learning Algorithms: Supervised Learning - Linear Regression - Logistic Regression - Decision Trees - Random Forests - Gradient Boosting (XGBoost, LightGBM) - Support Vector Machines (SVM) - K-Nearest Neighbors (KNN) Unsupervised Learning - K-Means Clustering - Hierarchical Clustering - Principal Component Analysis (PCA) - t-SNE - DBSCAN Deep Learning - Neural Network architectures - Convolutional Neural Networks (CNN) - Recurrent Neural Networks (RNN) - Long Short-Term Memory (LSTM) - Transformer models - Basics of TensorFlow & PyTorch Python Programming Skills: Core Python: - Data structures (lists, dictionaries, sets) - List comprehensions - Lambda functions - Decorators - Error handling Data Science in Python: - NumPy - Array operations - Vectorization - Pandas - Data manipulation - Groupby operations - Merging and joining datasets - Scikit-learn - Model training and evaluation - Preprocessing techniques - Pipeline creation - Matplotlib & Seaborn for data visualization Practical Interview Preparation Tips: 1. Build a strong portfolio on GitHub 2. Practice coding on LeetCode and HackerRank 3. Participate in Kaggle competitions 4. Understand the business context of your models 5. Practice explaining complex concepts simply 6. Be prepared to discuss model selection, bias-variance tradeoff, and model evaluation metrics Key Interview Topics to Master: - Feature engineering - Model evaluation (precision, recall, F1-score) - Overfitting and underfitting - Cross-validation techniques - Handling imbalanced datasets - Model interpretability - Basic software engineering principles for data scientists
@@AjaySharma-dg7cc This is the right time if you want to transition, develop some small projects in this domain and enrich your github. Next start applying for jobs with those experiences.
Hey Ayush, I'm new to AI and ML, and I came across your course. Does this course cover everything needed to build AI/ML systems from scratch to a mastery level? Also, will it help me build real-world projects and systems by the end?
great course and one thing I think with all those slides u multiple time explained it, made it a drag to get through, I think till rag it could be completed in 30-35 mins
Yep, ML is more about making something work with explainable approach and it doesn't make sense if we just write it off within 3 lines of code. Consider checking out my course on Core ML on youtube for free.
@AyushSinghSh so what should I actually do, because everywhere they do train() fit() and done. Btw, I am 13 years(learned python, libraries,linear algebra, probability and statistics,will learn ML next) just like you. And many people say that getting ML job as a fresher is not possible, then how did you get one
ua-cam.com/video/IRrhpAXib-Y/v-deo.html what ui/tool are you using when showing us the table's data? I don't understand that. Can you put tool and explanation for that as well please.
Make production grade systems or scale it via Self-host or try Timescale Cloud for free here: github.com/timescale/pgai
Timestamps
00:05 - Most students fail to master GenAI fundamentals through quick tutorials.
02:02 - Learn to build production-grade systems in GenAI with a focus on NLP foundations.
06:09 - Understanding text processing for computer efficiency is crucial.
08:31 - Text processing fundamentals for NLP and machine learning.
12:47 - Preprocessing text by removing punctuation, emojis, and stop words for NLP.
14:55 - Overview of text processing techniques, including tokenization.
19:17 - Understanding tokenization and embeddings for NLP models.
21:28 - Embeddings convert words into vectors to reveal their relationships.
25:44 - Understanding word embeddings and their relationships.
27:44 - Understanding tokenization, embeddings, and attention mechanisms in language processing.
31:39 - The attention mechanism helps models focus on key words in sentences.
33:43 - Attention mechanisms highlight key words for better model performance.
37:42 - Attention models prioritize words by retention scores for better comprehension.
39:48 - Attention mechanisms focus on key information in data for effective NLP.
43:51 - Transformers improve language processing by analyzing entire sentences at once.
45:54 - Transformers use attention mechanisms for effective understanding of language.
50:05 - Transformer models refine understanding through layered processing.
52:02 - Understanding Transformers and text similarity measurement.
55:57 - Explains various similarity measures for comparing text meaning.
57:50 - Understanding mathematical similarity and information retrieval for text analysis.
1:01:52 - Representing and ranking documents for effective information retrieval.
1:03:45 - Indexing enhances information retrieval efficiency in systems like Google Search.
1:07:46 - Retrieval models act as expert librarians for finding specific information.
1:09:44 - Retrieval models are crucial for accurate answers in generative AI systems.
1:13:47 - Dense retrieval models improve information search by focusing on meaning instead of exact word matches.
1:15:53 - Retrieval models enhance generative AI by connecting relevant data.
1:19:34 - Understanding TF-IDF for keyword relevance in search queries.
1:21:39 - Understanding Inverse Document Frequency in TF-IDF for word importance.
1:25:37 - BM25 improves retrieval with frequency saturation and document length normalization.
1:27:45 - Document length normalization ensures fair importance across varying document lengths.
1:32:09 - Dense retrieval models enhance relevance beyond keyword matching.
1:34:05 - Dense retrieval models enhance search relevance by understanding meaning over exact word matches.
1:38:13 - Embeddings in Dual Encoder Architecture enhance understanding of meaning in document retrieval.
1:40:24 - DPR training uses positive and negative document-query pairs for relevance.
1:44:21 - Generative models address limitations of retrieval models in processing queries.
1:46:23 - Importance of clear and coherent responses in generative models.
1:50:26 - Generative models build coherent responses word by word.
1:52:26 - Generative models adapt responses based on user knowledge levels.
1:56:26 - Understanding retrieval and generative steps in AI models.
1:58:21 - Understanding photosynthesis through vector similarity matching.
2:02:18 - Understanding the retrieval and generation process in GenAI.
2:04:19 - Understanding RAG architecture enhances AI's response accuracy and clarity.
2:08:11 - Understanding Retrieval-Augmented Generative Systems for accurate AI responses.
2:10:09 - Understanding the RAG pipeline for effective information retrieval and generation.
2:13:52 - Text chunks are converted into 1536-dimensional embeddings for better understanding.
2:15:53 - Overview of the retrieval process in RAG systems.
2:19:53 - Efficient operations for embedding and similarity search using PG Vector.
2:22:39 - Introduction to essential libraries for GenAI integration.
2:26:30 - Creating and storing embeddings using OpenAI and PostgreSQL.
2:28:17 - Utilizing PG Vector for efficient question retrieval and embedding similarity.
2:32:01 - Using prompts to enhance AI chat responses effectively.
2:33:53 - Understanding randomness in model outputs through temperature adjustment.
2:38:02 - Setting up the database is crucial for project functionality.
2:40:00 - Setting up a database for document storage and categorization.
2:44:04 - Overview of document and tax metadata storage in the database.
2:45:48 - Establishing a many-to-many relationship between documents and tags.
2:49:49 - Understanding document chunking and embeddings for efficient data storage.
2:51:44 - Overview of generative models and database integration using SQL.
2:55:31 - Overview of document deletion and chunking processes in AI.
2:57:37 - Using AI to extract key facts from larger text chunks.
3:01:21 - Handling retries and processing PDF chunks with AI.
3:03:11 - Role of the system in generating and validating facts from user input.
3:07:05 - Function extracts and matches semantic tags from documents asynchronously.
3:09:04 - The process of tag matching and asynchronous API calls for document handling.
3:12:58 - Extract and validate PDF text using API for generating matching tags.
3:14:59 - Integrating document upload with tagging and chunking processes.
3:19:02 - Splits long text into smaller, manageable chunks for processing.
3:21:01 - PDF chunk processing involves creating asynchronous tasks for data extraction.
3:25:18 - Understanding asynchronous task execution for efficient data processing.
3:27:11 - Utilizing asynchronous methods for efficient data processing in GenAI.
3:31:36 - Overview of document chunking and embedding in the RAG system.
3:33:23 - Create an interactive Streamlit interface for PDF management.
3:37:10 - Document management system allows uploads and deletions with user prompts.
3:38:49 - Understanding the message class definition for chat applications.
3:42:24 - Utilizing vector similarity for efficient document retrieval.
3:44:26 - Document management for enhanced conversation input.
3:49:19 - Managing message handling in a conversational AI system.
3:51:20 - Managing references and messages in a chat application workflow.
3:54:56 - Enhancing document analysis with multi-file support and machine learning.
3:56:42 - Transform Lexi chat into a faster, smarter, production-ready system.
4:00:13 - Automating PGI vectorizer for efficient database interaction and retrieval.
4:01:55 - Setting up Python environment and Docker for GenAI project.
4:05:47 - Install and set up PG Vector for enhanced embedding generation.
4:07:39 - Challenges of previous document chunking and embedding processes.
4:11:34 - Automated embedding creation enhances efficiency using OpenAI with PostgreSQL.
4:13:25 - Automating document chunking and embedding with scheduled checks.
4:17:02 - Utilizing PGI vectorizer for efficient data handling and storage.
4:19:09 - Understanding optimization challenges in large datasets.
4:22:47 - Utilizing disk-based storage enhances data organization and retrieval speed.
4:24:36 - Efficient chunk retrieval enhances speed and accuracy in the RAG system.
4:28:06 - PG Vector Scale optimizes SQL queries for faster data retrieval.
4:30:00 - Optimizing database interactions reduces latency and errors in retrieval systems.
4:33:43 - Using PGI for efficient data retrieval and response generation.
4:35:37 - Understanding automatic chat completions and embedding creation.
Thank you..❤
Thank you
It's a crisp and concise that has everything needed to kickstart GenAI journey. Thanks for your time and efforts.
02:23:58 Where is link of the form to get openai credits?
you should start from scratch database setup, pgvector installation and all
❤👍 just now thought of learning GenAI and here it is
where can i find the ppt of the video . thanks for the course man
It's true, from last 2 weeks I working to train llm model for conversation chat bot, but the training model is simple 10 minutes code, but it won't work as expected
Glad to hear it :)
Truly acceptable!
Thanks ayush for bringing such a quality content in such a chaos of sources in new tech AI , Gen AI
Hi, I’m 25 years old and currently working as a Java backend developer with nearly 3 years of experience. I’m looking to transition into the field of AI development, specifically focusing on generative AI technologies. My goal is to build generative AI applications that solve real-world problems and create impactful solutions.
That’s how I came across your UA-cam channel! I really admire your work, and I was hoping you could provide some guidance. From a career perspective, how should I begin this transition into AI and generative AI development?
Your advice would mean a lot to me, and I’d be truly grateful for your help!
Firstly Learn the Basic Mathematics(Linear algebra, calcus, statistics & probability ), Python(Your Java knowledge will help to transistion into different programming) because it has AI library(numpy, pandas and sckitlearn) to easy your hectic work, ML concept and understand the math behind in each of the concepts, DL algorithms and undertand the usecase of each algorithms, error reducing and validation.
This is basic concept to move into GEN AI.
then you can explore different LLM models, NLP, transformer etc....
Statistical and Mathematical Foundations
Probability and Statistics
- Descriptive and inferential statistics
- Probability distributions (Normal, Poisson, Binomial)
- Hypothesis testing
- Confidence intervals
- Central Limit Theorem
- Bayesian vs. Frequentist approach
- A/B testing methodology
Mathematics
- Linear Algebra
- Matrix operations
- Eigenvectors and eigenvalues
- Vector spaces
- Calculus
- Derivatives and gradients
- Optimization techniques
- Gradient Descent
- Regularization techniques
Machine Learning Algorithms:
Supervised Learning
- Linear Regression
- Logistic Regression
- Decision Trees
- Random Forests
- Gradient Boosting (XGBoost, LightGBM)
- Support Vector Machines (SVM)
- K-Nearest Neighbors (KNN)
Unsupervised Learning
- K-Means Clustering
- Hierarchical Clustering
- Principal Component Analysis (PCA)
- t-SNE
- DBSCAN
Deep Learning
- Neural Network architectures
- Convolutional Neural Networks (CNN)
- Recurrent Neural Networks (RNN)
- Long Short-Term Memory (LSTM)
- Transformer models
- Basics of TensorFlow & PyTorch
Python Programming Skills:
Core Python:
- Data structures (lists, dictionaries, sets)
- List comprehensions
- Lambda functions
- Decorators
- Error handling
Data Science in Python:
- NumPy
- Array operations
- Vectorization
- Pandas
- Data manipulation
- Groupby operations
- Merging and joining datasets
- Scikit-learn
- Model training and evaluation
- Preprocessing techniques
- Pipeline creation
- Matplotlib & Seaborn for data visualization
Practical Interview Preparation Tips:
1. Build a strong portfolio on GitHub
2. Practice coding on LeetCode and HackerRank
3. Participate in Kaggle competitions
4. Understand the business context of your models
5. Practice explaining complex concepts simply
6. Be prepared to discuss model selection, bias-variance tradeoff, and model evaluation metrics
Key Interview Topics to Master:
- Feature engineering
- Model evaluation (precision, recall, F1-score)
- Overfitting and underfitting
- Cross-validation techniques
- Handling imbalanced datasets
- Model interpretability
- Basic software engineering principles for data scientists
i want a referal to join as a python developer so can anyone help me in this situation
Same , i am also from java backend with 2.6 yrs and looking to transition in AI dev
@@AjaySharma-dg7cc This is the right time if you want to transition, develop some small projects in this domain and enrich your github. Next start applying for jobs with those experiences.
Excellent conceptual teaching approach. 👍 adding timestamp to the video will be appreciated. Take care.
Thanks! Will do!
For the beginners it is a little difficult to configure the environment I phase difficulties if possible can u guide us for the environment to set up
Hey Ayush,
I'm new to AI and ML, and I came across your course. Does this course cover everything needed to build AI/ML systems from scratch to a mastery level? Also, will it help me build real-world projects and systems by the end?
have knowldege on nlp and deep learning
Can someone help me with the notes ? I couldnt find the link
hardware requirements??
Ayush can u tell the pre-requisites for this
Its covers basics of NLP too. So Python will be requirement
Bro just I advise plz!! How do you plan to learn new skill and be consistent throughout ??
can anyone share the docker compose file consisting of timsescaledb and pgai
Can you turn on the transcript? It will help us in great way for making notes.
Why the customised (task) chatbots were not working as expected ?
great course and one thing I think with all those slides u multiple time explained it, made it a drag to get through, I think till rag it could be completed in 30-35 mins
OG AYUSH IS BACK!
Yesssssss
can you provide the colab notebook you mention in the video
Again here after year and a half. 7415D 02:07
Could you please share the form for key credits??
Slides /Notes kaha milega
what i have to learn first ML or Gen AI or Python
Start with genai basics :) see where you’re at
Real Ayush is back💥✨
YESS
will you give the python not4ebooks access?
Yes will attach in description
Bro, you once said writing three line of code in jupyter is not ML. Can you please explain a bit
Yep, ML is more about making something work with explainable approach and it doesn't make sense if we just write it off within 3 lines of code. Consider checking out my course on Core ML on youtube for free.
@AyushSinghSh so what should I actually do, because everywhere they do train() fit() and done.
Btw, I am 13 years(learned python, libraries,linear algebra, probability and statistics,will learn ML next) just like you. And many people say that getting ML job as a fresher is not possible, then how did you get one
Companies Asking Masters and Research Publications what to do for that as belonging from decent college get easily the job of avg 12lpa ????
hey ayush!...can you please attach the ppt...and also the same codes of the notebooks
Will make sure to attach them :)
Is it for non technology person?
where is form of free chatgpt api key
a time stamp for each section would be nice
Can this be done by a non tech people? Thank you so much! 😊❤
Yes you can!
Is this gen ai full course worth it for campus placements??
Yes :) try it
When will you bring a course in hindi ?
Soon
Bro where is time stamp?
Can u make one?
these 5 hours is worth it
❤️❤️
Thanks a lot Ayush
ua-cam.com/video/IRrhpAXib-Y/v-deo.html what ui/tool are you using when showing us the table's data? I don't understand that. Can you put tool and explanation for that as well please.
How old are you?
bro wherewrer you
This is like they added maths on english grammar
💥✨
😅put timestamps bro
Put some chapter breaks bro
ua-cam.com/video/NWONeJKn6kc/v-deo.htmlsi=LO62KmQ3RQZrfk0a ....is this video is the prerequisite??