The Secret to 90%+ Accuracy in Text Classification
Вставка
- Опубліковано 6 лип 2024
- In this video, we will be providing a beginner's guide to fine-tuning BERT, one of the most powerful natural language processing (NLP) models available. BERT, which stands for "Bidirectional Encoder Representations from Transformers," has been trained on a massive amount of text data and can be fine-tuned for a variety of NLP tasks, such as text classification, machine translation, and named entity recognition. We'll be walking through the process of fine-tuning BERT using the Hugging Face library, and provide examples and code snippets to help you get started with your own fine-tuning projects. This video is perfect for anyone who is new to NLP and wants to learn more about BERT and how to use it in their own projects. So, if you are a beginner in the field of NLP and want to learn more about BERT, this video is for you!
Source Code:
Kaggle Notebook: bit.ly/3GTQklf
🔗 Social Media 🔗
📱 Twitter: bit.ly/3aJWAeF
📝 LinkedIn: bit.ly/3aQGGiL
📂 GitHub: bit.ly/2QGLVYV
Timestamps:
00:00 Introduction
00:33 Loading BERT from HuggingFace 🤗
01:52 Loading Tokenizer from HuggingFace 🤗
03:42 Output of BERT 👀 (Understanding Encoder Representations)
05:50 Loading the Dataset
07:22 Building the Model (BERT for Classification)
09:18 Fine-Tuning/Training BERT
09:47 Evaluating BERT (92% Yeyy!!)
10:05 So what are you fine-tuning next? 👀
10:22 Outro. See you soon! 👋
Tags:
#BERT #FineTuning #NLP #MachineLearning #BeginnersGuide #DeepLearning #NaturalLanguageProcessing #Tutorial #howto #BERTModel #TextClassification #LanguageModeling #TransferLearning #NeuralNetworks.
Keywords:
BERT, Fine-Tuning, NLP, Machine Learning, Beginners Guide, Deep Learning, Natural Language Processing, BERT Model, Text Classification, Pre-trained Models, Language Modeling, Transfer Learning, Neural Networks, Fine-Tune BERT, Fine-Tuning BERT
Thank you,
Pritish Mishra - Наука та технологія
![BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token](http://i.ytimg.com/vi/90mGPxR2GgY/mqdefault.jpg)
![BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token](/img/tr.png)
Subscribe.
Best explanation in a simple and easy way.
from the last 3-4 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.
best video i could find , easy, simple and to the point.
Thanks a lot for this video. It's more than a simple tutorial, you really explain the most important concepts in a way that's clear
Really really amazing Pritish. This video is not like those boring lecture videos. The animations are amazing. your explanation is clear with goof pronunciation. Amazing. Keep it up. I hope you continue posting these type of videos. ❤❤❤❤
This is the best way of explaining the Models! Keep it up!!! I expect some plots/graphs on accuracy and predictions details!
Explanation done by you is the best compared to any others....awesome work Pritish ....keep it up
best explanation of fine tuning of bert , got good understanding from video thanks
Very Nice Explanation and nice Animation 🔥🔥🔥🔥
Keep it up 👍🏻
You have so much potential, amazing!
Amazing work Pritish. You definitely deserve more views. Hopefully you will get it soon❤.
thank bro!
Good explanation, it was easy to understand
Absolutely brilliant!
Very nicely explained
Awesome explanation 👌
Nice explanation.
Can you do a video on how to do Natural language inference with Bert? Thanks!
i have some text files in which there are elements and their values but the pattern in which the text is displayed in the file are different from file to file. Is it possible to train Bert on these files so that when I ask it to extract only the element names and their corresponding values it will do that regardless of the text pattern?
Really Good video.
for the bert text summarization can we do in this way????
Can we use bert for context aware similarity?
Excellent video. I got one error while running the code.
inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True,
return_tensors='tf')
inputs
For this line I got the following error:
TypeError Traceback (most recent call last)
Cell In[50], line 1
----> 1 inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True,
2 return_tensors='tf')
3 inputs
TypeError: 'BertTokenizer' object is not callable
Can you please help?
pritish will you create a video of simulating a robotic arm which is controlled by a GPT-language model , and can cook food in simulation ?
Very nice video; I wonder if it is possible to save the classifier for future use.
You can save the classifier as follows:
classifier.save("filename.h5")
I'd like to ask, a paper I am trying to use for another dataset said they had optimal performance at epochs=50, however at epochs=3, it's already getting decent performance. May I ask why this is? Also, do you run bert in inference mode?
so we don't need to freeze any layer of the pretrained model? i have a problem this is with VIT my image shape is 24x24 but the pretrained model input shape is 224x224 it is possible to fix that? and the learning parameter are 8900000 and i want to fine tune it on my dataset
I’m not clear on what pooling in the video is.
How to import our dataset and train and test them
it show me train_dataset is not defined????
it was good video
Hey, nice video! Just one question: how can you serialize a custom Keras model, such as yours -- class BERTForClassification(tf.keras.Model)?
You can serialize a custom Keras model like our BERTForClassification using the model.save('filename') method. This will save the entire model, including the architecture, weights, and optimizer, to a file. You can then load the saved model using the tf.keras.models.load_model('filename') method.
If the model.save() method doesn't work for you, you can use model.save_weights() instead. This method saves only the weights of the model to a file, so you will need to define the model architecture exactly as it was at the time of saving the weights in order to load the saved weights correctly.
model = ...
model.load_weights('/path/to/file')
@@PritishMishra This doesn't work. This is my code:
classifier.save(save_path)
classifier_2 = tf.keras.models.load_model(save_path)
I get this error:
TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 27), dtype=tf.int64, name='input_ids/attention_mask').
@@WoWmastersonTuralyon Could you please try with model.save_weights? Please let me know if this causes any errors.
@@PritishMishra Yes, upon further research I came upon this solution too. The code that works looks like this:
classifier.save_weights(save_path)
classifier_2 = BERTForClassification(bert_model=transformers_model, num_classes=no_classes)
classifier_2.load_weights(save_path)
example_input = "just some dummy input string"
encoded_example = tokenizer.encode(example_input, padding=True, truncation=True, return_tensors="tf")
_ = classifier_2(encoded_example)
The input is needed, because the bert input layer has a dynamic size, and gets built after running an input through it. Otherwise, other functions regarding the model (such as classifier_2.summary()) would return an error.
Thanks for your help!
@@WoWmastersonTuralyon thanks for posting the solution too.
which model would be suitable for classifying if text is written by human or generated by LLM?
Yes.
recreated this in pycharm, when i want to use the model (i saved it first) i get this error: TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 78), dtype=tf.int64, name='input_ids_attention_mask'). is there a way to fix this without retraining the model?
all i want is to calculate precision, recall and f1 score of the model btw.
great video, can you make a video on question answering? and can we make a chatbot just using bert or will we be needing a Decoder along with bert for that
Thanks a lot for this video. Could you write the code how to do inference through pooler_outpt?
hi bro i facing error
I'm using hugging face dataset if sentiment analysis.
where in the dataset contain sentiment column and data type of sentiment column is string how to convert into integer and label number 1,2,3
in your case automatically convert into int64
please guide me I'm stock from last 7 days
thanks for your attention
how to actually decode the output back to the classes is something this video did not explain : \
bert is not an LLM
It used to be a LLM, but obviously now in the gen of trillions parameters model we can't say millions parameters model a LLM, but earlier it was a LLM , after few years these llama 2 and gpt 3 can't be LLM according to future models standard, but that does not change the fact of current scenario. hence, BERT is a LLM which trained on less parameters.
Hey, I really liked your video, although I have few doubts on a few matters, I would love to have a chat with you if it were possible, so you can help me. Do you think we can have a chat over Discord or similar?
Great video! any clue why I got so low accuracy ?
The Secret to 90%+ Accuracy in Text Classification
32/32 ━━━━━━━━━━━━━━━━━━━━ 313s 10s/step - accuracy: 0.3567 - loss: 1.5853
[1.5781687498092651, 0.3684999942779541]
same here
accuracy: 0.3741 - loss: 1.5632
Hey, there's some problems with the code. Let me try to fix this.
from the last 5-6 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.