8. Text Classification Using Convolutional Neural Networks
Вставка
- Опубліковано 5 гру 2018
- Follow along with Lukas to learn about word embeddings, how to perform 1D convolutions and max pooling on text using Keras.
If you want to test your knowledge try to use CNNs to improve our example project at github.com/lukas/ml-class/tre...
Github repo: github.com/lukas/ml-class
See all classes: wandb.ai/site/tutorials
Weights & Biases: wandb.ai/site - Наука та технологія
![9. Hybrid LSTMs [Long Short-Term Memory]](http://i.ytimg.com/vi/NysY9FN9Uac/mqdefault.jpg)
![9. Hybrid LSTMs [Long Short-Term Memory]](/img/tr.png)
Thanks for the in depth explanation! I was trying to tokenize by character instead of by word which was causing all kinds of problems. Great vid
Awesome. One of the best tutorials on the internet.
I like your background. Very raw and natural
couldn't thank you enough! your explanation is the best!
Was searching for Keras+CNN simpler explanation and found you! Thanks a lot. Please keep up the good work.
Also, I just checked out your portal and created account as well. It's super cool.
awesome!
Thanks for this video I was really stuck on using cnn with text and this video really helped me
Thanks for the nice comment! I'm so glad to hear that.
Thanks for clear explanation and visualizations
I'm glad it was helpful!
thank you so much, this is very helpful 😇🙏👍
thanks so much!
Thank you , it helped me alot
This is very helpful thank you so much
Why not share the results of the second model, using GLOVE? did it help or hurt performance?
Here, the imdb-cnn.py file provided in the folder is completely different to the one you explain in the video. Can you please supply the version you used?
Thanks for this video , really good but you didn't show the Val loss after adding pretrained embedding
well done man.....I subscribed :)
I have dataset on Roman Urdu and want to classify the text. Basically want to perform #Sentiment_Anlaysis. I have tried with word2vec (300 fixed length) and deep neural network, but my accuracy is not more than 42%. Please guide
I've corrected the .py file however on execution the required input file can't be found.
Traceback (most recent call last):
File "imdb-cnn.py", line 25, in
(X_train, y_train), (X_test, y_test) = imdb.load_imdb()
File "C:\Users\su_matthewbennett\wandb\ml-class\videos\cnn-text\imdb.py", line 8, in load_imdb
X_train.extend([open(path + f).read() for f in os.listdir(path) if f.endswith('.txt')])
FileNotFoundError: [WinError 3] The system cannot find the path specified: './aclImdb/train/pos/'
Hello Lucas, Many Thanks for this wonderful video. Will you kindly put some shadow on the changes the model need if we want Multilabel text classification with CNN. i hope u can cover this thing .......In most of the cases we used the already available data for the videos. If we want to use our own data using word embedding (word2vec) etc. what changes will we need. It will be of great help. I am hopefully and really wish look forward for a video from you in your series based on multi label text classification while using our own collected data. Thank you very much...
If you are willing to do multilabel text classification you will have to one hot encode your target variable in an unique manner where every row will not have only one 'one' signifying one class label it belong to instead will have multiple 'one' denoting multiple label. Maybe this blog post can help you with multilabel classification. www.pyimagesearch.com/2018/05/07/multi-label-classification-with-keras/
Can we use of for requirement classification ?
hey mate, I have a question:
Is it possible to have two different categories of input (with different dimensionalities) at the same time? For instance, inputting the word embeddings, as well as a feature for let's say part of speech tags and average sentence length for one and the same document (for a list of documents)?
How do I design such an architecture?
You can always combine different architectures at the end with one or more dense (also known as fully connected) layers. So you could use all the word embedding stuff on the actual words but then add an extra layer at the end that takes the output of the word specific network as well as average sentence length.
Its a really a good video, could you please upload a video on how to use a pre-trained network like BERT for text classification and other tasks.
I hope you find this useful. ua-cam.com/video/-9evrZnBorM/v-deo.html
any particular reason for not including this video in your playlist(intro to ML) though it has *8* as title?
Our mistake, thanks for catching that! The playlist is now updated!
@@WeightsBiases cool! 😀
Hello, is there any link with the IMDB data set you used?
Maybe this will be useful. www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews
Why is the pooling at 8:18 performed on text data and not the output from the convolution?
He was just trying to show what pooling does maybe.
Thank you for yout video
Question: if i want to do prediction using this model can you provide me some details about it?
If you want to do prediction say classify your text which you want to either predict as positive or negative you first will have to convert it into the embedding (GloVe as shown) and pass in the embedding through your model. A model.predict() on this embedding will give you your predicted class. I hope it's helpful.
i am having problem with the input dimensions to this network.
at 6:46 (max pooling output 3X2), shouldn't the output number at (2,2) be 60 instead of 65?
Yeah it should be 60. :P
this video helped me greatly. thanks a lot !
thanks!
Hello Lucas, Many Thanks for this wonderful video. Will you kindly put some shadow on the changes the model need if we want Multilabel text classification with CNN. In most of the cases we used the already available data for the videos. If we want to use our own data using word embedding (word2vec) etc. what changes will we need. It will be of great help. I really wish and look forward for a video from you in your series based on multi label text classification while using our own collected data. Thanks a lot
Thanks! You should be able to run your own data, but if you want to make it a multilablel classification then you should change the last Dense(1) layer to Dense(num_classes). You will also want to change binary_crossentropy to categorical cross_entropy - I go over some of this in the very first video in the series - check it out and let me know if you have any questions or if you are able to get something working.
@@WeightsBiases Thanks Lucas for your kind answer. I have changed the num_classes in Dense as per the maximum number of labels. But as I have changed the loss function to categorical_cross entropy my accuracy goes down to somewhere around 70% which was around 90% when using binary_crossentropy as the loss function, without overfitting. Kindly have a look at the model:
#This models give 94.64% accuracy, with just a slight overfit in one epoch then came back to normal. Using fasttext model for word embeddings with 64000 training sentence and around 17000 testing sentences.
model = Sequential()
# Input / Embdedding
model.add(Embedding(num_words, 100, input_length=max_length, trainable = False))
# CNN
#model.add(SpatialDropout1D(0.3))
model.add(Dropout(0.2))
model.add(Conv1D(128, kernel_size=3, padding='same', activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.2))
model.add(Conv1D(64, kernel_size=3, padding='same', activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.2))
#model.add(GlobalMaxPool1D())
model.add(Flatten())
# Output layer
model.add(Dense(8, activation='sigmoid'))
model.summary()
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
Many Thanks in advance.
Ah, the reason is that binary_crossentropy makes the accuracy measure a different thing. When you use binary crossentropy and you make a prediction it counts each of the individual values, so if there are eight choices and you make the right prediction it counts as getting eight correct answers and if you make a wrong prediction it counts six correct values and two incorrect values. With categorical crossentropy the accuracy is more what you would expect.
@@WeightsBiases I see, that is why the accuracy is very high. Thanks for your kind explanation. I am learning a lot from your videos. I am sure your channel will be quite big in no time. The way you explain logic and things is very information and full of humor. Keep the good work up. Thanks and regards.
@@yogikumar221 thanks so much, I really appreciate it
can we use it for urdu language?
On running this code on the IMDB dataset with the GloVe Vectors, the model does no better than chance. Why so?
Maybe you would want to do better text preprocessing on the IMDB dataset. You can also use Embedding layer(keras.io/layers/embeddings/) to learn the embedding instead of using precomputed embedding using GloVe
I am facing an issue executing imdb-embedding.py How do I create './aclImdb/train/pos/' Directory? Also the imdb-cnn.py code is different from what is being shown in the video tutorial.
To make a new directory using Python you can use os.mkdir(path/dirname).
i built a model for text classification it is giving same output while prediction. I used tokenizer , glove lstm layers.
Can anyone help me, why I'm getting same result every time
Is your class distribution balanced? are you using the right loss function? how many classes do you have? what is the weight initialization method? what is the minimum loss are you getting? what optimizer are you using? what are your weight and biases distribution? could you share your code?
Im always getting error in my imports but i have already install keras and tensorflow and cuda! Need help please!
my bad, i have not installed pip install keras.models im new in this hahaha sorry
it gives me this error when i try to run the code in jupyterlab : ModuleNotFoundError: No module named 'wandb.keras'
any help please ?
first install wandb library. It's a library made by them
Try pip install wandb -q in your terminal
Where can i get this notebook ?
All of the code is in github.com/lukas/ml-class/videos
@@ran_domness Sorry, use github.com/lukas/ml-class/tree/master/videos
Interesting, however, it would be nice if you filled in the gaps in this video (where do the weights come from? Exactly what do you mean when you say "move the square over" to change the weights?)
weight is w of y=wx+b
where y is the predicted output
Can we use CCN for text data.? Please
You can if you can represent your text in a 1D vector. The maybe CNN can be used. I am unsure about the performance gain if any. But yeah it would be a good experiment.
Thank you dear
Does glove work with languages other than english
I guess not.
Can I steal your workshop?
I would like to do the same. 😛
Hi nice video, but I think your illustration on how convolution after the Embedding works is wrong, the kernel goes over the whole Embedding for each word and not only single numbers of the Embeddings, this also explains the "high" nr of weights when you look at the output (its embdim * nr_kernels+nr_kernels).
This video shows it: ua-cam.com/video/wNBaNhvL4pg/v-deo.html
Hey, does anyone know how to do a phishing URL classification using CNN ?
Why did you have to plug your framework that i am not interested in learning .... It was a good tutorial but I already have like a milion machine learning framework adding one more does not help me
Every framework exists for a reason. There is maximum of two-three popular frameworks doing the same thing at any given time.