I suppose as a beginner if someone had a similar project, they could just highlight one item like the white sections and have everything else be background blue. Then run the filters to find just the white sections. That would save time setting the problem up and testing it.
those masked images you made of the original must be one of the most important steps in the process. It must be best to know that one should set up multiple colors each of which are meant to highlight the crucial details of the structure that you are looking at. Probably best to have colors that aren't similar distinguishing items near each other too.
I've been following your tutorial with great interest. To avoid typos I generally use your source code. The RandomForestClassifier took a very long time to fit. Since the youtube was fast I wondered why? It turns out that in line 147 the code has n_estimators=100, while you used 10. I see that the default was changed from 10 to 100 in the documentation, which explains the difference in the code to what you demonstrated. For 10 it took about 12 seconds, and for 100 it took 2 min (10*12sec). The precision was a bit better, but it was already great at 10. I'm wondering if you should change your code to fit what you demoed, instead of the documentation default.
Thank you so much for the time and effort to make these videos. They are indeed a rare find in UA-cam and they have taught me a lot. I'm working a set of multidimensional data right now and have heard you mention in the video that you can use the same method on multiple images to train. However, I'm not sure how we can perform feature extraction when the input array is an array of arrays where each row is an array containing all the pixels of 1 image?
Hi, thank you so much for this series of tutorials! They're really detailed and very well explained! I've been through so many other videos and articles, and this series seems to be the most clear in terms of explanation of concepts and actual application in code! Thank you so much! I'm currently working on an image classification project. I specifically want to do a comparison between the performances of models built using a traditional ML approach vs. deep learning (CNNs). As such, I do have 2 questions that I would like to ask you: 1. The features dataframe that you built for the classifier in this video is basically individual pixel values of various filtered forms from one single image. So, the entire dataframe is from one single image, and each row corresponds to one single pixel from that image. Is there any way I can apply this approach to a bunch of images (image classification instead of image segmentation)? Specifically, I would want to create a features dataframe from a bunch of images, where each row corresponds to one single image (instead of just one pixel), and the columns are still the various filtered forms of those images. I understand that in this case, each individual value in my feature dataframe will end up being an array of pixel values (instead of single pixel values like in your case), and a ML algorithm would not accept that as an input. Is there any way I could convert these arrays of values into a single value which can then be input into a classifier like SVM or RandomForest? Or is my approach totally wrong here? 2. I'm working in Google Colab so as to utilize the GPUs for my deep learning models. Does Tensorflow work with GPUs by default? Or is there a specific configuration/command that I need to run first before Tensorflow can switch over to using GPUs. (Sorry if you've answered this question in one of your other videos. I only discovered your channel 2 days ago, and haven't been able to go through all the videos yet!) Apologies in advance for the verbosity of my questions! :D
1. Yes you can (and should) extract features from multiple images. For that, you just need to read one images after other and then add features to the dataframe. In other words, just concatenate values from multiple images into a single table / dataframe. 2. You can turn on GPU capability on Google Colab. Once you do that then it automatically uses GPU when you use tensorflow (or keras). Please search for this topic on UA-cam, there are numerous videos on how to enable GPU on Colab.
Thanks for the useful information. Are you using only one image Sandstone_Versa000.tif to generate features using Gabor filter? What if we have a bunch of pictures?
You just need to read multiple images & masks instead of one. Then perform the same operations (extracting features) on all images and concatenate the data. In other words, repeat the process on multiple images and add those pixels as new rows to your Pandas DataFrame. The best way would be define a function to perform the feature extraction process and apply the function to multiple files and keep concatenating the data. I showed some of that work in part 5 video titled - '67 - Image Segmentation using traditional machine learning - Part5 Segmenting Images'.
@@janasstory4974 Not sure if I understand exact issue but if everything goes well you should see 32 Gabor features plus a few other features corresponding to Sobel, Prewitt, etc. for each labeled image. In the example I used you will have a total of 41 features (columns) and 1019904 pixels (rows) as my input image had dimensions of 1024 x 996. If you have 2 labeled images for training then you will have 2 * 1019904 rows but the number of columns should be the same as you are extracting same number of features for every image. If this is not the case then you may have to debug for logic errors - not fun exercise to do but can be a good learning experience.
Masked images are ground truth images that you need to generate yourself. There are companies here in the bay area that offer these services. They hire cheap labor from other countries to paint pixels in order to annotate them. I always wondered how these external non-technical people understand the application in order to annotate images. In any case, if you want to annotate your own images you can do it for free here: www.apeer.com/annotate
@@DigitalSreeni When exporting the labeled image from APEER Anotation, it exports a gray color image instead of a colored labeled image. Am I doing something wrong? Thanks in advance professor!
For face recognition you probably do not want to segment every pixel but rather classify the entire face. So you need to encode entire image with the face extracted to a value that you need to train and predict. In summary, you are looking for image classification rather tan segmentation. Also, search for VGG Face, you will find pre-trained models on celebrity faces.
Not sure what you are exactly looking for. If you are inquiring about using multiple images for training then I did a video on that, check out video 67b. ua-cam.com/video/sD2xL36Xdu0/v-deo.html
Obviously you are performing machine learning where the system is trying to fit a model based on the input data. You need to have high powered systems for any machine learning tasks.
@@DigitalSreeni Sure, but I also tried this code in collaboratory with 25gb of ram and it also consumed all the memory, maybe I need to change some parameter of the random forest function?
Not sure about a good book on this topic. Please look for machine learning books focused on computer vision. Hopefully you will find a good book that covers the topics of traditional machine learning (e.g., Random Forest, SVM, etc.)
I can understand both theory and application easily from your videos. Great, thanks.
You are welcome.
I suppose as a beginner if someone had a similar project, they could just highlight one item like the white sections and have everything else be background blue. Then run the filters to find just the white sections. That would save time setting the problem up and testing it.
those masked images you made of the original must be one of the most important steps in the process. It must be best to know that one should set up multiple colors each of which are meant to highlight the crucial details of the structure that you are looking at. Probably best to have colors that aren't similar distinguishing items near each other too.
Hitch-hiker's Guide to the Galaxy reference made my day!
:)
would be better if you printed some prediction images. I learned to not trust accuracy the hard way. But other than that great tutorial.
I was able to use the histogram segmentation from previous videos to make my own test image data. so far so good! 42!!
You can try the annotation tool from here in case histogram segmentation is not possible. www.apeer.com/annotate
I made an account, but I haven't had a chance to look yet. I'm hoping to finish the series before I use other programs. Keep up the good work!
Thanks
I've been following your tutorial with great interest. To avoid typos I generally use your source code. The RandomForestClassifier took a very long time to fit.
Since the youtube was fast I wondered why? It turns out that in line 147 the code has n_estimators=100, while you used 10.
I see that the default was changed from 10 to 100 in the documentation, which explains the difference in the code to what you demonstrated.
For 10 it took about 12 seconds, and for 100 it took 2 min (10*12sec). The precision was a bit better, but it was already great at 10.
I'm wondering if you should change your code to fit what you demoed, instead of the documentation default.
Thank you so much for the time and effort to make these videos. They are indeed a rare find in UA-cam and they have taught me a lot.
I'm working a set of multidimensional data right now and have heard you mention in the video that you can use the same method on multiple images to train. However, I'm not sure how we can perform feature extraction when the input array is an array of arrays where each row is an array containing all the pixels of 1 image?
Hi, thank you so much for this series of tutorials! They're really detailed and very well explained! I've been through so many other videos and articles, and this series seems to be the most clear in terms of explanation of concepts and actual application in code! Thank you so much!
I'm currently working on an image classification project. I specifically want to do a comparison between the performances of models built using a traditional ML approach vs. deep learning (CNNs). As such, I do have 2 questions that I would like to ask you:
1. The features dataframe that you built for the classifier in this video is basically individual pixel values of various filtered forms from one single image. So, the entire dataframe is from one single image, and each row corresponds to one single pixel from that image. Is there any way I can apply this approach to a bunch of images (image classification instead of image segmentation)? Specifically, I would want to create a features dataframe from a bunch of images, where each row corresponds to one single image (instead of just one pixel), and the columns are still the various filtered forms of those images. I understand that in this case, each individual value in my feature dataframe will end up being an array of pixel values (instead of single pixel values like in your case), and a ML algorithm would not accept that as an input. Is there any way I could convert these arrays of values into a single value which can then be input into a classifier like SVM or RandomForest? Or is my approach totally wrong here?
2. I'm working in Google Colab so as to utilize the GPUs for my deep learning models. Does Tensorflow work with GPUs by default? Or is there a specific configuration/command that I need to run first before Tensorflow can switch over to using GPUs. (Sorry if you've answered this question in one of your other videos. I only discovered your channel 2 days ago, and haven't been able to go through all the videos yet!)
Apologies in advance for the verbosity of my questions! :D
1. Yes you can (and should) extract features from multiple images. For that, you just need to read one images after other and then add features to the dataframe. In other words, just concatenate values from multiple images into a single table / dataframe.
2. You can turn on GPU capability on Google Colab. Once you do that then it automatically uses GPU when you use tensorflow (or keras). Please search for this topic on UA-cam, there are numerous videos on how to enable GPU on Colab.
Great tutorial thanks Srini
Thanks for the useful information. Are you using only one image Sandstone_Versa000.tif to generate features using Gabor filter? What if we have a bunch of pictures?
this is done for single image...? my question is how to do for many images
You just need to read multiple images & masks instead of one. Then perform the same operations (extracting features) on all images and concatenate the data. In other words, repeat the process on multiple images and add those pixels as new rows to your Pandas DataFrame. The best way would be define a function to perform the feature extraction process and apply the function to multiple files and keep concatenating the data. I showed some of that work in part 5 video titled - '67 - Image Segmentation using traditional machine learning - Part5 Segmenting Images'.
Python for Microscopists for single image I should get 1*32 features ..but I'm getting n* 32 features for one image...by this operation..
@@janasstory4974 Not sure if I understand exact issue but if everything goes well you should see 32 Gabor features plus a few other features corresponding to Sobel, Prewitt, etc. for each labeled image. In the example I used you will have a total of 41 features (columns) and 1019904 pixels (rows) as my input image had dimensions of 1024 x 996. If you have 2 labeled images for training then you will have 2 * 1019904 rows but the number of columns should be the same as you are extracting same number of features for every image. If this is not the case then you may have to debug for logic errors - not fun exercise to do but can be a good learning experience.
In the dataset there is train_mask , How do you create a masked image of the original image?
Masked images are ground truth images that you need to generate yourself. There are companies here in the bay area that offer these services. They hire cheap labor from other countries to paint pixels in order to annotate them. I always wondered how these external non-technical people understand the application in order to annotate images. In any case, if you want to annotate your own images you can do it for free here: www.apeer.com/annotate
@@DigitalSreeni When exporting the labeled image from APEER Anotation, it exports a gray color image instead of a colored labeled image. Am I doing something wrong? Thanks in advance professor!
very thank you good job! sir
can you give the masked image for this that is the labeled image
Thanks for sharing the tutorial.Can you please provide dataset link and python file.
when I apply this gabor filters for face recognition, the label shouldnt be for each pixel, how can i add labels?
For face recognition you probably do not want to segment every pixel but rather classify the entire face. So you need to encode entire image with the face extracted to a value that you need to train and predict. In summary, you are looking for image classification rather tan segmentation. Also, search for VGG Face, you will find pre-trained models on celebrity faces.
I wish you wouldn't test on (pixels from) the train image itself
Humor level: 42 ;-)
can i use 2 or more photo of same data and put their data in 1 csv file in row, will it increase the model performance ?
Not sure what you are exactly looking for. If you are inquiring about using multiple images for training then I did a video on that, check out video 67b.
ua-cam.com/video/sD2xL36Xdu0/v-deo.html
Valeu!
Thank you very much Rafael.
my pc consumes all its ram memory when I execute the line: model.fit (X_train, y_train),
What can be caused this?
Obviously you are performing machine learning where the system is trying to fit a model based on the input data. You need to have high powered systems for any machine learning tasks.
@@DigitalSreeni Sure, but I also tried this code in collaboratory with 25gb of ram and it also consumed all the memory, maybe I need to change some parameter of the random forest function?
you could try another IDE, such as google colaboratory, it may help
Hi sir, Which is the best book to be referred to on this topic?
Not sure about a good book on this topic. Please look for machine learning books focused on computer vision. Hopefully you will find a good book that covers the topics of traditional machine learning (e.g., Random Forest, SVM, etc.)
@@DigitalSreeni any reference for pixel wise feature extraction?
@DigitalSreeni Sir my model is taking a lot of time to get trained .