330 - Fine tuning Detectron2 for instance segmentation using custom data
Вставка
- Опубліковано 11 вер 2024
- This video tutorial explains the process of fine tuning Detectron2 for instance segmentation using custom data. It walks you through the entire process, from annotating your data, to training a model, to segmenting images, to measuring object morphological parameters, to exporting individual masks (results) as images for further processing.
Code generated in the video can be downloaded from here: github.com/bns...
All other code:
github.com/bns...
Detectron2 repo: github.com/fac...
Annotations were done using Makesense: www.makesense.ai/
Dataset from: leapmanlab.git...
Direct link to the dataset: www.dropbox.co...
Data courtesy of:
Guay, M.D., Emam, Z.A.S., Anderson, A.B. et al.
Dense cellular segmentation for EM using 2D-3D neural network ensembles. Sci Rep 11, 2561 (2021).
Data annotated for 4 classes:
1: Cell
2: Mitochondria
3: Alpha granule
4: Canalicular vessel
Hi 👋 I'm the creator of MakeSense! Thanks a lot for using the tool in your tutorial!
Nice to meet you. Thanks for creating a very useful tool :)
@SkalskiP. You have made a fantastic tool. Great job! I found a minor bug in it. I believe the coco-json structure has evolved. Under "annotations" it is expected to have a category_id, which your tool does not populate.
I honestly believe you have the best tutorial videos on youtube for vision ml. It's very thorough providing all the necessary information for somebody to work through it from scratch. Keep it up. Love these
The hardest part is annotating the images, I have done a lot of annotations before. You have done a good job by explaining annotations in detail which is very rare. Can you please share the type of applications that are in demand so that we can try out new things with the knowledge you give us here. Thanks for all the extra effort you put in to explain each line of code.
KING YOU ARE A KING!!!!!!!!THANK UUUUUUUUUUUUUUUUUUUUUUU U ARE THE BEST!!!!!!! I THINK U SHOULD GET AN AWARD. THANK YOU!!!
I love you Doc. I needed this. Thank you, DigitalSreeni!
Happy teacher's day bhaiya. You've made me fall in love with machine learning. Your videos are to the point and nicely explained. Thanks❤
Appreciate your tutorials a lot! Keep the good work going.
your resources have been incredibly helpful. thank you so much
Merci , super tuto , bien clair , bien explique en un mot "amazing"
thank you again for this great tutorial. You make amazing job for us. Detectron works a lot better for me then Yolo8 or Faster RCNN. I will play a bit more with parameters in future, but even with small dataset I was able to get reasonable result =)
Similar experience as mine. I too like Detectron2 results better and I find it to be more reliable compared to other instance segmentation frameworks available.
please make a tutorial on DETR instance segmentation. Your videos are really helpful.
A point you made at the very end of this video is very important. U-NET is not capable of segmenting overlapping objects. Would you think of making a video comparing between different models for segmentation or other computer vision tasks? Thank you!
23:17
I love coffee as well hahaha . Thanks sir for the content
Thank you very much for this helpful tutorial. God bless!
Thanks for this sir. I understand each and very thing very clearly. I have few questions. 1) - Why size of the model_final.pth is 334.2 MB and how we reduce size of model without loosing model accuracy and detection rate. 2) - For getting best accuracy and detection rate from detectron2 what should the minimum annotated labels per class and how many max classes we involve in single model. 3) - How, we check our model performance after training, is there any metrices and anything so that we can get rough understanding about model accuracy.
Could you provide the annotations and json files you used? I was not able to reproduce the results -- the accuracy on the test images was quite poor.
Great job. We will reference you. Keep impacting
Awesome, thank you!
Awesome explanation
Hey DigitalSreeni. Love the video!
Where can I access the .json files and .png images? I see tiff files in the dropbox but not the others
If unaccessible, do I need to create them myself?
Excellent Tutorial!
Thank you for such a great explanation. It would be very nice to complement the tutorial with the workflow to build on top of detectron2, i.e., to use the pooled feature map that the object detection and the masking model use to add a new module within the architecture.
Very nice content. But I think your link in description is not working well. Please fix it. Thank you!
If you are referring to the dataset link then try this link: www.dropbox.com/s/68yclbraqq1diza/platelet_data_1219.zip
Do you know how to adapt this process in order to work on images with multiple bands? For instance, for aerial images of homogeneus areas sometimes having just RGB images could not provide enough information and the accuracy of the segmentation could result too poor. Hence, to improve accuracy more bands (such as a Digital Terrain Model, or a slope band) can be used, since they provide more information for the segmentation of the object. Do you think something like this is feasible?
great video.
Thanks again. Do you know if it is possible to train mask r cnn using only bounding boxes that still performs the segmentation after training?
I appreciate your free tutorial. The problem is that I want to use the weights and the output of TensorFlow, but the result in the output folder is not like what I had in my YOLO training. I don't know how to find the weight folder and TensorFlow output images.
at 26:02, does initiate weight means exactly same as transfer learning ?
I looked at some notebooks in the Sartorius - Cell Instance Segmentation competition on Kaggle but people don't report quantifying metrics. It's kind of confusing.
Do people don't really care about the metrics like AP in segmentation tasks?
Thanks for sharing knowledge!
Again i cannot find the code link!!!!
You will find the code link now. There is a 4 to 5 hr gap between the video publishing time and me waking up and pushing the code to github. You caught me while I was still asleep :)
Thanks for the tutorial, do you know a reference to train instance segmentation using label mask only (same image size with original image) ? I have one class but I would like to differentiate different object.
Thank you for worderful lecture, but I get in trouble with dataset detached in description. I don't know how to convert it to COCO format to apply detectron2, I think data include image and label so I don't need to use makesense to label it manually. Give me some tips for using data provided.
Why do no results appear in the image after training with images with only 1 class? Of course I changed it to NUM.CLASSES = 1.
Thank you very much for this heloful tutorial. I have a question regarding the unites of area of each class- is it pixel?
hi there love the content can you please do a video for panoptic segmentation using are own dataset and please make it from scratch meaning how we get the instance segmentation mask and the semantic segmentation mask then how we combine them and run it on own own dataset.
It will be very helpful if you look into MMsegmentation framework. It's quite similar to this. But there is very few videos about this...😊
Hey, i watched your tutorial and it was very clear but i would love to use custom data format (like tiffs with more than 3 bands) and i find it very difficult to do
Hi Sir, I want to ask about the evaluation part, how to find performance metrics other than mAP, such as confusion matrix, and IOU? Find so many code online no show example with using detectron2 to find these.
same doubt bro, did u get it?
You didn't discussed about finetuning why the tile that you have kept
sir..can you show how to display result performance using tensorboard for this example .for total loss
if you start the training process ( trainer.train() ) inside the same block as default trainer, tensorboard should work
for d in random.sample(train_dataset_dicts, 2):
img = cv2.imread(d["file_name"])
visualizer = Visualizer(img[:, :, ::-1], metadata=train_metadata, scale=0.5)
vis = visualizer.draw_dataset_dict(d)
plt.imshow(vis.get_image()[:, :, ::-1])
plt.show()
I dont see the right labels, they are labeled as 1-12 in my dataset instead of the category names. Probably its a problem with my json but does anyone know a solution?
Hey , as i work on aerial imagery, i thinking of using detectron2 resnet101 model and create my own pretrained model as they used coco datatset for creating pretrained in .pkl format than fine tune the model according my needs how can i do that thank you helping 👍
@DigitalSreeni I have been thinking to fine-tune dataset on cell-lineage analysis for C .elegans??
Do you have the training data? Often times it comes down to data and knowledge to annotate.
Detectron2 is a Two-Stage-Detector right? Is it therefore better at detecting and segmenting filigree defects? And does the library mentioned offer the possibility for hyperparameter tuning as i sparsely see any content about it with regards to the topic of instance segmentation
I do not know what Filigree is so I cannot share my opinion based on any experience. But if you are trying to detect specific objects in an image then this approach will work for you. Regarding hyperparameter tuning, you can do it via adjusting the parameters in the config file.
@@DigitalSreeni Thanks a lot. Filigree = very small details :)
Thanks for educating me :) Learning something new every day :)
@@DigitalSreeni :D Back to my question, 2 stage detectors should be better for such details right?
thanks for your great tutorials!
are there any benefits using detectron2 with maskrcnn instead of, e.g, matterports maskrcnn?
Detectron2 makes it easy to access Mask R-CNN compared to matterport or other forms. Underneath, they are all using the same approach. So, no benefit when it comes to results but super beneficial when making the technology accessible.
@@DigitalSreeni ok i see.
very helpful, thanks alot!
Dear Sreeni, I love all of your videos. I have a specific use case. Is there any chance to connect with you.? I have tried in Linkedin, but unfortunately couldn't connect with you yet. :(
Thanks for sharing! I really needed exactly this. I cannot find the dataset. The link is not working
Can you try this direct link for the dataset, I hope it works for you (unless region-blocked for some reason): www.dropbox.com/s/68yclbraqq1diza/platelet_data_1219.zip
Is MakeSense safe to use? Any data security issues?
Not sure what you mean by safe. Please check their data policy. In general, if you are using any product or service for 'free', there is a good chance that your data will be used to improve their future offering. This can range from learning about how their product gets used to using customers' data to train their own DL models. Nothing is free in life!!!
thanks sir .🎉🎉
Most welcome
very goood explanation,how can i contact u?
thanks a lot sir, can you show us how to run it in python under windows environment?
for annotation tool lets use roboflow for auto segmentation process...🤚🤚🤚👍
You can try Roboflow on these images. Hopefully you will have better luck than I.
@@DigitalSreeniI tried using roboflow to annotate and apply your code for detection, but I experienced an error.
Sir, I need to try instance segmentation with Mask RCNN using my custom dataset could you help me with this.
That is exactly what this video is all about.
Sir, i have a separate annotation file for every single image in the form of .csv can i convert all those files into single .json file?
please make a video of detectron2 for panoptic segmentation...
Is it safe to use MakeSense?
Not sure what you mean by safe. Please check their data policy. In general, if you are using any product or service for 'free', there is a good chance that your data will be used to improve their future offering. This can range from learning about how their product gets used to using customers' data to train their own DL models. Nothing is free in life!!!
Can you make a video on how to perform virtual staining using deep learning?
Hopefully in future. Here is a link from my bookmarks. I saved it to test for future video but haven't found time yet. github.com/softmatterlab/live-apoptotic-virtual-staining
@@DigitalSreeni Thanks a lot for your response. I will go through the github link and see if I can crack it.
Sir please guide how to deploy custom model on web app thx you ^^
ua-cam.com/play/PLZsOBAyNTZwYSk07iwEbWFaDxeRw1NXt6.html&si=d8qH4MK-ACg6pNya
Thank you❤
What a pain to install it on Windows...
were you able to do it?
trainer.train() time error results like RuntimeError Traceback (most recent call last)
in ()
----> 1 trainer.train() #Start the training process
13 frames
/content/detectron2/detectron2/layers/wrappers.py in nonzero_tuple(x)
151 return x.nonzero().unbind(1)
152 else:
--> 153 return x.nonzero(as_tuple=True)
154
155
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Reason for the error??