@@mrbot4onehello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
Hi and Thank you for this video, for me it’s so informative at my (very low) level. Also, GO SPURS! Question for Peter: In this code, “milk bottle” is what? Is it words that will be AI translated into a bounding box? Then “bottle” is what? … the instantiating and assignment to a class? ontology=CaptionOntology({"milk bottle": "bottle", "blue cap": "cap"}). Asked another way, if I had a bunch of images of my dog, could I say =CaptionOntology({“dog”:”My_Dog_Trudy”})? Would that lead to a model that had a decent chance of identifying her with a bounding box in video of her at a dog park?
After executing the code, the 'confidence.txt' file is not being created in the specified output folder (output_folder). I have checked the directory structure, file extensions, and paths, but the problem persists. Any assistance in identifying the issue would be appreciated. after using this part of code im facing the following problem: from autodistill_grounded_sam import GroundedSAM base_model = GroundedSAM(ontology=ontology) dataset = base_model.label( input_folder=IMAGE_DIR_PATH, extension=".png", output_folder=DATASET_DIR_PATH) FileNotFoundError: [Errno 2] No such file or directory: '/content/dataset/annotations/confidence-3_result.txt'
Hey Roboflow team. We are still waiting for the method to work on the annotated images. When we annotate the image from Autodistill then we have no method to change or update the wrong annotations . Please provide a solution for that. Thank You.
yes they do not. But we are still waiting for the method to work on the annotated images. When we annotate the image from Autodistill then we have no method to change or update the wrong annotations . Please provide a solution for that. Thank You.@@Roboflow
looks promising! so if I understand correctly - there are two phases: (1) auto-label an image dataset, something that you already showed in the G-SAM previous video (2) split the dataset to test/train/val and build a CV model with YOLO-8 q1: since auto-labeling is not 100% fullproof, what about the human in the loop? q2: what if the base model doesn't recognize the labels (name of specific boats I have in a marine dataset), how can I fine-tune it? so the model will find a boat, but will also "know" the brand of the boat
Overall yes. But it doesn’t need to be G-SAM and YOLOv8. You could use other base and target models. You could upload your auto labeled datasets to Roboflow for human refinement and download it back to your environment for second stage. If the base model won’t detect objects you want to find you should try some prompt engineering. But there are definitely types of objects that we won’t be able to auto annotate. Good thing is that with every base model autodistill will become more powerful.
Great Video! Just wonder after deploying to Roboflow, how to verify the roboflow evaluation metrics like mAP, Precision and Recall with the one in google colab?
hi , i need a help , i need to detect shape like circle ,triangle ,rectangle in image and check all shaope are correctly drown are not , and guide me I try all shape detection model but unable tom find solution ,
If I want to train a model, to detect and classify very specific objects, of which the foundation model doesn't have any labels (like on a highway senario, I don't only want to detect vehicles, like in a category car, but I also want to tell the brand of the car, or even more specific the model of the car.) would it still work? Can we teach a model with that complex classes without annotation?
Hi everyone, you're great! Just one question, is it possible to use this "technique" to create custom models, or for the recognition of particular objects not present in the basic ones recognized by yolo? Thanks so much in advance for your reply and help!
How precise is the annotation? Like...for example if I want to label 3 types of insects...does it ricognize the type or will i have just a general 'insect' label?
Hi 👋🏻 It is Peter from the video. You should always try and test, but my assumption is that it won’t be able to distinguish between different types of insects. They will semantically be to close to each others but insect class sound very reasonable.
For each insect, can't you take a video, and then adjust the CaptionOntology? I guess this would create 3 models, but maybe they could be combined or something?
I am working on project to detect a car model in real time i train 5 images but i didn't know whether they are trained well or not if they are than how to keep that data and detect car model in real time i so confuse how does this happen when i install yolo v8 it already has some in-built data detection in realtime than how can i put my own custom data in that classes
how can i finetune the grounding sam model in this setup? i got some quite good results on my first attempts - but i think it can do better! thanks for the awesome tutorial!
You can’t fine tune G-SAM at least not without multiple GPU setup. But if you got good enough result with your first try. I thing best idea is to use your result YOLOv8 model to auto annotate more data and use it for training.
hi, thank you for the helpful information. truly appreciate it. i have a question, is there any way for the autodistill to only return the rectangle bounding box value and not a polygon?
Hello Hero,,, I hope you are well Is Grounded_Segmeny_Anything working on a night mode or just images within day mode Finally thanks for sharing your knowledge with us
I tried it with detecting people on night vision footage and it works. Best idea is to try and test :) I’m actually super curious if it will work for you.
Thank you for sharing. I have question about results of YOLOv8 model , after the training of the model it results in a 3-dimensional confusion matrix taking the background as a class knowing that I have a binary classification my project is classification of preforms whether it is defect or not. What can be the raisaon of appearance of this class "backgroud" and how I can solve it ? if you help me I would be grateful.
Combine this with yolo as-one and it’s pretty easy to see how different parts of a larger process can be combined into an overall framework. Could be a pipeline framework, could be something else. Probably already being worked on if not released
Does this framework support anything? Like galaxy object detection? For example, I have Galaxy images(about 300k) and the objects visible in the image are stars and different types of galaxies. My objective is to classify galaxies into two class. Is it possible to auto annotate?
Only if they are in caption ontology. only those models if they relate to the base model. But if you do the labeling of your images and use as a base model for others it will be ground breaking
hello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
Great video ! It helped me annotate my dataset from scratch Question : does the G-SAM annotation make a big difference compared to annotation with G-DINO ( without segmentation) ?
hello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
@@Roboflow if the process is very long and I need to stop in the middle to continue later, is there a way to do that or it is an atomic only operation?
Nice Video Big Fan of your videos can you make another on Football and continue from where you left like team segmentation and count the number of passes and shoots. It would be a big help
Small request for next video: data collection -> autolabel -> load to roboflow for manual correction -> train -> compare accuracy with today's result
Hi 👋🏻 It is Peter from Roboflow. So comparing automatically build model with the one trained on manually annotated data?
@@Roboflow compare manually is OK
@@Roboflow use autodistill to label the data then check it manually for the correct anotations them train with yolov8
@@mrbot4onehello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
@@SAITAMA-07-13 Use YOLO for your purpose
Hi and Thank you for this video, for me it’s so informative at my (very low) level. Also, GO SPURS! Question for Peter: In this code, “milk bottle” is what? Is it words that will be AI translated into a bounding box? Then “bottle” is what? … the instantiating and assignment to a class?
ontology=CaptionOntology({"milk bottle": "bottle", "blue cap": "cap"}). Asked another way, if I had a bunch of images of my dog, could I say =CaptionOntology({“dog”:”My_Dog_Trudy”})? Would that lead to a model that had a decent chance of identifying her with a bounding box in video of her at a dog park?
After executing the code, the 'confidence.txt' file is not being created in the specified output folder (output_folder). I have checked the directory structure, file extensions, and paths, but the problem persists. Any assistance in identifying the issue would be appreciated.
after using this part of code im facing the following problem:
from autodistill_grounded_sam import GroundedSAM
base_model = GroundedSAM(ontology=ontology)
dataset = base_model.label(
input_folder=IMAGE_DIR_PATH,
extension=".png",
output_folder=DATASET_DIR_PATH)
FileNotFoundError: [Errno 2] No such file or directory: '/content/dataset/annotations/confidence-3_result.txt'
Same problem here! Anybody know the resolution?
Hey Roboflow team. We are still waiting for the method to work on the annotated images. When we annotate the image from Autodistill then we have no method to change or update the wrong annotations . Please provide a solution for that. Thank You.
^^ is there a solution for this yet ? Need to correct and tweak the auto-labeling
No brother @Roboflow team is not replying in any of the video @@adhr1t
Requesting again for the update @roboflow @peterskalski
Well done Roboflow, I am glad a distillation video worked out, i wasnt disappointed!!
Thanks a lot. I hope our videos - in general - do not disappoint. Haha
yes they do not. But we are still waiting for the method to work on the annotated images. When we annotate the image from Autodistill then we have no method to change or update the wrong annotations . Please provide a solution for that. Thank You.@@Roboflow
looks promising! so if I understand correctly - there are two phases: (1) auto-label an image dataset, something that you already showed in the G-SAM previous video (2) split the dataset to test/train/val and build a CV model with YOLO-8
q1: since auto-labeling is not 100% fullproof, what about the human in the loop?
q2: what if the base model doesn't recognize the labels (name of specific boats I have in a marine dataset), how can I fine-tune it? so the model will find a boat, but will also "know" the brand of the boat
Overall yes. But it doesn’t need to be G-SAM and YOLOv8. You could use other base and target models.
You could upload your auto labeled datasets to Roboflow for human refinement and download it back to your environment for second stage.
If the base model won’t detect objects you want to find you should try some prompt engineering. But there are definitely types of objects that we won’t be able to auto annotate. Good thing is that with every base model autodistill will become more powerful.
Great Video! Just wonder after deploying to Roboflow, how to verify the roboflow evaluation metrics like mAP, Precision and Recall with the one in google colab?
hi , i need a help , i need to detect shape like circle ,triangle ,rectangle in image and check all shaope are correctly drown are not , and guide me I try all shape detection model but unable tom find solution ,
Thanks for sharing the autodistill library. This is awesome. The era of labelling is gone!
Not yet. But we are definitely moving into that direction.
If I want to train a model, to detect and classify very specific objects, of which the foundation model doesn't have any labels (like on a highway senario, I don't only want to detect vehicles, like in a category car, but I also want to tell the brand of the car, or even more specific the model of the car.) would it still work? Can we teach a model with that complex classes without annotation?
No I’m afraid it won’t work. That query is to specific. Zero shot detections are not as powerful.
Can I use it for segmentstion task ?
Yes! Labels we generated are actually bounding boxes + polygons! You can use YOLOv8 to train both object detection and instance segmentation models.
Hi everyone, you're great!
Just one question, is it possible to use this "technique" to create custom models, or for the recognition of particular objects not present in the basic ones recognized by yolo?
Thanks so much in advance for your reply and help!
Yup! That’s pretty much the idea behind autodistill.
@Roboflow thank you very much, very kind!
You're the best. I love you. You've made a significant impact on computer vision tasks.
Thanks a lot! 🙏🏻 This complement goes rather to the whole RF team. I just recorded video.
Thanks for so much useful information, BTW is it also possible to classify drones without any labeling like this video?
Thanks a lot! You intend to distinguish between drone types or just detect them on photo?
How precise is the annotation? Like...for example if I want to label 3 types of insects...does it ricognize the type or will i have just a general 'insect' label?
Hi 👋🏻 It is Peter from the video. You should always try and test, but my assumption is that it won’t be able to distinguish between different types of insects. They will semantically be to close to each others but insect class sound very reasonable.
For each insect, can't you take a video, and then adjust the CaptionOntology? I guess this would create 3 models, but maybe they could be combined or something?
I am working on project to detect a car model in real time i train 5 images but i didn't know whether they are trained well or not if they are than how to keep that data and detect car model in real time i so confuse how does this happen when i install yolo v8 it already has some in-built data detection in realtime than how can i put my own custom data in that classes
I have 35 images to label but it is only labelling 1
how can i finetune the grounding sam model in this setup?
i got some quite good results on my first attempts - but i think it can do better!
thanks for the awesome tutorial!
You can’t fine tune G-SAM at least not without multiple GPU setup. But if you got good enough result with your first try. I thing best idea is to use your result YOLOv8 model to auto annotate more data and use it for training.
@@Roboflow thanks alot!
hi, thank you for the helpful information. truly appreciate it. i have a question, is there any way for the autodistill to only return the rectangle bounding box value and not a polygon?
Thnx for the content, Can I know how to open webcam in yolov7 method in colab I try hard but I get nothing thanks again.
Great as always! I have a question. How can I use my relatively good custom segmentation model to annotate more images in yolov8 forrmat?
Supervision could help with that. We have a ton of utilities that can make that easier.
@@Roboflow Thanks, I'll check it out.
This does not work for custom datasets.
Is there a smaller version of Yolov8 that can be trained on the ade-20k dataset?
Hello Hero,,,
I hope you are well
Is Grounded_Segmeny_Anything working on a night mode or just images within day mode
Finally thanks for sharing your knowledge with us
I tried it with detecting people on night vision footage and it works. Best idea is to try and test :) I’m actually super curious if it will work for you.
@@Roboflow I am trying to detect parking lane but unfortunately didn't work
@@sulaimanmahmoud7120 parking lane in night vision can be hard. :/ What is your image resolution?
It's high resolution
Width : 2592
Height: 1944
Thank you for sharing. I have question about results of YOLOv8 model , after the training of the model it results in a 3-dimensional confusion matrix taking the background as a class knowing that I have a binary classification my project is classification of preforms whether it is defect or not. What can be the raisaon of appearance of this class "backgroud" and how I can solve it ? if you help me I would be grateful.
You can use pseudo labeling and data augmentation. No extra models are necessary. Create your own flow.
Combine this with yolo as-one and it’s pretty easy to see how different parts of a larger process can be combined into an overall framework. Could be a pipeline framework, could be something else. Probably already being worked on if not released
Yeah it is awesome to se powerful open source projects being developed 🔥
this is for any object and i only need to load the image and ontology or is it only the objects contained in yolo coconame that can label
thanks for saving tons of hours !!
That’s what I wanted to hear :)
Is it possible to make your own custom base model like Grounding DINO?
Thanks Mate ! May I know the reason why are you choosing to use Yolov8 but not Yolo-NAS ?
Does this framework support anything? Like galaxy object detection? For example, I have Galaxy images(about 300k) and the objects visible in the image are stars and different types of galaxies. My objective is to classify galaxies into two class. Is it possible to auto annotate?
Only if they are in caption ontology. only those models if they relate to the base model. But if you do the labeling of your images and use as a base model for others it will be ground breaking
can I use it with tensorflow object detection or does it only work with yolo?
Any specific detector that you are interested?
@@Roboflow I’m working on a model using faster rcnn and I have 200k+ Images to label
why aren't the bottles and caps segmented in the video?
Is it possible to auto annotate ants using this.?
Depending on how large are the ants on the image?
Nice!
hello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
What model you want to train?
@@Roboflow I want to train multiple cars dataset like BMW Mercedes lamborghini and it should be recognized through live web camp
Great video ! It helped me annotate my dataset from scratch
Question : does the G-SAM annotation make a big difference compared to annotation with G-DINO ( without segmentation) ?
hello I am working on project like it will detect the car model in real time but I search a lot I didn't find how to train model and test it can you please suggest me it will be big help
Is there incremental auto distillation?
Could you elaborate please?
@@Roboflow if the process is very long and I need to stop in the middle to continue later, is there a way to do that or it is an atomic only operation?
Please, is there a way to save the labels and also determine the percentage of the train dataset versus the validation model?
Labels are automatically saving in the folder in train and valid folder
Nice Video Big Fan of your videos can you make another on Football and continue from where you left like team segmentation and count the number of passes and shoots. It would be a big help
I’d love to do it. We have large backlog of video ideas. I’m afraid I can’t promise anything but maybe I’ll do it in my free time.
@Roboflow Thanks for your honest reply
I was disappointed by the poor data annotation
Is it possible to change yolov8 backbone?
Yes
This is epic
Agree!
Not everyone wants to use Python
:) you can definitely deploy those models with other languages
How can I use this if I have two classes or more ?
Hi 👋🏻 It is Peter from the video. We have two classes in our example.
@@Roboflow can it be used for more than two
Nicely explained @Piotr and awesome content @Roboflow
Thanks a lot 🙏🏻