Wow, there is not even a single end to end video on the internet. You made a complete package Thank you very much man. You earned my subscription, like and a share.
just copy the following to copy the model from the artifacts folder to the model folder, destination_directory="model" file_path=r"artifacts/training/model.h5" if not os.path.exists(destination_directory): os.makedirs(destination_directory) shutil.copy(file_path, destination_directory) print("Model copied to Model directory")
If you could do end to end project like focusing on one cloud like AWS and the best practices of mlops with aws in the same way best practices of mlops with azure will really help everyone in their realtime projects.
Hey You are doing some amazing work here , one quick sugesstion please be a little slow while explaing the code .I have subscribed your channel after seeing your vedios on the other channel .
Excellent tutorial in modular approach Can we consider the components explained here are containers for k8s when using kubeflow or deployKF ? Because if it is that way then the components should be independent (ex microservices) for CI or CD Here the components seems to be dependent on each other like pipeline process
Thanks for your efforts in this video, it helps me a lot with my capstone project. In the future, could you please teach us an end to end project using AWS sagemaker?
@@dswithbappy can you please upload videos on NLP LLM projects with deployment using open AI API.. projects like text summarization( summerizing conversation of customer with service resolver), and text classification,
while running main.py module to check logger will work or not, below error is coming. How to solve it and what is the reson sir? PermissionError: [Errno 13] Permission denied: 'D:\\project\\entbappy(Bappy Ahmed)\\End-to-End-Chest-Cancer-Classification-using-MLflow-DVC\\logs\ unning_logs.log'
Hi, thank you for the video, although I'm on early part of the video, but I'm curious about two things, are we using fastapi in this project, and second are we using something called feature pipeline in it, I'm new in mlops, so i want to understand what is feature pipeline, ingestion pipeline. Also, I guess this video considers all those things that I wanted to learn such as DVC, MLFLOW, DOCKERISATION , BIG THANKS
Dear Bappy sir after everything when using dvc repro I am getting ModuleNotFoundError showing I dont have module named 'src', please help me to rectify the error
I'm getting the following error. Error when deserializing class 'InputLayer' using config={'batch_shape': [None, 224, 224, 3], 'dtype': 'float32', 'sparse': False, 'name': 'input_layer'}. Exception encountered: Unrecognized keyword arguments: ['batch_shape']. Please help me in sorting this when loading the odel in model training
ভাইয়া, আমি basic numpy, pandas, matplotlib, seaborn, Machine learning supervization... এখনো পর্যন্ত করছি। এখন কি করব বুঝতেছিনা,,,industry based learning hcche na mne hy...but kun path e jabo?.... Btw apnar shondhan pelam...i was like emn gem Bangladesh e ase...you need to get highlighted... ❤❤❤❤
hello! Stages have been deprecated in the new Model Registry UI. It seems they are changin the way how to handle this in the new mlflow version. do you know how to work with aliases and tags?
Hi Bappy, When I run my 04_model_evaluation.ipynb file (from pipeline module) the version run is getting logged and it is visible on mlfllow ui. But when I run the pipeline from main.py, the new model version is getting but it is not showing me the new version run on mlflow ui. What could be the problem?
Please help I'm getting module not found for src/cnnClassifier when running dvc repro dont know why the code was working fine with dvc but when running dvc repro it says no module found src/cnnClassifier...
what if i train the model on google colab (because i need gpu) and then i save everything locally but this way i don't get an conda.yaml file in my mlruns, is that a problem? also please launch a paid course where you deploy heavy tensorflow projects (cv or nlp, large datasets) using mlflow, dvc and dagshub and github-actions
anybody having issue with this: 'export' is not recognized as an internal or external command, operable program or batch file. if yes,how you resolved it.
(UPDATE) Hi Bappy, I got an error towards the end of the MLflow with Dagshub part (of the basic Linear Regression model). when I executed python example.py, I got this Error: mlflow.exceptions.MlflowException: API request to endpoint /api/2.0/mlflow/runs/update failed with error code 403 != 200. Response body: '' (Update): Use git bash terminal instead of of windows powershell (from VS code). Bappy's pc has his default terminal as git bash so he didn't have to do it.
@@priyanshukumarsingh235 use git bash terminal instead of windows powershell while running the python example.py command . This was told in one of Krish Naik's videos and it worked for me.
Hi, I have one doubt , here is it layer.trainable = False ? @staticmethod def _prepare_full_model(model, classes, freeze_all, freeze_till, learning_rate): if freeze_all: for layer in model.layers: model.trainable = False elif (freeze_till is not None) and (freeze_till > 0): for layer in model.layers[:-freeze_till]: model.trainable = False
sir in the last after completing the notebook 4 when we execute after completing the modular code after model evaluation, in my pc it is saving a file called mlruns instead of getting connected to mlflow. pls help @4:11:00 onwards
hi @DSwithBappy I am getting error regarding parameter from mlflow raise RestException(json.loads(response.text)) mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Response: {'error_code': 'INVALID_PARAMETER_VALUE'} though it was running properly first and tracking experiment but know it is not tracking experiment and showing this error. though i have not done any changes in params files. can anybody get this issue let me know.
You keep repeating the same end to end projects structure; create videos with other tools like kubeflow involved; using apis for real time data, other tools like kafka e.t.c,
Bappy bro he will post all unwanted comments take your time to use kubeflow Kafka, ansible, Prometheus you are excellent bro none of the youtubers have implemented modular coding except you which is highly appreciated
ভাইয়া, আমি basic numpy, pandas, matplotlib, seaborn, Machine learning supervization... এখনো পর্যন্ত করছি। এখন কি করব বুঝতেছিনা,,,industry based learning hcche na mne hy...but kun path e jabo?.... Btw apnar shondhan pelam...i was like emn gem Bangladesh e ase...you need to get highlighted... ❤❤❤❤
This guy is so so underrated. An eloquent Data Scientist ❤️
I deeply appreciate the well-structured project and video you've provided. Thank you from the bottom of my heart💖
You are so welcome!
Wow, there is not even a single end to end video on the internet. You made a complete package Thank you very much man. You earned my subscription, like and a share.
Amazing, one of the very first , easy to understand , e2e DL model deployment
Will it charge us to deploy and do same thing like him while ofcourse deleting services being used later?
just copy the following to copy the model from the artifacts folder to the model folder,
destination_directory="model"
file_path=r"artifacts/training/model.h5"
if not os.path.exists(destination_directory):
os.makedirs(destination_directory)
shutil.copy(file_path, destination_directory)
print("Model copied to Model directory")
You have done a great job. I really admire your knowledge and how proficient you are in all and every details of an ML project.
You deserve my subscription, please make videos with details about MLops like that. Happy New Year!
Please teach us GitHub actions from basics. I'm expecting you would do that soon for us😅.
thank you , till 3 hours 41 mins.....looking forward to complete and learn more
If you could do end to end project like focusing on one cloud like AWS and the best practices of mlops with aws in the same way best practices of mlops with azure will really help everyone in their realtime projects.
Love it. Stay tuned my brother.
learnt everything at one go
Very nice approach! Keep up the good work!
Hey You are doing some amazing work here , one quick sugesstion please be a little slow while explaing the code .I have subscribed your channel after seeing your vedios on the other channel .
Excellent tutorial in modular approach
Can we consider the components explained here are containers for k8s when using kubeflow or deployKF ?
Because if it is that way then the components should be independent (ex microservices) for CI or CD
Here the components seems to be dependent on each other like pipeline process
your hardwork is appericated❤❤
Thank you!
Thank you so much Bappy 😀
Amazing playlist. Thanks so much.
Thank you sir!! I really appreciate your effort to make this video for us!!!
It's my pleasure
Well Defined , well-structured project 🥰
Thank you!
Great work. Thanks Buppy. Keep going. Full support
Thanks :)
Learned a lot thank you!
Thanks for your efforts in this video, it helps me a lot with my capstone project. In the future, could you please teach us an end to end project using AWS sagemaker?
Thank you Bappy bhaiii
Very nice explanation❤
Thank You.
Man how someone can go such indepth to explain and make understand the things easily with baisc things..
Really appreciated🎉
thanks :)
@@dswithbappy can you please upload videos on NLP LLM projects with deployment using open AI API..
projects like text summarization( summerizing conversation of customer with service resolver),
and text classification,
very well explained.
Thanks bappy.
Very Helpful, Thanks a lot ❤
This is super. Thanks
Welcome
Thank you sir for this project
Thank you!
why the model is predicting normal and not predicting another class (adenocarcinoma) while testing in app.py(ui).
Thanks a ton❤
my model is predicting ever image as cancer image,,,why it is happening,,anyone who knows ,please help
getting this error at the time stamp 3:21:46
ImportError Traceback (most recent call last)
Cell In[11], line 8
6 prepare_base_model.update_base_model()
7 except Exception as e:
----> 8 raise e
Cell In[11], line 5
3 prepare_base_model_config = config.get_prepare_base_model_config()
4 prepare_base_model = PrepareBaseModel(config=prepare_base_model_config)
----> 5 prepare_base_model.get_base_model()
6 prepare_base_model.update_base_model()
7 except Exception as e:
bro did u solve the error n finished project by now?
thank you till 2 hours 42 min
In dagshub i could not find username, password and url instead I am getting some other code. How can I solve sir could you please tell me
thank you soo much
while running main.py module to check logger will work or not, below error is coming. How to solve it and what is the reson sir?
PermissionError: [Errno 13] Permission denied: 'D:\\project\\entbappy(Bappy Ahmed)\\End-to-End-Chest-Cancer-Classification-using-MLflow-DVC\\logs\
unning_logs.log'
thank you till 1 hour 3 min
Hi, thank you for the video, although I'm on early part of the video, but I'm curious about two things, are we using fastapi in this project, and second are we using something called feature pipeline in it, I'm new in mlops, so i want to understand what is feature pipeline, ingestion pipeline. Also, I guess this video considers all those things that I wanted to learn such as DVC, MLFLOW, DOCKERISATION , BIG THANKS
make NLP projects with mlops practices
Dear Bappy sir after everything when using dvc repro I am getting ModuleNotFoundError showing I dont have module named 'src', please help me to rectify the error
I have been getting same error. Main.py runs well but DVC repro doesn't. Have you been able to solve it?
I'm getting the following error.
Error when deserializing class 'InputLayer' using config={'batch_shape': [None, 224, 224, 3], 'dtype': 'float32', 'sparse': False, 'name': 'input_layer'}. Exception encountered: Unrecognized keyword arguments: ['batch_shape']. Please help me in sorting this when loading the odel in model training
please deploy for edge device
ভাইয়া, আমি basic numpy, pandas, matplotlib, seaborn, Machine learning supervization... এখনো পর্যন্ত করছি। এখন কি করব বুঝতেছিনা,,,industry based learning hcche na mne hy...but kun path e jabo?....
Btw apnar shondhan pelam...i was like emn gem Bangladesh e ase...you need to get highlighted... ❤❤❤❤
Apnar learning path ta blle valk hy....like kisher pr ki shikhsen..,
Your great😎
😀
this is really good one. however, if you want to bring the new model which is in stage how do i do it?
help me solve this error .
the model is predicting all images as normal. Why it might be happening?
same issue with me
mate can we not use something like react for the frontend bits , it'l be much more pretty with it .
yes we can
Very good work bro can you give the dataset you used?
hello! Stages have been deprecated in the new Model Registry UI. It seems they are changin the way how to handle this in the new mlflow version. do you know how to work with aliases and tags?
AWS account is not free😢
Hi Bappy, When I run my 04_model_evaluation.ipynb file (from pipeline module) the version run is getting logged and it is visible on mlfllow ui. But when I run the pipeline from main.py, the new model version is getting but it is not showing me the new version run on mlflow ui. What could be the problem?
having the same issue...pls help to fix this @DSwithBappy
@@ajee1801 remove the commented part and run #evaluation.log_into_mlflow()
Uncomment the line number 20 in `cnnClassifier/pipeline
/stage_04_model_evaluation.py`
Same
@@pr0t0n86 Still not working
Please help I'm getting module not found for src/cnnClassifier when running dvc repro dont know why the code was working fine with dvc but when running dvc repro it says no module found src/cnnClassifier...
Yes same thing happens with me too....
what if i train the model on google colab (because i need gpu) and then i save everything locally but this way i don't get an conda.yaml file in my mlruns, is that a problem?
also please launch a paid course where you deploy heavy tensorflow projects (cv or nlp, large datasets) using mlflow, dvc and dagshub and github-actions
anybody having issue with this:
'export' is not recognized as an internal or external command, operable program or batch file. if yes,how you resolved it.
ello mate , perhpas ur using windows. In windows u have to use 'set' instead of export
@@haz5248 thank you so much.
if you are using terminal within the VScode export command will not work, use ''set" instead of expoprt
are there any prerequisites for this project bro. can you please suggest if any required.?
Python & basic DL
WHAT ALL EXTENSION YOU USE IN THE VSCODE?
PLEASE SHARE.
Hi thank you for the project, I have a question. You are making a binary classification but what should we do to make it multiclass classification?
just change the output layers , select the neuron with respect to the classes you have
use softmax as output layer
During running dvc repro. । got an error that output : artifacts/prepare_base_model could not exist
@dswithbappy Please solve my problem
(UPDATE)
Hi Bappy, I got an error towards the end of the MLflow with Dagshub part (of the basic Linear Regression model).
when I executed python example.py, I got this Error:
mlflow.exceptions.MlflowException: API request to endpoint /api/2.0/mlflow/runs/update failed with error code 403 != 200. Response body: ''
(Update): Use git bash terminal instead of of windows powershell (from VS code). Bappy's pc has his default terminal as git bash so he didn't have to do it.
did you find the solution?
@@priyanshukumarsingh235 use git bash terminal instead of windows powershell while running the python example.py command . This was told in one of Krish Naik's videos and it worked for me.
Hi, I have one doubt , here is it layer.trainable = False ?
@staticmethod
def _prepare_full_model(model, classes, freeze_all, freeze_till, learning_rate):
if freeze_all:
for layer in model.layers:
model.trainable = False
elif (freeze_till is not None) and (freeze_till > 0):
for layer in model.layers[:-freeze_till]:
model.trainable = False
that's transfer learning process
upload mlops more projects
Could you kindly share the link to the dataset, please?
link is there in config.yaml file
Got it thank you@@dswithbappy
gdown has issues,anyone experienced this issue
i am getting error for the library from box import configbox please any one can help me to resolve
downgrade your python to 3.8 from your conda environment.
Wow
please share your dataset images
Hindi mlops course ❤
sir in the last after completing the notebook 4 when we execute after completing the modular code after model evaluation, in my pc it is saving a file called mlruns instead of getting connected to mlflow. pls help @4:11:00 onwards
Bro did you find any solution for this issue? I am facing the same issue
same
Nope @sagar
all images are predicted as normal image 😪
Same thing happened here
Same thing happened here
@@prithwishghosh1449 try checking with many random images ,only a few gets predicted correctly
hi @DSwithBappy
I am getting error regarding parameter from mlflow
raise RestException(json.loads(response.text))
mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Response: {'error_code': 'INVALID_PARAMETER_VALUE'}
though it was running properly first and tracking experiment but know it is not tracking experiment and showing this error.
though i have not done any changes in params files.
can anybody get this issue let me know.
OK?
You keep repeating the same end to end projects structure; create videos with other tools like kubeflow involved; using apis for real time data, other tools like kafka e.t.c,
its in pipeline ,, I have plan to do
@@dswithbappy Cool,, you're channel will blow up, If you do something different from other UA-camrs, just add those spice,, All the best,and great job
Bappy bro he will post all unwanted comments take your time to use kubeflow Kafka, ansible, Prometheus you are excellent bro none of the youtubers have implemented modular coding except you which is highly appreciated
dear bappi sir ,please provide link to your dataset in this example🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏☝☝☝☝🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏🙏
available in config.yaml
bondho
ভাইয়া, আমি basic numpy, pandas, matplotlib, seaborn, Machine learning supervization... এখনো পর্যন্ত করছি। এখন কি করব বুঝতেছিনা,,,industry based learning hcche na mne hy...but kun path e jabo?....
Btw apnar shondhan pelam...i was like emn gem Bangladesh e ase...you need to get highlighted... ❤❤❤❤
How much did it cost you to deploy it on AWS?