Which frameworks would you recommend if you had to scale to +1000 models? I am looking at custom FastAPI and MLFlow with AWS Lambda, but where each inference request will load the model from object storage and call .predict. The models are generally lightweight and predictions would only have to be made on an hourly basis, so I don't think its necessary to serve them in memory.
Thank you for the video! I have a question: If I need to make updates to an existing service, do I have to go through the entire process again, or is there a more efficient way? Bentoctl build seems quite time-consuming. Appreciate your help!"
Appreciate your comment! If the change is inside of your ML model or the serving logic (service.py) you will have to rebuild the image. However, the second time around some layers should be cached (docs.docker.com/build/guide/layers/ ) so in theory it should be faster (it depends though). Another thing you can do is to build the image in some virtual machine rather than locally. A common setup is that you build it + upload to ECR in your CI (e.g. GitHub actions) Just some ideas:)
Hi, if you accept suggestions, can you look up into implementing something from H3, S4, S5, etc? Structured State Spaces occupy at least half of top10 architectures on LRA and there are about zero intuitive explanations of them.
hi, im getting this error: ""'sagemaker_service:svc' is not found in BentoML store , you may need to run `bentoml models pull` first'."" any idea ? Thnks a lot
Hmmm, if the problem still persists you can create an issue here: github.com/jankrepl/mildlyoverfitted/issues Describing exactly what you did and I can try to help!
Using VIM, Tmux and an audible keyboard never gets old!
hehe, agree:)
Thanks for the video man. there aren't many resources on bentoml so I appreciate your contribution. can you please at more in the future.
Appreciate your message:) Thank you! I will very likely do more BentoML related stuff in the future:)
Which frameworks would you recommend if you had to scale to +1000 models? I am looking at custom FastAPI and MLFlow with AWS Lambda, but where each inference request will load the model from object storage and call .predict. The models are generally lightweight and predictions would only have to be made on an hourly basis, so I don't think its necessary to serve them in memory.
If you are not experiencing a cold start (or you don't care) then Lambda is definitely a great solution:)
thank you for introducint BentoML ~~~~ it looks so nice
You're welcome 😊
Thank you for the video! I have a question: If I need to make updates to an existing service, do I have to go through the entire process again, or is there a more efficient way? Bentoctl build seems quite time-consuming. Appreciate your help!"
Appreciate your comment! If the change is inside of your ML model or the serving logic (service.py) you will have to rebuild the image. However, the second time around some layers should be cached (docs.docker.com/build/guide/layers/ ) so in theory it should be faster (it depends though). Another thing you can do is to build the image in some virtual machine rather than locally. A common setup is that you build it + upload to ECR in your CI (e.g. GitHub actions)
Just some ideas:)
Perfect as always
Great!
Amazing light 😁
Hi, if you accept suggestions, can you look up into implementing something from H3, S4, S5, etc? Structured State Spaces occupy at least half of top10 architectures on LRA and there are about zero intuitive explanations of them.
Hey there!
Actually, I never heard of those! I am adding it to my reading list:) Cannot promise I will make a video about them though:)
Thank you!
Thanks a lot for a content!
You are welcome!
hi man, do you offer some training or mentorship?
terminal and theme name please
tmux + gruvbox
hi, im getting this error:
""'sagemaker_service:svc' is not found in BentoML store , you may need to run `bentoml models pull` first'.""
any idea ? Thnks a lot
Hmmm, if the problem still persists you can create an issue here: github.com/jankrepl/mildlyoverfitted/issues
Describing exactly what you did and I can try to help!
@@mildlyoverfitted solved, i did It. The problem come with bentoml versión, i had install bentoml==1.1.11 this solve the problema for me