Excellent video! Would be interesting to see these frameworks, but within LightningAI Studios. Also, I saw CrewAI will be having a more golden standard approach to their code structuring in the near future.
Thank you! And you're definitely right about CrewAI moving towards YAML. When CrewAI+ drops, I plan on making a lot more content around this new format for you guys! And, I haven't tried out LightningAI Studio yet so I'll definitely have to try it out this weekend. Thanks for the suggestion!
@@bhancock_ai Great! I like the YAML approach. William Falcon that started LightningAI (PyTorch Lightning) likes my posts on LinkedIn as I'm a huge fan in developing with it when I mention them. Will be studying your approach with the latest updates and hopefully with their Studio. Thanks!!
@@bhancock_ai /llama3-crewai is this automate-youtube-with-crewai? or crewai-updated-tutorial-hierarchical or crew-ai-crash-course or nextjs-crewai-basic-tutorial or crew-ai-local-llm or crewai-groq-tutorial git hub I can't work out what file relates to this video?
Good luck getting local, quantized models to reliably function call, or use any kind of 'tool'. They need so much more supervision, which is where frameworks like langgraph can help, rather than crew
With both the Groq 8b and 70b with crew max_rpm set at both 1 or 2 I do get it halting for a while with: [INFO]: Max RPM reached, waiting for next minute to start.
The problem is that Groq is so fast that it ends up processing too many tokens so it ends up hitting a rate limit and failing. To get around that, we have to slow down our crew by setting the max RPM. Feel free to bump it up to get your crew to move faster!
Hey Brandon great video by the way. There seems to be an error as such. It seems we encountered an unexpected error while trying to use the tool. This was the error: Invalid json output: Based on the provided text, a valid output schema for the tool is: { "tool_name": str, "arguments": { "query": str } } This schema defines two keys: `tool_name` which should be a string, and `arguments` which should be a dictionary containing one key-value pair. The key in this case is `query`, with the value being another string. 'str' object has no attribute 'tool_name'
That is using a very old version of CrewAI - if you run it with the current version of CrewAI it fails because of lack of expected_output parameter in the Tasks
Great stuff. Maybe a silly question, but when it was fetching to read data from remote website (the analysis part), does it read it remotely OR does it capture screenshots & download text to feed into its prompt and then clear this cached data or such local cached data needs to be cleaned eventually? Hope it simply reads remotely without too much data saved locally as I plan to use this approach to analyze many websites without flooding my local storage.
Great content as always! Do you know if it's sustainable to use a single groqcloud API key to host LLM access for a multi-user app? Or would a service like AWS Sagemaker be better for simultaneous users? Cheers!
cant get the tools.search_tools module to run says not foun, ive trued pip install command or just install command with tools name, no luck any ideas please
Thank you for this and for the code.. How does Llama 3 compare to Dolphin-Mistral 2.8 running locally as the more junior agents do you know? Dolphin-Mistral with its extra conversatuon/coding training and bigger 32k context window appeals! Ive had agents go round in circles creating nonsense with other frameworks as they dont remember what they are supposed to do! A big context window defo could bring some benefits! I try and avoid using GPT3.5 or 4 for coding preferring for this reason. Id then like to use Claude 3 Opus with his 200k context window and extra capability for the heavy liftin and oversight!
I get a error, then I noticed that we need search api, I added that but still get the error pydantic_core._pydantic_core.ValidationError: 1 validation error for Task expected_output Field required [type=missing, input_value={'description': "Analyze ...e business landscapes.)}, input_type=dict]
I ran the code on Ollama and on Groq and I'm getting a loop "It seems we encountered an unexpected error while trying to use the tool. This was the error 'organic'" [Info]: Max RPM reached, waiting for next minute to start
Can someone help me, I got the following when I ran llama3 in Ollama. Created a chunk of size 1414, which is longer than the specified 1000 Created a chunk of size 1089, which is longer than the specified 1000 Created a chunk of size 1236, which is longer than the specified 1000
Please for the love of god somebody explain to me why we are using Ollama to download local models and then using Groq anyway to run the model in the cloud. Why can't we just skip the ollama part? I beg you i see all the videos using Ollama with Groq and i don't understand the aspect! thank you. Does ollama do something special to make it work better for crewai then a direct Groq connect?
Hi Brandon, the groq rate limit is a big issue for my use case, can i use this same method to use another similar hosted llama 3 70b with crewai like openrouter api or can any api be used instead of groq with your method?
Very good video demonstration. I noticed that you chose to use serper search in the video. I would like to know the difference between serper and duckduckgo search and how to choose between them. If you know, please introduce it to me. Thank you.
Hello Brandon thanks for the awesome video, I was trying to do the same setup with crew AI, but I am getting an error while running, I added the error message below Traceback (most recent call last): File "D:\crew_ai\crew.py", line 114, in result = crew.kickoff() ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 252, in kickoff result = self._run_sequential_process() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 293, in _run_sequential_process output = task.execute(context=task_output) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 173, in execute result = self._execute( ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 182, in _execute result = agent.execute_task( ^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\agent.py", line 207, in execute_task memory = contextual_memory.build_context_for_task(task, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 22, in build_context_for_task context.append(self._fetch_stm_context(query)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 31, in _fetch_stm_context stm_results = self.stm.search(query) ^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\short_term\short_term_memory.py", line 23, in search return self.storage.search(query=query, score_threshold=score_threshold) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\storage ag_storage.py", line 90, in search else self.app.search(query, limit) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\embedchain.py", line 635, in search return [{"context": c[0], "metadata": c[1]} for c in self.db.query(**params)] ^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\vectordb\chroma.py", line 220, in query result = self.collection.query( ^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 327, in query valid_query_embeddings = self._embed(input=valid_query_texts) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 633, in _embed return self._embedding_function(input=input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\types.py", line 193, in __call__ result = call(self, input) ^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\utils\embedding_functions.py", line 188, in __call__ embeddings = self._client.create( ^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai esources\embeddings.py", line 113, in create return self._post( ^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1232, in post return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 921, in request return self._request( ^^^^^^^^^^^^^^ File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1012, in _request raise self._make_status_error_from_response(err.response) from None openai.NotFoundError: 404 page not found
![How To Connect Local LLMs to CrewAI [Ollama, Llama2, Mistral]](http://i.ytimg.com/vi/0ai-L50VCYU/mqdefault.jpg)
![How To Connect Local LLMs to CrewAI [Ollama, Llama2, Mistral]](/img/tr.png)
This man knows what the people want , getting up and running with LLMs and Agents for the F R E E 😮💨 !
“Now this is pod racing !” 😂🙏🏻👍
Man, I do not know how to create and write code but you have made a video and I think I can do this! Bless you my friend!
Thank you! I'm confident you can do it! Let me know if you nee help with anything!
Friendly commment: You look better with glasses, more professional. Great content.
Hey D! Thanks!
I love wearing glasses and hate my contacts so I think I might need to go full glasses mode 🤓
Your channel is THE real gold mine. Thank you so much.
I totally agree!
Appreciate your support (with those contents), the only drawback, was the need to subscribe to get access to a project that isn't yours. 😞
Excellent video! Would be interesting to see these frameworks, but within LightningAI Studios. Also, I saw CrewAI will be having a more golden standard approach to their code structuring in the near future.
Thank you! And you're definitely right about CrewAI moving towards YAML. When CrewAI+ drops, I plan on making a lot more content around this new format for you guys!
And, I haven't tried out LightningAI Studio yet so I'll definitely have to try it out this weekend. Thanks for the suggestion!
@@bhancock_ai Great! I like the YAML approach. William Falcon that started LightningAI (PyTorch Lightning) likes my posts on LinkedIn as I'm a huge fan in developing with it when I mention them. Will be studying your approach with the latest updates and hopefully with their Studio. Thanks!!
Hey Brandon, welcome back after your vacay!
Feels good to be back! I'm recharged and ready to go!
I hope we can get access to your skool soon! Its been a few days. So I can learn from your group.
refereshed after short vacation.. hope to see you frequently
The quality is 10/10 thanks mate
Thank you Mario!
@@bhancock_ai /llama3-crewai is this automate-youtube-with-crewai? or crewai-updated-tutorial-hierarchical
or crew-ai-crash-course or nextjs-crewai-basic-tutorial
or crew-ai-local-llm or crewai-groq-tutorial git hub I can't work out what file relates to this video?
Good luck getting local, quantized models to reliably function call, or use any kind of 'tool'. They need so much more supervision, which is where frameworks like langgraph can help, rather than crew
thank you for your help 🤘🤘
thanks! Your crewai tutorial are top-of-the-shelf stuff. do you have any crewai proejct with streamlit connected to show output on the ui? thanks
With both the Groq 8b and 70b with crew max_rpm set at both 1 or 2 I do get it halting for a while with:
[INFO]: Max RPM reached, waiting for next minute to start.
The problem is that Groq is so fast that it ends up processing too many tokens so it ends up hitting a rate limit and failing.
To get around that, we have to slow down our crew by setting the max RPM.
Feel free to bump it up to get your crew to move faster!
Hey Brandon great video by the way. There seems to be an error as such.
It seems we encountered an unexpected error while trying to use the tool. This was the error: Invalid json output: Based on the provided text, a valid output schema for the tool is:
{
"tool_name": str,
"arguments": {
"query": str
}
}
This schema defines two keys: `tool_name` which should be a string, and `arguments` which should be a dictionary containing one key-value pair. The key in this case is `query`, with the value being another string.
'str' object has no attribute 'tool_name'
That is using a very old version of CrewAI - if you run it with the current version of CrewAI it fails because of lack of expected_output parameter in the Tasks
Thanks! Is Claude 3 opus still not supported though? How come?
Brandon excellent tutorial 👍
Thanks man! I really appreciate it!
Great stuff. Maybe a silly question, but when it was fetching to read data from remote website (the analysis part), does it read it remotely OR does it capture screenshots & download text to feed into its prompt and then clear this cached data or such local cached data needs to be cleaned eventually? Hope it simply reads remotely without too much data saved locally as I plan to use this approach to analyze many websites without flooding my local storage.
Great content as always! Do you know if it's sustainable to use a single groqcloud API key to host LLM access for a multi-user app? Or would a service like AWS Sagemaker be better for simultaneous users?
Cheers!
This is awesome! 😎
Thanks! 😁
cant get the tools.search_tools module to run says not foun, ive trued pip install command or just install command with tools name, no luck any ideas please
Thank you :)
Thank you for this and for the code.. How does Llama 3 compare to Dolphin-Mistral 2.8 running locally as the more junior agents do you know? Dolphin-Mistral with its extra conversatuon/coding training and bigger 32k context window appeals! Ive had agents go round in circles creating nonsense with other frameworks as they dont remember what they are supposed to do! A big context window defo could bring some benefits! I try and avoid using GPT3.5 or 4 for coding preferring for this reason. Id then like to use Claude 3 Opus with his 200k context window and extra capability for the heavy liftin and oversight!
What would you say are the minimum hardware requirements to make it run smoothly?
I get a error, then I noticed that we need search api, I added that but still get the error
pydantic_core._pydantic_core.ValidationError: 1 validation error for Task
expected_output
Field required [type=missing, input_value={'description': "Analyze ...e business landscapes.)}, input_type=dict]
You get this resolved?
This is resolved by adding expected_output in your task
Can I use this and deploy it live so that other people can use it? Does it run for free only locally or its free when its deployed also?
might want to revoke those keys you revealed if you haven't
your awesome, thank you!
Thanks Mike! You're awesome too!
Hello sir
Nice content
Thanks Pratyush! I really appreciate it!
@@bhancock_ai how can I connect with you
I ran the code on Ollama and on Groq and I'm getting a loop "It seems we encountered an unexpected error while trying to use the tool. This was the error 'organic'" [Info]: Max RPM reached, waiting for next minute to start
Can someone help me, I got the following when I ran llama3 in Ollama.
Created a chunk of size 1414, which is longer than the specified 1000
Created a chunk of size 1089, which is longer than the specified 1000
Created a chunk of size 1236, which is longer than the specified 1000
Just curious why VSCode doesn't display file types on Mac. I'm going bonkers trying to figure out what to save the Modelfile as.
Hey! There actually isn't a file type for that file. You can just leave it how it is. Hope that helps!
@@bhancock_ai Thanks for the quick response. I figured out that ".txt" works for those of us in other operating systems.
Please for the love of god somebody explain to me why we are using Ollama to download local models and then using Groq anyway to run the model in the cloud. Why can't we just skip the ollama part? I beg you i see all the videos using Ollama with Groq and i don't understand the aspect! thank you. Does ollama do something special to make it work better for crewai then a direct Groq connect?
Is there an easy way to build No Code AI Assistants or Agents with Groq? I know zero code
Hi Brandon, the groq rate limit is a big issue for my use case, can i use this same method to use another similar hosted llama 3 70b with crewai like openrouter api or can any api be used instead of groq with your method?
Oh i see it has to be an api alreadyy supported by langchain correct or it wont work?
Max RPM reached, waiting for next minute to start. How to solve it...
What graphic card do you use on your computer when running local with Ollama?
The key is VRAM. I can run smoothly llama 3 70b on RTX3090 hiting about 16GB of VRam (if remember correctly)
Nice! 😃
gyus do u know any way to run crewai and\or llama on gpu? only CPU is soooooooooooooooooooooooo sloooooooooooooow
llama_cpp (what ollama is based on) or vllm
Very good video demonstration. I noticed that you chose to use serper search in the video. I would like to know the difference between serper and duckduckgo search and how to choose between them. If you know, please introduce it to me. Thank you.
Hello Brandon thanks for the awesome video, I was trying to do the same setup with crew AI, but I am getting an error while running, I added the error message below
Traceback (most recent call last):
File "D:\crew_ai\crew.py", line 114, in
result = crew.kickoff()
^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 252, in kickoff
result = self._run_sequential_process()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\crew.py", line 293, in _run_sequential_process
output = task.execute(context=task_output)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 173, in execute
result = self._execute(
^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\task.py", line 182, in _execute
result = agent.execute_task(
^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\agent.py", line 207, in execute_task
memory = contextual_memory.build_context_for_task(task, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 22, in build_context_for_task
context.append(self._fetch_stm_context(query))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\contextual\contextual_memory.py", line 31, in _fetch_stm_context
stm_results = self.stm.search(query)
^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\short_term\short_term_memory.py", line 23, in search
return self.storage.search(query=query, score_threshold=score_threshold)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\crewai\memory\storage
ag_storage.py", line 90, in search
else self.app.search(query, limit)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\embedchain.py", line 635, in search
return [{"context": c[0], "metadata": c[1]} for c in self.db.query(**params)]
^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\embedchain\vectordb\chroma.py", line 220, in query
result = self.collection.query(
^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 327, in query
valid_query_embeddings = self._embed(input=valid_query_texts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\models\Collection.py", line 633, in _embed
return self._embedding_function(input=input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\api\types.py", line 193, in __call__
result = call(self, input)
^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\chromadb\utils\embedding_functions.py", line 188, in __call__
embeddings = self._client.create(
^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai
esources\embeddings.py", line 113, in create
return self._post(
^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1232, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 921, in request
return self._request(
^^^^^^^^^^^^^^
File "D:\crew_ai\.my_crew_env\Lib\site-packages\openai\_base_client.py", line 1012, in _request
raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: 404 page not found
Have you tried use ChatGPT to solve you erro? I used and I had a good result.