Using AUTOGEN & GUIDANCE to code LLM Control Flow & JSON Agents (No Prompt Engineering)
Вставка
- Опубліковано 7 лип 2024
- These may be the two MOST IMPORTANT AI AGENT types. If you've ever created a prompt with something like "choose between only [x, y, z]..." and "please respond in this JSON format" and received inconsistent results, you've found the video you've been looking for. In this video we build on our multi-agent postgres data analytics tool and create two new mission critical agent types using AutoGen and Guidance.
By combining AutoGen and Guidance we're going to build the Decision Agent and the Structured JSON Agent. Big shout out to the code bros at Microsoft once again.
Thanks to our agent orchestrator, conversation types, multi-agent team capabilities, instruments and validators we easily build these new agents with minimal changes. To build agentic systems you need systems and agents that can make decisions and drive the flow of your application while providing useful, structured responses. That's the key issue we solve in this video.
The structured responses and useful syntax from guidance allows us to side step some prompt engineering hassles. By integrating a control flow agent, we prevent our downstream teams from burning cash by running OpenAIs GPT-4 prompts that SHOULD NOT RUN if the prompt is invalid. We then generate novel SQL insights based on the incoming prompts and return in a structured JSON format using a custom guidance agent.
Don't miss out - Like, Sub and Join the journey as we tap into the future of engineering, today.
👍 THE CODEBASE
github.com/disler/multi-agent...
🔥 TALK TO YOUR DATABASE
talktoyourdatabase.com
🤖💻 AI Engineering Resources
Microsoft's Autogen: microsoft.github.io/autogen/
Microsoft's Guidance: github.com/guidance-ai/guidance
(1) Watch Part One
• Prompt Engineering an ...
(2) Watch Part Two
• One Prompt is NOT enou...
(3) Watch Part Three
• Make AutoGen Consisten...
(4) Watch Part Four
• AutoGen Token Tactics:...
(5) Watch Part Five
• AutoGen SPYWARE: Codin...
📘 Chapters
00:00 PREVIEW
00:29 Two Of The Most Important AI Agents
01:48 AutoGen and GUIDANCE
02:30 Recap Our Multi-Agent Data Analytics Tool
05:14 Flow Control With Agents
07:40 SCRUM MASTER AGENT
11:40 The Gate Team Lowers GPT-4 Costs
12:44 Creating a new multi-agent team
14:25 Data Insights Team
15:43 Round Robin Conversation Type
17:40 Structured JSON Responses
20:20 Novel SQL Insights in JSON
22:00 Our Agentic Building Blocks are stacking up
22:22 Let's Light Some Money On Fire
24:29 THE CODEBASE - couple words on the code
25:26 What is this series leading up to?
🐛 tags
#dataanalytics #agentic #promptengineering - Наука та технологія
To everyone tuned into this video, let's take a moment to appreciate what Dan brings to the table. Unlike the sea of UA-camrs out there, Dan is all about sharing incredible code. He's not chasing the spotlight; he's here to genuinely contribute to our knowledge. It's rare to find someone who teaches with such passion and clarity-Dan reminds us of those remarkable teachers we've all had, the ones who made a lasting impact.
The thing about Dan is not only is he so eloquent with his programming, but you can tell he 100% fully understands how this works. A lot of people that I follow just show the homepage of whatever and say the same things that everyone else does.
💯 agreed. Nothing but gems shared here.
This series is exactly what I need right now. Thank you.
Dan knows that reputation comes from serious work and not just self-promotion and selling miracles, as many other people do on the Internet. 👏
I'm now addicted to this unique contents and unique style in coding and talking
Thank you Dan !
Dan - thank you for the series.
This is really the kind of infrastructure code I am not able to produce myself since I am not a software engineer.
I really have some application ideas however - this is great foundational groundwork to use a structured input:output schema for whatever you might want to build yourself. Thanks for sharing!
Dude, you are the man, this is exactly what I've been struggling with!
Dan, you’re a legend. Thanks for all that great content!
thank you for providing the Codebase.. now i have to watch your vids again ;-)
you should be paid for this series.... Great job, thank you! ;)
This is brilliant!
excited to be this early! haven't even watched the video yet, but I know it'll be a great one!
FIRST!
Nice
came back to say that I finished the video; tis great. it'll be cool to see this with a front end; there are endless possibilities.
Incredible work. Please more utilization of specialized models for specialized jobs in the team.
Awesome video series. I’m trying to get my students in a MS in business analytics program build something like this as a new way to work with sql and also learn llm development
Decision Agent and Structured JSON Agent joining the chat... Prompt engineering nightmares, begone! 🧙♂💻 IndyDevDan's coding dojo just keeps on giving - it's like having a GPS for JSON in the wild AI landscape. Big up to the 'code bros' for the guidance! Subscribed & ready for the next episode of 'Avoiding Token Wasteland'. 🚀🤖 #NoMorePromptPanic
BRAVO!
Phenomenal
Same video but for mistral on vllm api , with mem gpt will be the perfection :)
Working on this with OS LLM on run pod. What memgpt features are you thinking of?
Dolphin is looking promising.
Really good
Hey Dan, loving the series - huge thanks! Curious about your thoughts on the recent OpenAI Dev Day announcements. Do you think any of the recent announcements might impact your plan for this series at all?
Coding will be an extinct art in 2 years.
For human
Great stuff Dan. You are doing some of the best work I've seen anywhere. Is context window optimization on the agenda? I can see how in this example it may not be super important but with others it might. Question: Do you have an export of the db?
Would love it if you add Chat2DB support, i mainly use redis rather than sql, and would love to see you add a generic Data Access Layer
This series is great, thank you for all the hardwork. Looking forward to generating my own agentic systems.
What is you're opinion on chatdev?, would it be better to generate a agentic system from scratch for code generation on a specific language, or use chatdev with gpt4 for this usecase?.
Thank you.
Amazing and super informative. Will you do a video using an open source model (since costs can be a concern using open ai API)? Such as WizardCoder?
What do you think abou memgpt or spr for enhancing the recolection of past interaction with the system?.
Thank you again for the awesome content 👏
But if executed at postgres_da_ai_agent folder: ImportError: cannot import name 'GenericAlias' from partially initialized module 'types' (most likely due to a circular import)
nice love it you have a download link for all files maybe? im gonna watch and like all your video's (A)
Lot of nuance to consider for the NLQ ranking logic. Is there an external evaluation process [even logs would help] to audit these rankings?
You're totally right. I currently use trial and error and suites of tests.
I'm actively searching for a concise and minimalistic LLM evaluation framework.
I'll share it on the channel when I find something worth my & your time.
Saw this test/eval approach; rather complex but thoughtful…
Would you consider making a video for people that are not developers but would love to be able to create and organise a multi-dimentional agent framework?
Is there a possibility for Guidance to hallucinate / provide an incorrect probability?
Hi Dan, your work is great. I've searched for a contact link for you but don't find one so I'm using this to reach out to see if we could discuss a paid consulting engagement with you. please reply to let me know. All the best and great work.
🎯 Key Takeaways for quick navigation:
02:28 🤖 *Two crucial agent types: The first enables multi-agent systems for autonomous decision-making, consistently providing output. The second generates structured responses for LLMs, eliminating the need for prompt engineering.*
05:42 🛡️ *Flow control with agents: Introducing a new "Scrum Master" agent to filter out invalid prompts, allowing only legitimate queries to proceed. This helps control the flow of the application.*
10:43 💼 *Custom agents for decision-making: Demonstrates creating a custom "Defensive Scrum Master" agent to make concrete decisions based on the confidence level of a prompt, showcasing the power of agent-driven flow control.*
13:55 🚀 *Enhancing Postgres tool with Data Insights team: Introduces a new "Data Insights" team using a round-robin conversation flow to generate novel insights from the database. Utilizes guidance and a custom agent for structured JSON output.*
17:26 📊 *Json-formatted insights: The Data Insights team successfully generates valuable insights in a JSON format, showcasing the potential for asynchronous functionality and parallel team execution in the future.*
18:37 🔄 *The round-robin conversation flow is a valuable tool, allowing circular interactions between agents. Adjusting the number of loops can cater to various use cases, offering flexibility in validating code or conducting research.*
19:19 🌐 *The Data Insights guidance agent leverages guidance's templating syntax to generate valuable SQL insights in a structured JSON format. This showcases the power of combining control flow agents and specific output generation.*
22:29 💰 *The multi-agent application, including gate teams, data engineering teams, and insights teams, is effectively demonstrating the ability to filter queries, run complex SQL, and generate novel insights in a cost-efficient manner.*
24:36 🧩 *The provided codebase is not a ready-to-use library but offers valuable patterns, ideas, and implementations for building agentic software. It emphasizes the evolving nature of coding practices and encourages developers to adapt and learn from patterns presented.*
26:01 🚀 *The upcoming series conclusion promises the release of an application named "Talk to Your Database," offering a user-friendly way to interact with a PostgreSQL database using natural language. The final videos will focus on deploying, building the frontend, and making the application battle-tested for real-world use.*
Made with HARPA AI
Great video/content, thank you for it! One thing though, could you please not include the pop-up words in the middle of the screen... they distract more than help. It's like consistently having pop-up ads while trying to watch a video. [ EDIT: I see you removed them after 2 mins or so... Got it, thanks :) . Much better without. ]
With JSON mode, you probably don't need guidance anymore!
off topic: Your signup doesn't support plus addressing.
Please add support for plus addressing.
(Not all people who use plus addressing are criminals.)
I have found in practice that plus addressing is extremely helpful.
But I still only limit signups to one root on my own auth systems (if there is a reason to block multiple accounts being managed by a single email)
And you can just throw unique error text back and really mess with the folks who being nefarious anyway (fun stuff)
Also random error codes is my fav with slightly altered text responses broken with some symbols. >;) (even more fun stuff)
Truly plus addressing is an email/password organizational godsend imo... give it a chance!
-Plus Addressing Advocate
Flailing hands are really annoying, they do not need to be present in the video