Відео

00:05 Meta prompting involves using language models for prompt engineering. 01:15 Meta Prompting method uses a Conductor LLM to coordinate experts in an iterative process. 02:45 Generating prompts and evaluating outputs for improvement 04:02 Various methods for prompt generation and evaluation 05:27 User and model collaboration in prompt generation 06:40 DeepPrompY is an adaptive open-source tool for refining prompts using scoring mechanisms 08:05 Textual gradient feedback in The Prompt system 09:29 Different meta prompting methods and free tools

Interesting

thanks! some very helpful nuggets throughout. and I'm happy not to have read and mentally translated those papers myself.

Good one

Hey Dan, great video! CoT is definitely the most effective technique. But it‘s a lot of work to come up with the reasoning steps. Is there a good way to get the model to reverse engineer reasoning steps from given input-output pairs? Or just output ideally? This could be a good topic for a new article or video.

Thanks for sharing! Your blog and videos are my #1 resource on the web for learning how to improve my prompts. I love that your content is always based on scientific studies.

Nice Daniel Kahneman click bait. 😉

Nice clear explanation.

personally i changed the usage on my model to use thought ! < it does not make responses only tool calls : so the final answer is the response tool: if it need to think it can query itself for knowledge first : then use the returned context to enrich the response : so it has its own self query tool : so when you ask a question it will execute a tool , or query itself , or create a tool ? after it has done its internal thinking or fuctin calls etc it will use the final answer tool ! Then i also use the thought / action / pause for result / Obeservation > Final answer ! << this is probably the future way ! i do not use rag , but if the model requires to search for knowledge or stroe the conversation or given information it will be added to the rag with the search tool ! < it can query its shortterm emeory ( the rag to also retrive context to enrich the result) It was very interesting of hugging face to release a agents library as it also used Models as Tools!: this also drastically changed the way the model is being used now ! < so given any input can bedirected to the corect tool to return its caption or what ever then if generation is required it can go to its imgae gen or audio gen ! <<<! the thought Action loop ! << this actually changed the outpput of the model drastically : so i trained the model as a planner and also as a tool user ! Also to output json etc : So these Chain of thought are still in use my friend , they are within the external wrapper paradigm of tool use and GameLoop! <<< So you can give the model tools or not ! or you can drop them as single funcitons in a folder and it will load them as required ! >> Brother i dont know what these university STUDENTS think we are doing out here !!! << We are actually in front of you guys ! Dragging you into the future ! hopefully without you guys writing some fake ethical code against us ! ( we hate to share with you guys ! ) examine this chat you just gave it was pure rhetoric ! i bet you fancy a cuppa tea mate !

Great Video! You publish the best content on Prompt engineering in the whole AI space! (Maybe next to Moritz Kremb - the Prompt Warrior).

Hey Dan, any chance I could get access to the code to test it out my self?

Umm I think you are misunderstanding quite a few points. The paper states directly that they view Perplexity with respect to entropy, i.e. tokens with a higher perplexity have more information to yield to the model at inference. Because of this, the authors tell us that they create a subset of the demonstrations by first ordering the demonstrations from highest PPL -> lowest PPL, and then continuously adding to the subset until they reach the token budget for the demonstrations. Saying that "It keeps the low perplexity ones, gets rid of the high ones" is just flat out incorrect. Furthermore, "compresses the low ones, and does that for all the demonstrations" is a bit misleading, as the coarse level compression is essentially just picking and choosing which demonstrations get added to the subset to use for iterative level compression.

I am usually polite anyway. (Never know when these things will turn sentient!)

Dan what data format (for your candidate dataset--say possible articles) are you using? .csv passed to the LLM?

Thanks for this Dan

Excellent work, Dan.