This is a concept I'd been considering myself, but I never thought of it as autodifferentiated text. Fantastic that research is being done in this direction. I knew it'd be a good idea.
I criticized this has to be done manually, but never thought of chaining 2 LLMs to achieve it. Though, it does make getting slightly better answers 3x more expensive. I guess it's useful for unsupervised learning though.
Thanks for the video, it’s very insightful! I have 1 thought: 1. Textgrad and DSPy can be combined. As DSPy is mostly based on ICL and this framework focuses more on signature optimization. Additionally, the researchers in Stanford mentioned that the combined prompt on one occasion improved the prompt by 1% and it should be further studied.
Here's an unpopular opinion: could this be considered a misuse of the notation for auto-differentiation and backpropagation? For any graph to be differentiable, it must be acyclic-like a Directed Acyclic Graph (DAG), which is typical for neural networks. However, in the LLM sphere, we see pipelines incorporating cycles, such as the RAG where blocks are repeatedly cycled through, forming what might be described as Directed Cyclic Graphs (DCGs). While using PyTorch's clean and modular syntax is appealing, applying auto-differentiation in this context could be seen as a stretch (personal opinion).
OK, I'm a n00b. But why should I use two models when the smarter one can give me the optimal answer straight away? In which scenarios do I need all these expensive iterations? Will I then have predefined prompts for recurring queries of the same type that can be answered directly on my smartphone by a small model?
Great video, very informative. Textual Gradient is such a pretentious concept for me, but I do look forward to try TextGrad out. At least it is a systematic method to perform prompt optimization.....
You said that you used in on your tasks. Can you release part of that code in the wild? It would be really great to see a live example. That was the thing I found very challenging with DSPy. Only with the storm project I started understanding how it should work ;-)
Start with the four Jupyter Notebooks that I provided and you will see that you have immediately multiple new ideas for your specific tasks. I plan a new video on my insights, given my testing and maybe I have an idea how to optimize the TextGrad method further ....
Amazing video! But pseudo as in pseudo-code is pronounced like sudo (syuudo) Not smart enough to correct anything else in this video lmao, keep up the good work! Love the channel
This is a concept I'd been considering myself, but I never thought of it as autodifferentiated text. Fantastic that research is being done in this direction. I knew it'd be a good idea.
I criticized this has to be done manually, but never thought of chaining 2 LLMs to achieve it. Though, it does make getting slightly better answers 3x more expensive.
I guess it's useful for unsupervised learning though.
Thanks for the video. I missed the boat with DSPy but it's good to know you can just go ahead with TextGrad.
Thanks for the video, it’s very insightful!
I have 1 thought:
1. Textgrad and DSPy can be combined. As DSPy is mostly based on ICL and this framework focuses more on signature optimization. Additionally, the researchers in Stanford mentioned that the combined prompt on one occasion improved the prompt by 1% and it should be further studied.
DSPy is ICL and prompt optimisation combined. I hope they add text grad in somehow though
@@matty-oz6yd yea, good correction. I hope they add Textgrad in as an optimizer.
Their mipro v2 optmizes literally doing the same
sounds like we need a middleware complexity assesor that can sit in the middle and auto reject if it doesnt meet that balance
Here's an unpopular opinion: could this be considered a misuse of the notation for auto-differentiation and backpropagation? For any graph to be differentiable, it must be acyclic-like a Directed Acyclic Graph (DAG), which is typical for neural networks. However, in the LLM sphere, we see pipelines incorporating cycles, such as the RAG where blocks are repeatedly cycled through, forming what might be described as Directed Cyclic Graphs (DCGs). While using PyTorch's clean and modular syntax is appealing, applying auto-differentiation in this context could be seen as a stretch (personal opinion).
OK, I'm a n00b. But why should I use two models when the smarter one can give me the optimal answer straight away? In which scenarios do I need all these expensive iterations? Will I then have predefined prompts for recurring queries of the same type that can be answered directly on my smartphone by a small model?
great video, hope for you more advanced explain and experience on TextGrad!
Thanks stanford, though I would have called it backpromptigation. ;)
Great video, very informative. Textual Gradient is such a pretentious concept for me, but I do look forward to try TextGrad out. At least it is a systematic method to perform prompt optimization.....
Solid. I knew if the guy behind DSPy could build that, there was a better version imminent
Thanks for another great video! I like your presentation style. What kind of software do you use for your slides?
It is 3 months later, has either of the two approaches proven to be practically useful and is being used today?
You said that you used in on your tasks. Can you release part of that code in the wild? It would be really great to see a live example. That was the thing I found very challenging with DSPy. Only with the storm project I started understanding how it should work ;-)
Start with the four Jupyter Notebooks that I provided and you will see that you have immediately multiple new ideas for your specific tasks. I plan a new video on my insights, given my testing and maybe I have an idea how to optimize the TextGrad method further ....
26:51 what does 0 demonstrations mean? No examples of good output, only original prompt?
Answer from Copilot: Yes
Superb explanation! Thank you!
How is this different from prompt tuning (not engineering)?
Explained in the video.
Seems like one can prompt optimize for the same level system and never lack coherence.
Very informative.
Thanks
Thanks for the links to colabs…
Great! Thanks
Amazing video!
But pseudo as in pseudo-code is pronounced like sudo (syuudo)
Not smart enough to correct anything else in this video lmao, keep up the good work! Love the channel