Hey everyone! I received some great notes from Thomas Ahle, the creator of DSPy's TypedPredictors. These have been added to the video description, but going to leave them here as a pinned comment as well to make them easier to find! **ERRATA** 1. `dspy.TypedPredictor` can be used directly instead of `dspy.functional.TypedPredictor` 2. When creating a pydantic type, `list[Topic]` can be used directly in the Signature without needing the `Topics` wrapper. 3. The default `max_retry` for TypedPredictor is 3, and can be set when creating the TypedPredictor. 4. Setting `TypedPredictor(explain_errors=True)` can help with retry errors by providing clearer descriptions of what needs to change.
Just FYI people, about models for function calling and returning JSON, Hermes Pro 2 7B from NousResearch is also working extremely well, and was trained with those use cases in mind.
Hi Connor, thanks the awesome video! This serves as a great intoduction to using DSPy for Constrained Output Generation Tasks! I was wondering if there is any way instead of using the default JSON functionality of TypedPredictor as a way to tell the model the correct format to also use the Tool/Function call functionality that many LLM APIs provide -> so in a sense similar to what the instructor package allows. If there is not yet a native way doing this in DSPy, is there another possibility to use DSPy togeter with the instructor library? What I would love to do is to use DSPy on top of instructor to optimize over the instruction part of the prompt (if we imagine prompt = constant instruction + changing input) that is sent to the LLM via instructor. The current only workaround I found is to use the DSPy integration of Langchain as Langchain also offers constrained output generation via Tool/Function call functionality. But maybe there is a better alternative that would allow me not to use Langchain (as very much bloated up). Any hints or directions would be very much appreciated! Thanks a lot for your many DSPy tutorials, this is simply awesome!
Thanks for the great content. How will you combine instructor with DSPY ? Instructor seems to have resolve structured output in a more elegant way. I like fully embracing framework but Instructor have an almost invisible abstractions for addressing structured outpout
Great content, well presented! Thanks for the whole series. I have two questions: What are the fundamental differences between a retrieval model and a "main" LLM? I know the idea of DSPY is to hide the LLM from coding choices. In the context of LLMs, what are the teacher/student scenarios used for? What actually happens when one compiles a DSPY component? I'd appreciate a deep dive into this topic.
Love this type of thinking!! I have a project on github -> weaviate-tutorials -> Hurricane, definitely more to come! Arize Phoenix gives you a nice UI for program traces if that is what you mean with this question instead.
Hey everyone! I received some great notes from Thomas Ahle, the creator of DSPy's TypedPredictors. These have been added to the video description, but going to leave them here as a pinned comment as well to make them easier to find!
**ERRATA**
1. `dspy.TypedPredictor` can be used directly instead of `dspy.functional.TypedPredictor`
2. When creating a pydantic type, `list[Topic]` can be used directly in the Signature without needing the `Topics` wrapper.
3. The default `max_retry` for TypedPredictor is 3, and can be set when creating the TypedPredictor.
4. Setting `TypedPredictor(explain_errors=True)` can help with retry errors by providing clearer descriptions of what needs to change.
Awesome video, Connor! Excited to try out the TypedPredictors with assertions
Thanks so much Erika! Yes! TypedPredictors and Assertions are the Peanut Butter and Jelly of LLM systems!
Amazing guide! Was trying to format llama-3-8B for a while and this provides so much depth to help me understand how to go about it.
Thanks for sharing this! I'd love to see you doing something with the ReAct module, especially optimizing it using DSPy.
Awesome idea! Definitely looking forward to learning more about function calling with DSPy and the ReAct framework for doing so!
Thanks for the video, just starting to wrap my head around these techniques and your (and your colleague's) videos have been helpful. Cheers.
Thank you so much! Means a lot to hear that! Cheers!
I'm very grateful for this videos series.
I'm learning a lot.
Thank you so much! So happy to hear it!!
Wow this video blew my mind!! I didn’t know any of this before, this was super helpful Dr. Connor!
Fantastic! Thank you so much!
Just FYI people, about models for function calling and returning JSON, Hermes Pro 2 7B from NousResearch is also working extremely well, and was trained with those use cases in mind.
Epic! Going to check it out, thanks so much for the tip!
Hi Connor, thanks the awesome video! This serves as a great intoduction to using DSPy for Constrained Output Generation Tasks!
I was wondering if there is any way instead of using the default JSON functionality of TypedPredictor as a way to tell the model the correct format to also use the Tool/Function call functionality that many LLM APIs provide -> so in a sense similar to what the instructor package allows.
If there is not yet a native way doing this in DSPy, is there another possibility to use DSPy togeter with the instructor library? What I would love to do is to use DSPy on top of instructor to optimize over the instruction part of the prompt (if we imagine prompt = constant instruction + changing input) that is sent to the LLM via instructor.
The current only workaround I found is to use the DSPy integration of Langchain as Langchain also offers constrained output generation via Tool/Function call functionality.
But maybe there is a better alternative that would allow me not to use Langchain (as very much bloated up).
Any hints or directions would be very much appreciated! Thanks a lot for your many DSPy tutorials, this is simply awesome!
Thanks for the great content. How will you combine instructor with DSPY ? Instructor seems to have resolve structured output in a more elegant way. I like fully embracing framework but Instructor have an almost invisible abstractions for addressing structured outpout
Great content, well presented! Thanks for the whole series. I have two questions:
What are the fundamental differences between a retrieval model and a "main" LLM? I know the idea of DSPY is to hide the LLM from coding choices. In the context of LLMs, what are the teacher/student scenarios used for?
What actually happens when one compiles a DSPY component? I'd appreciate a deep dive into this topic.
Hey Connor, thanks so much for your work
is there going to be any GUI implemented on top of that?
Love this type of thinking!! I have a project on github -> weaviate-tutorials -> Hurricane, definitely more to come! Arize Phoenix gives you a nice UI for program traces if that is what you mean with this question instead.