GraphRAG - Will the Dust Settle?

[@Othit]

Fantastic, I watched the whole thing. I like the "teacher -student" presentation style, even though you are both experts :-)

[@awakenwithoutcoffee]

appreciated as always. While I am excited about graphRAG it does raise some important questions:
- how well can this integrate with LangChain/LangGraph Agentic frameworks?
- the last time we checked the amount of tokens required was almost 10x than that of RAG.
Any techniques on the horizon for limiting the cost ? If not , how can we justify those costs ?
- are there automatic graph creation methods or are quality retrievals still dependent on manual construction of the Graphs ? (couldn't we use an LLM chain to automatically do this for us ?)
🙏💻🍀
ps. Are you planning to update the RAG System workbook ?

[@MehdiAllahyari]

Thank you! To answer your questions;
- There is no direct integration with langchain, but I think either langchain or microsoft will develop that part. That said, I think it's possible to use it within langchain since it's a separate library.
- Yes indeed. The amount of tokens consumed by this approach is ridiculously high and that's one of the main reasons that I think make it not practical to use in production.
- Regarding cost, what I would do is to avoid using LLM as much as possible for KG construction phase. There are many other BERT models you can use for entity/relation extraction. Which make is very cheap to build the KG.
- Building KG is a semi-automatic process. If you want to make it completely automatic, you either have to use LLM which makes it costly or end up with a graph that is not completely reliable. I mean, there could be noise in there. The best way is to define a schema manually for your KG and then populate it with info with/without LLM.
- Which RAG system workbook? Do you mean the book that I authored a few months ago?

[@awakenwithoutcoffee]

@@MehdiAllahyari Appreciate the response. I'm indeed referring to the "A practical approach to RAG" pdf (great read btw! but It could be nice to have continuous updates seeing how fast this industry moves ).
Some more thoughts/Questions :
- if we consider graphRAG not (yet) production ready do we consider Hippo-RAG under the same umbrella ?
- for technical documentation QA (pdf) is there a specific RAG set-up you would recommend learning more about ?
Cheers !

[@MehdiAllahyari]

@@awakenwithoutcoffee I'd love to update the book as there are so many new techniques and methods have been proposed. I need to find the time. :)
- Yes graphRAG and hippoRAG both use KGs as the main source of knowledge. But the way they represent and query KG is different. I'd personally like hippoRag better as it's simple to implement and maintain.
- For QA over technical documentations, there is no universal solution. I would go with a very custom one depending on the nature of the documentation and domain. The general RAG process is the same however, the implementation details will be totally different. You need to do several experimentations to figure the right end-to-end solution.

[@kenchang3456]

Thank you for your explanation and perspective.

[@aGianOstaLgia]

Thanks!!

[@fozantalat4509]

Thanks and I really like your take on whether its practical in a production scenario, keep making more videos like this.

[@simaim8447]

Thank you! This is a great video with a clear explanation.

[@themax2go]

running triplex local (ollama) = almost instant return @ $0 (well maybe 1c for power usage... or non if using solar)

[@dailywisdomquotes518]

can this be applicable for ticketing (customer support)

[@TwoSetAI]

Potentially, you can prototype with this library. We also shared pros and cons in this video about this approach. Hope it helps!

[@alexdevoid4651]

What do they call the parameter you have to enable for entity resolution?

[@JumpDiffusion]

Could you explain where/how embeddings are used here? It is not mentioned in the paper (From Local to Global..), where it seems like all the steps are done by using/calling LLM.
Also, how do they decide if a query is global or local? A kind of LLM-based router that decides a query type?

[@MehdiAllahyari]

It's not mentioned in the paper, nevertheless they mention it on their website. They are using openAI embedding. but you can change it to other embedding models via configuration. Here's more details:
microsoft.github.io/graphrag/posts/config/env_vars/

[@ZIaIqbal]

I heard that there is a code chunk to use it with llama3, have you tried that? That should get rid of the token cost.

[@MehdiAllahyari]

You can replace openAI models with local ones. But usually the quality is lower and latency is a lot compare to more classical methods.

[@themax2go]

you might want to experiment w/ sciphi/triplex

[@user-vu4or4ih8p]

Thanks!