Convert Any Text into a Knowledge Graph
Вставка
- Опубліковано 30 вер 2024
- Medium Post: towardsdatasci...
Code: github.com/rah...
Schema method: • Graph ML: Build Knowl...
#datascience #machinelearning #deeplearning #datanalytics #predictiveanalytics #artificialintelligence #generativeai #largelanguagemodels #computervision #naturallanguageprocessing #agents #transformers #embedding #graphml #graphdatascience #datavisualization #businessintelligence #optimization #montecarlosimulation #simulation #LLMs #python #aws #azure #gcp
If you found this content useful, pleases consider sharing it with others who might benefit. Your support is greatly appreciated :)
Thanks for the video! Can you share the exact source code you have used in the video, I'm not able to find the exact source code in the github link you have provided.
Check this - github.com/rahulnyk/knowledge_graph/blob/main/extract_graph.ipynb
Great Video on Text to Knowledge Graph. Can you please tell me how to add context extracted from Knowledge Graph to Retrieval Augmented Generation to get a better response fromLLM
Pls look at GraphRAG or combining Graphs with RAG. I've a number of video on this topic.
If you found this content helpful, please consider sharing it with others who might benefit. Your support is greatly appreciated :)
sir, i tried running the code in the way shown by you, but in google colab it's saying ollama.client is not being able to be imported error, i tried to use chatgpt but it is unable to rectify it. how can i modify this code using my own dataset and is it going to work in google colab
whats the error? Can you pls share your colab notebook?
@@SridharKumarKannam sure sir, could you send your gmail, ill forward it to you
@@SridharKumarKannam sure sir, can you please send your email, ill forward it to you
hello sir, is this completely free to use or is there any API key or anything else that I need to purchase? thanks for the video!
you need a language model (LLM) for this. You can get free API from OpenAI, Groq, HuggingFace etc or run the LLM locally using Ollama, vLLM, etc.
sir, when running the "res = extractConcepts(prompt = pages[0].page_content)",
an error occurred that "Max retries exceeded with url: /api/generate".
Can you help me pls
Pls check/increase your openAI rate limits or use open source llms using ollama.
Hey! What's the python package to use helpers library.
Those helper functions are in the same codebase - github.com/rahulnyk/knowledge_graph/tree/main/helpers
If you found this content helpful, please consider sharing it with others who might benefit. Your support is greatly appreciated :)
Great video! Thank you for patiently explaining this so deeply
thank you :)
Hi, thanks so much for putting this video. It is great. I see you three videos for same thing - text to Graph.. one is this, second one using langchain and third one using libraries similar to this. what are high level differences between the three videos and which approach would you recommend if the scale is more?
the main difference is control over what entities and relationships you want in your model. You can start with open ended or unbounded schema, but eventually want to refine it to have only the desirable entities and relationships
Great video mate. You save me a lot of time......🚀🚀🚀🚀🚀🚀🚀
If you found this content helpful, please consider sharing it with others who might benefit. Your support is greatly appreciated :)
most common format for text: pdf. txt and md are already “easy” to build a graph…
Thanks for your support :)
first, thank you for the video, second, i tried to use the code but it seems i have problem with ollama, i installed it but i get no model named ollama.client, is there any way to fix it?
Are you able to fix the issue, otherwise, pls let me know, I'll help...
@@SridharKumarKannam I have the same problem. Is there a way to fix it? Im working on a Mac. Btw, super useful video!:D
@@SridharKumarKannam Nvm, Im stupid! Have a nice day:D
Can we also create Knowledge graph for Time series data?
Thanks for the suggestion, using KGs we represent entities and the relationships between them. For a time series data we don't have such a thing. Pls see - stackoverflow.com/questions/71780910/knowledge-graph-for-time-series-data
Hi there! I'm having some troubles at the part before calculating contextual proximity:
"ConnectError: [WinError 10061] No connection could be made because the target machine actively refused it"
Do I have to open ollama client (on my local machine) before running the code?
Also there were quite a number of tweaks I had to make to the code before it could run, is that typical from your experiences?
Some examples of these changes are:
helpers.df_helpers -> df_helpers
.prompts -> prompts
model_name -> model (mistral-openocra)
etc.
I'd imagine this might be due to my file management within Jupyter itself? But when I create a folder within the folder, it seems to not be able to find or extract the file. So I've positioned it into the same main folder.
Looking forward to hearing from you! Thank you so much in advance for your time and informative video.
>> Do I have to open ollama client (on my local machine) before running the code?
Yes
>> Also there were quite a number of tweaks I had to make to the code before it could run, is that typical from your experiences?
Yes, thats my experience too, these libraries are new and being developed actively by many. The APIs have been keep changing.
It hard to say whats going on without running the code myself. I'm sure you will be able to fix all the issues one by one, it takes some time, but same with me too. All the best
Nice sir, ❤
Thank you.
If you found this content helpful, please consider sharing it with others who might benefit. Your support is greatly appreciated :)
4 points: ❗❗❗❗
1 = The wording of the edge descriptions seems to be different than the short edge descriptions I've seen in the past. 👎
2 = The edge descriptions also seems to repeat the words used in node1 or node2. 👎
3 = I wish the examples used some text that could be broken down in to factual statements and relations like a description of a Github project, or Documentation, or textbook style text. 👎
4 = I hate sports and the example edge descriptions just seem stupid and useless. 👎
these are meant for quickly getting started and learning. Thanks for your support
Are you using neo4j for the graph? If so how can i access it from neo4j Database
In this video, I've used networkx not neo4j. Once you have the nodes and relationships exacted, you can load them any graph DB of your choice...
can u share here the previous video link where we have explicit relation and built a knowledge graph
ua-cam.com/video/sK61Y22wvr8/v-deo.html
I would like to build a taxonomy of skills and an ontology of skills, can I use the same process you used ?
Pls check this video - ua-cam.com/video/sK61Y22wvr8/v-deo.html I've a few other videos on defining the entities and relationships, taxonomy, pls check...
Thank you so much for this ❤
Thanks for your support. Pls share with your friends :)
How can we do this for Wikipedia content. Where we need each every possible mode and relationship type between nodes.
As I mentioned, broadly we have 2 methods, one is open ended where by default LLM extract all nodes and relationship, if it misses any, you can guide the LLM using the prompt. In the 2nd method, you can explicitly define all types of nodes and relationships you want to extract.
@@SridharKumarKannam when can we have next part of the vedio. Actually we are trying to identify nodes relationships and then ontology or knowledge graph based on that. What approach can help us better here. Is there any other generative models which can identify entities and relationships and their properties
awesome video
thank you :)
Thanks for putting this together. 👍🏾
thank you :)
@@SridharKumarKannam Certainly!
Thank you🙏
Thanks for your support :)
Can you also cover how we can add this to neo4j and run query on that data.
I've a couple of videos on adding such nodes/relationships to neo4j. Pls check - ua-cam.com/video/sK61Y22wvr8/v-deo.html
How can I use this .html as LLamaIndex KnowledgeGraphIndex?
Simple Directory Reader should read all docs including html in the folder - docs.llamaindex.ai/en/stable/examples/data_connectors/simple_directory_reader.html