I am doing tech for 20 years You Sir are an excellent teacher. Pointing to the documentation and providing us pointers to nice tools to stitch it all together in comprehensive way.
Totally, if there's something I don't understand at that moment just rewinding it to couple of mins and then it makes more sense.. one of creators who isn't just walking through the code and saying what the command does in the notebooks Matt is really someone who knows how to empower the other person
Matt, your energy is so calm. I did fine tune with MLX, but I freaked myself out with all the steps and feel like it’s hard to do again. When you explain it so nicely, my fear goes away and I’m ready again. You’re spot on about the data prep is the “dark arts”. So true!!
Sorry if unrelated, am I the only one who thinks that Matt’s voice has that soothing-gentle-teacher like voice? Like I can hear him narrate for a natgeo documentary
I was totally agreed with you sir, u are the most easier way to let me learn about mlx in the past 2 month I’m keep finding UA-cam for all the information. Thank you so much for the video.
Thanks great breakdown of the process! A note about JSONL not being an array. It can be processed by old school unix tools like awk, grep, sed - and used in streaming data with unix pipes where lines are the delimiters. These tools don't do well with json array syntax on large datasets.
This is exactly correct. A JSON array, even if by convention has an object on each line, is not "valid" JSON unless the entire array, including the closing bracket, is pulled into memory. JSONL ensures that each line is its own mini valid JSON document.
I have heard many folks talk about doing a fine tuning to make a model work better with the aider tool. It seems like everyone is struggling to figure out what the dataset should be. It would be fantastic to get your thoughts on it! Thanks for all the great content!
Thanks for the great video! Could you use an llm to generate question answer pairs for the dataset out of basic text or documents? Would be interested in such a video!
super clear and helpfull instruction, thanks so much! I think the jsonl format has to do with training with super large datasets, making it possible to handle them row by row?
Nicely done! It's worth noting that what Matt demonstrated is "Fine-Tuning with LoRA" and not Actual Fine Tuning. Low rank adaptation (LoRA) makes customising a model more accessible than actual fine tuning of a model by "freezing" the original weights and training a small subset of parameters. Actual Fine-Tuning: Adjusts all parameters, requires significant resources, but yields high-quality results. Low rank adaptation (LoRA): Trains fewer parameters using low-rank matrices, reducing memory and compute needs while maintaining quality.
Matt, this is utterly awesome and I can't thank you enough. I'd seen the compute resources people were using and the code and gone "that's just too time and money intensive to investigate further". Now, I just need the script from terminator, a code interpreter and, oooh, 5 minutes? Don't worry, I'll keep control of it...
This is a great vid! Especially if you're at least a hobbyist. The best complete layperson onramp I've seen for fine-tuning? Is Cohere. And it is free. After that, dip into more of the dark arts. But to whet folks' whistle I usually point them there. It's their whole biz model. It quickly gets old, but you can whip up a trained, finetuned bot in a half day depending on dataset.
Hi Matt! Excellent series! Love your no-hype/nonsense approach to education! JFYI, in order to run mlx-lm against llama or mistral models on HF, you must first agree to the terms published on the HF in the repo of interest. Otherwise you mlx-lm.lora command will exit with `mlx_lm.utils.ModelNotFoundError: Model not found for path or HF repo: mistralai/Mistral-7B-Instruct-v0.3.` Cheers!
Thanks Matt, your explanations are effective and entertaining. If you could in a future video , would you dive into more detail about fine-tuning? E.g., why would you want to, how to choose your data, etc. Thank you!
I don't want to nessesarily train it on "how to respond" but I want to expand it's knowledge base of Python code so it knows "what" to respond with. For instance, I'd like it to have better knowledge of some of the more advanced concepts of Python. So do I fine tune to accomplish this or what is your recommendation. I'm just starting out learning and all I've been able to create is a mock jpeg file corruption recovery tool that doesn't actually work because jpeg is a lossy format to begin with. But it looks like it works. Lol
Thanks a lot for your fantastic videos. I'm actually using Unsloth to fine tune Llama3 for a text classification task. I'll be happy if you'll upload a video for such purposes
Great video, thanks! I only wonder what's the way to add new data to the model if finetuning is more about changing the format. My use case is like this: I use a framework which changes syntax with each new version I want the LLM to be updated with the newest documentation for the last version. How can I do that?
If you could make a guide of installing Unsloth on Windows and train Llama 3.1 on Ollama to use my tone of voice to reply to emails AND have knowledge to answer the common questions I get, that would be a godsent! Thank you for the great videos!
I am interested in the idea of fine tuning, and I am starting to regularly come to your videos for stuff now that Ollama is my primary tool I am using to connect to other front end for serving my models. But jeez, i feel like an idiot sometimes with some of this stuff because this still feels complex to me. "Step 1" of curating the data honestly feels like the easy part to me. I am curious about unsloth as its one I have looked at before but had decided to circle back to when I finally worked my way up to fine tuning. I am also curious about vision models, Llava or otherwise, I would be really curious to see how curating data for that would differ from an LLM. Also, would be nice in future videos related to this so see a before and after test. I know we can assumed what you are suggesting is making an effect, but it would still be nice to see the results in action :)
I like your presentation ...Regarding the fine tuning heard unsloth I have seen many of the AI experts uses it. What is your thought on the same lines?
Thanks, Matt-super spot-on video as usual. You raised a doubt in my mind: You mentioned that fine-tuning is not suitable for adding new information to the original LLM (perhaps I misunderstood). This leaves me a bit perplexed, and I know it’s a debated issue within the community. I agree with you that the best use of fine-tuning is to personalize the style and tone, rather than being used in the "traditional" way to train older (pre-GPT) models like BERT. However, many people argue that fine-tuning could be an alternative to RAG for injecting specific domain knowledge into the LLM. Personally, I’ve never tried fine-tuning a model due to the costs, especially with cloud-based LLMs. In any case, I think it would be valuable to explore these topics further. My hope is that fine-tuning could become a native feature in Ollama in the future. Lastly, it would have been useful to see the fine-tuning JSONL data (at least an example). I have my own answer to your question: why JSONL? It might be because of its line-by-line simplicity in Unix pipe scripting.
@@technovangelistBy "slower," do you mean that the fine-tuned model has increased latency during inference compared to the original model? That's interesting-I’ve never heard about that before.
@@solyarisoftware I think Matt has been finetuning an instruct model. Afaik instruct finetuning is usually done after training the model on the next word prediction task which is the step where the general knowledge is injected into the weights. The next step, the instruct finetuning is supposed to make the model better at following instructions and producing responses aligned with user intent.
Thank you Matt for again a great helpful video. It would be great to see samples of the test.jsonl, valid.jsonl and test.jsonl files. Or a template of them. Thanks
Very useful, however 1) is there anything we can do with 16 gig on m1? 2) can you maybe show how to do it? Step-by-step: it is a little hard to understand the MLX part thanks
Can a model be fine tuned to improve the way a model uses tools? Can fine tuning be used for chain of thought? One example would be to take a number of prompts, then a good chain of thought to solve that type of problem?
Could you do the same thing as fine tuning by creating a RAG database of examples, and just use the off-the-shelf LLMs? This might make your application LLM-agnostic and futureproof. Thoughts on pros/cons?
@@technovangelist Mmhh yes and no. Full-parameters finetuning is OK for adding new knowledge in a more systematic way (after all, it's just continued pretraining..).
so if finetuning is how you make the model respond in a better style, how do you teach it more stuff? which are the best ways to make the AI learn more aside from RAG
thank you Matt, you are amazing. As I know, mlx is for apple silicone, what about finetune on nvidia rtx card? which library should we use ? can you make a video for this please ?
Yup. I mentioned that I did this first for apple silicon. And I intend to do the same thing for unsling and maybe axolotl which are windows and Linux based
hi, why do i need a validation dataset and a test dataset? can i create them but let them empty? because i don’t understand anyway how validation and test would work for a llm
ok so i created the three sections, but now get a data formatting error heres a sample of some data of mine: {"prompt": "info.", "response": "info"} what should be different here?
jsonl is used to read line by line, it is easier for python, because it is reading line by line as far as i know. I.e. 1 iteration === 1 json from your dataset. Plus datasets are huge and reading whole json and parse it all in one go will take a decade and probably will crash your runtime.
@@technovangelist in fairness, it also allows you to do things without needing to parse the whole file - like cat something.jsonl | sort > sorted.jsonl or cat something.jsonl | head -n 10 > 10somethings.jsonl In this case, I imagine it’s convenient for shuffling the entries, but the main generic advantage over a big json list is that you don’t need to read the entire file to begin parsing it (which is a really nice language-agnostic property for files that might become extremely large) They’re definitely weird at first blush, but actually kinda clever and elegant for some use-cases IMO
I tried to fine tune llama3.1 on windows since I need nvidia gpu. Such a nightmare. I still didn’t figure out why my llm is not able to speak anymore, it just reply’s data that was in my dataset
that’s probably a topic that wouldn't happen here. Most of this channel focuses on local ai solutions due to the security and privacy risks of most online models.
Still not sure what the data file looks like for the framework? Is it a dataset? the below does not seem to work: "text": "This is the first piece of text."} {"text": "Here is another piece of text."} {"text": "More text data for fine-tuning."}
@@technovangelist Agreed. The thing i learned is that data is the directory where you put train.jsonl test.jsonl and valid.jsonl with the format you describe. Thank you!
sir i can't understand the way dataset jsonl file should be can u pls give me 1 block of the dataset jsonl file as example. i can't understand how to make 'em pls help me else someone out here pls help me too
you need to provide more info. Where did you get this? what version of ollama? what platform? how was it installed. Best to do all this on the discord.
If you only can change the style of the answer why bother with fine tuning, I don’t need the answers to be like a pirate , why would you need this for creating an enterprise level application? Is rag the way to go for this ?
Tweaking the style is a very important aspect for most enterprises. Some need it to respond as Sql every time or json or functions. Those don’t need new knowledge but rather told how the model should respond.
Matt, I had written down a long comment, but UA-cam deletes anything that has links to platform it doesn't like, probably. If you have time and will, please read my replies to your twitter thread related to this video, thanks.
I also hate jupyter notebooks... Agree that it is the worst for teaching... I always convert it to python file in the end and getting rid of all useless stuff...
hey, cool channel, but this is too complicated. i want a 1-click solution. i want to do 1 click, and the ai should exactly learn how my texting style is. i want to use it to prank whatsapp friends.
There are options for that. The one I have seen costs about 200 to 300 usd per fine tune run. Or you can spend 10 minutes doing it this was for free. Anyone can do this.
I am doing tech for 20 years You Sir are an excellent teacher. Pointing to the documentation and providing us pointers to nice tools to stitch it all together in comprehensive way.
I love the way Matt explains things in a way that is both detailed and yet really easy to understand. Thank you man.
Thanks so much
Totally, if there's something I don't understand at that moment just rewinding it to couple of mins and then it makes more sense.. one of creators who isn't just walking through the code and saying what the command does in the notebooks Matt is really someone who knows how to empower the other person
yes he is the best beast of explanation ❤ 💪 🥇 🔥
I'm glad that ollama went so far ahead and how creating a standards for open-source LLMs, like dockerfile-like specification files and so on.
Matt, your energy is so calm. I did fine tune with MLX, but I freaked myself out with all the steps and feel like it’s hard to do again.
When you explain it so nicely, my fear goes away and I’m ready again.
You’re spot on about the data prep is the “dark arts”. So true!!
Love it, thanks for sharing. It's great to see LLM fine-tuning become increasingly accessible to more people.
I live this kinda explanation full of details and step by step.
Thanks, for sharing!
I like your delivery stating it’s easy. I am going to try fine tuning and training with my new data
Thanks a lot for this tutorial Matt. It is by far the most straightforward fine tuning tutorial I have ever seen.
Thanks for pointing out that fine tuning is about teaching new behaviors, not injecting new data. So many misunderstand that.
Sorry if unrelated, am I the only one who thinks that Matt’s voice has that soothing-gentle-teacher like voice? Like I can hear him narrate for a natgeo documentary
Facts
Goat in teaching fine tuning
I was totally agreed with you sir, u are the most easier way to let me learn about mlx in the past 2 month I’m keep finding UA-cam for all the information. Thank you so much for the video.
Thanks great breakdown of the process!
A note about JSONL not being an array. It can be processed by old school unix tools like awk, grep, sed - and used in streaming data with unix pipes where lines are the delimiters. These tools don't do well with json array syntax on large datasets.
This is exactly correct. A JSON array, even if by convention has an object on each line, is not "valid" JSON unless the entire array, including the closing bracket, is pulled into memory. JSONL ensures that each line is its own mini valid JSON document.
I have heard many folks talk about doing a fine tuning to make a model work better with the aider tool. It seems like everyone is struggling to figure out what the dataset should be. It would be fantastic to get your thoughts on it!
Thanks for all the great content!
Thanks for the great video! Could you use an llm to generate question answer pairs for the dataset out of basic text or documents? Would be interested in such a video!
super clear and helpfull instruction, thanks so much! I think the jsonl format has to do with training with super large datasets, making it possible to handle them row by row?
Nicely done! It's worth noting that what Matt demonstrated is "Fine-Tuning with LoRA" and not Actual Fine Tuning. Low rank adaptation (LoRA) makes customising a model more accessible than actual fine tuning of a model by "freezing" the original weights and training a small subset of parameters.
Actual Fine-Tuning: Adjusts all parameters, requires significant resources, but yields high-quality results.
Low rank adaptation (LoRA): Trains fewer parameters using low-rank matrices, reducing memory and compute needs while maintaining quality.
Sure. But when most folks talk about fine tuning it’s lora.
Matt, this is utterly awesome and I can't thank you enough. I'd seen the compute resources people were using and the code and gone "that's just too time and money intensive to investigate further".
Now, I just need the script from terminator, a code interpreter and, oooh, 5 minutes?
Don't worry, I'll keep control of it...
amazing video matt thank you
Glad you enjoyed it!
Thanks I tried the mlx fine tune a few months ago. I think this mlx-lm might be more straightforward.
He said that fine-tuning is only for dealing with how the model responds but it's also for increasing domain expertise as far as I know
I need to say thank you I appreciate everything you're teaching us, you make things extremely easy to understand... Thank you.❤
This is a great vid! Especially if you're at least a hobbyist.
The best complete layperson onramp I've seen for fine-tuning? Is Cohere. And it is free. After that, dip into more of the dark arts. But to whet folks' whistle I usually point them there. It's their whole biz model. It quickly gets old, but you can whip up a trained, finetuned bot in a half day depending on dataset.
i so wish you could explain what loras are an how to do one. thank you for this amazing video i already fee much better
Hi Matt!
Excellent series! Love your no-hype/nonsense approach to education!
JFYI, in order to run mlx-lm against llama or mistral models on HF, you must first agree to the terms published on the HF in the repo of interest. Otherwise you mlx-lm.lora command will exit with
`mlx_lm.utils.ModelNotFoundError: Model not found for path or HF repo: mistralai/Mistral-7B-Instruct-v0.3.`
Cheers!
Better still to download it first
Hey Matt, did you record any video on Unsloth? Would love to see it 😀
Awesome video! Nice glasses! what's the camera setup you're currently using?
Thanks. you can find out about my entire setup with this video. It's been collected over years of doing this. ua-cam.com/video/LQe3DFjMYrE/v-deo.html
Thanks Matt, your explanations are effective and entertaining.
If you could in a future video , would you dive into more detail about fine-tuning? E.g., why would you want to, how to choose your data, etc. Thank you!
Amazing content, I will test it soon 🙏thanks!
Great video! 👍 Thanks
Clarity and digestability 100%
love your explanations. thank you !
I don't want to nessesarily train it on "how to respond" but I want to expand it's knowledge base of Python code so it knows "what" to respond with. For instance, I'd like it to have better knowledge of some of the more advanced concepts of Python. So do I fine tune to accomplish this or what is your recommendation. I'm just starting out learning and all I've been able to create is a mock jpeg file corruption recovery tool that doesn't actually work because jpeg is a lossy format to begin with. But it looks like it works. Lol
Thanks a lot for your fantastic videos. I'm actually using Unsloth to fine tune Llama3 for a text classification task. I'll be happy if you'll upload a video for such purposes
Great video, thanks!
I only wonder what's the way to add new data to the model if finetuning is more about changing the format.
My use case is like this: I use a framework which changes syntax with each new version I want the LLM to be updated with the newest documentation for the last version.
How can I do that?
If you could make a guide of installing Unsloth on Windows and train Llama 3.1 on Ollama to use my tone of voice to reply to emails AND have knowledge to answer the common questions I get, that would be a godsent! Thank you for the great videos!
I am interested in the idea of fine tuning, and I am starting to regularly come to your videos for stuff now that Ollama is my primary tool I am using to connect to other front end for serving my models. But jeez, i feel like an idiot sometimes with some of this stuff because this still feels complex to me. "Step 1" of curating the data honestly feels like the easy part to me.
I am curious about unsloth as its one I have looked at before but had decided to circle back to when I finally worked my way up to fine tuning. I am also curious about vision models, Llava or otherwise, I would be really curious to see how curating data for that would differ from an LLM.
Also, would be nice in future videos related to this so see a before and after test. I know we can assumed what you are suggesting is making an effect, but it would still be nice to see the results in action :)
I like your presentation ...Regarding the fine tuning heard unsloth I have seen many of the AI experts uses it. What is your thought on the same lines?
I didn’t use it because of the limited hw support. But I will next time
Thanks, Matt-super spot-on video as usual. You raised a doubt in my mind: You mentioned that fine-tuning is not suitable for adding new information to the original LLM (perhaps I misunderstood). This leaves me a bit perplexed, and I know it’s a debated issue within the community. I agree with you that the best use of fine-tuning is to personalize the style and tone, rather than being used in the "traditional" way to train older (pre-GPT) models like BERT. However, many people argue that fine-tuning could be an alternative to RAG for injecting specific domain knowledge into the LLM. Personally, I’ve never tried fine-tuning a model due to the costs, especially with cloud-based LLMs. In any case, I think it would be valuable to explore these topics further.
My hope is that fine-tuning could become a native feature in Ollama in the future.
Lastly, it would have been useful to see the fine-tuning JSONL data (at least an example). I have my own answer to your question: why JSONL? It might be because of its line-by-line simplicity in Unix pipe scripting.
What I read is that you can add knowledge but apparently it makes it slower.
@@technovangelistBy "slower," do you mean that the fine-tuned model has increased latency during inference compared to the original model? That's interesting-I’ve never heard about that before.
@@solyarisoftware I think Matt has been finetuning an instruct model. Afaik instruct finetuning is usually done after training the model on the next word prediction task which is the step where the general knowledge is injected into the weights. The next step, the instruct finetuning is supposed to make the model better at following instructions and producing responses aligned with user intent.
It’s generally well understood that fine is not well suited for adding new knowledge
@@technovangelist I agree :)
Thank you Matt for again a great helpful video. It would be great to see samples of the test.jsonl, valid.jsonl and test.jsonl files. Or a template of them. Thanks
I’m curious as to why the fine tuned can’t be for new knowledge?
Very useful, however
1) is there anything we can do with 16 gig on m1?
2) can you maybe show how to do it? Step-by-step: it is a little hard to understand the MLX part thanks
Can a model be fine tuned to improve the way a model uses tools? Can fine tuning be used for chain of thought? One example would be to take a number of prompts, then a good chain of thought to solve that type of problem?
Great tutorial. Can you do a non-mac version of this? I see things like qlora but I have no idea even where to start.
I plan to. Unfortunately most of the non Mac tools kinda suck.
Could you do the same thing as fine tuning by creating a RAG database of examples, and just use the off-the-shelf LLMs? This might make your application LLM-agnostic and futureproof. Thoughts on pros/cons?
Fine tuning and rag have different purposes. Rag adds new knowledge whereas fine tuning will mostly affect the way it outputs.
@@technovangelist Mmhh yes and no. Full-parameters finetuning is OK for adding new knowledge in a more systematic way (after all, it's just continued pretraining..).
parece até fácil olhando !
tks
Can you do one with unsloth or other free library? thanks
Well mlx is a free library but I would like to do some others
so if finetuning is how you make the model respond in a better style, how do you teach it more stuff? which are the best ways to make the AI learn more aside from RAG
I forgot to break up my data into the three sections (validate, train, test) will the model work the same?
Nice. I just did not get when to use the different test and validate files in the process.
Would love to see you try out InstructLab
thank you Matt, you are amazing. As I know, mlx is for apple silicone, what about finetune on nvidia rtx card? which library should we use ? can you make a video for this please ?
Yup. I mentioned that I did this first for apple silicon. And I intend to do the same thing for unsling and maybe axolotl which are windows and Linux based
hi, why do i need a validation dataset and a test dataset? can i create them but let them empty? because i don’t understand anyway how validation and test would work for a llm
Thanks you for this
Thanks for the comment
dark arts, lol. Love your vids, man.
ok so i created the three sections, but now get a data formatting error heres a sample of some data of mine: {"prompt": "info.", "response": "info"} what should be different here?
Use the format I used in the video. The. I show it in the next one. Just a text key.
jsonl is used to read line by line, it is easier for python, because it is reading line by line as far as i know. I.e. 1 iteration === 1 json from your dataset. Plus datasets are huge and reading whole json and parse it all in one go will take a decade and probably will crash your runtime.
Ahhh, ok. So it’s accommodating for the weaknesses of Python.
@@technovangelist in fairness, it also allows you to do things without needing to parse the whole file - like cat something.jsonl | sort > sorted.jsonl or cat something.jsonl | head -n 10 > 10somethings.jsonl
In this case, I imagine it’s convenient for shuffling the entries, but the main generic advantage over a big json list is that you don’t need to read the entire file to begin parsing it (which is a really nice language-agnostic property for files that might become extremely large)
They’re definitely weird at first blush, but actually kinda clever and elegant for some use-cases IMO
I tried to fine tune llama3.1 on windows since I need nvidia gpu. Such a nightmare. I still didn’t figure out why my llm is not able to speak anymore, it just reply’s data that was in my dataset
I would love to learn to fine tune Anthropic's Claude sonnet 3.5 on Jan ai... i'm finding it challenging
that’s probably a topic that wouldn't happen here. Most of this channel focuses on local ai solutions due to the security and privacy risks of most online models.
Thanks for the awesome video! Would you know anything about using ubuntu/intel arc with ollama. If so would you consider a guide one beautiful day :)
I haven’t played with any of the arc cards yet. I need to find a way to play with those
Does unsloth worth that hype?
Still not sure what the data file looks like for the framework? Is it a dataset? the below does not seem to work:
"text": "This is the first piece of text."}
{"text": "Here is another piece of text."}
{"text": "More text data for fine-tuning."}
Yup that’s what I showed in the video. Well, except you missed the first bracket
@@technovangelist Agreed. The thing i learned is that data is the directory where you put train.jsonl test.jsonl and valid.jsonl with the format you describe. Thank you!
sir i can't understand the way dataset jsonl file should be can u pls give me 1 block of the dataset jsonl file as example. i can't understand how to make 'em pls help me else someone out here pls help me too
Take a look at the second video on this
@@technovangelist which one sir "optimize your AI models" else which one sir can u pls say sir 🙇🛐
The other one with fine tuning in the name
@@technovangelist ok sir thanks a lot
Error: json: cannot unmarshal array into Go struct field .model.merges of type string
you need to provide more info. Where did you get this? what version of ollama? what platform? how was it installed. Best to do all this on the discord.
Running fine-tuning is easy, but getting LLM to do what you are fine-tuning for may be not so easy and at times even very hard.
Example for datasets?
I need to understand how json works
If you only can change the style of the answer why bother with fine tuning, I don’t need the answers to be like a pirate , why would you need this for creating an enterprise level application? Is rag the way to go for this ?
Tweaking the style is a very important aspect for most enterprises. Some need it to respond as Sql every time or json or functions. Those don’t need new knowledge but rather told how the model should respond.
Thanks! I can’t tell you how much I dislike Jupyter notebooks.
Matt, I had written down a long comment, but UA-cam deletes anything that has links to platform it doesn't like, probably. If you have time and will, please read my replies to your twitter thread related to this video, thanks.
Finetune with crosswords so the llm know how many characters in his response
I also hate jupyter notebooks... Agree that it is the worst for teaching... I always convert it to python file in the end and getting rid of all useless stuff...
mmmistral 😁
It will be good if you code rather than showing snippets
Mmmmmmmmistral hahaha 😂👏
Python Notebooks have to be the worst format for teaching - Amen to that!
hey, cool channel, but this is too complicated. i want a 1-click solution. i want to do 1 click, and the ai should exactly learn how my texting style is. i want to use it to prank whatsapp friends.
There are options for that. The one I have seen costs about 200 to 300 usd per fine tune run. Or you can spend 10 minutes doing it this was for free. Anyone can do this.
@@technovangelist 10 minutes is too long, i will pay the 300 dollars
@@mal-avcisi9783 😂
Von View
Yeah this is all gibberish. Convert your scripts json what?!