Hello Robert thanks again for this good tutorial :). I tried to create such VM in the region you mention and in many other regions, but every time I get the error saying it's currently unavailable (GPU shortage). I also encounter shortage with Nvidia T4. Did you find easily resources? Or it's because you are a GCP partner so you access resources in priority?
Hi Kim, good to hear from you again :). Well, I did not encounter GPU shortages, but unavailability of the required „a2-highgpu-2g“ machine type. What I did was to reach out to Google at Twitter and the very next day it worked for me: x.com/robertschmidpmp/status/1696870241584775368?s=46&t=5SAiC-TXlqIYFkhMf8DAMg Not sure it was by accident, however feel free to respond to my tweet. Alternatively you can provide feedback to Google from the Google Cloud Console. Or you can send me an email with your account details and I will reach out to my partner manager at Google directly.
I'm trying to create 100 000 reliable tutorials for hundred complex software like photoshop, blender, da vinci resolve etc.. Llama and gpt don't give reliable answer unfortunately. Do you think finetuning llama 7b would be enough (compared to 70b)? Do you know how much time/data that would take? I also heard about embedding but couldn't get it to work on large dataset. Would that be a better option? We have at least 40 000 pages of documentation I don't know what the better approach is.
@@BlueAntoinette So for you embedding is enough to answer complex question that rely on multiple part of a 800 page technical documentation? We don't have to finetune?
@@finnsteur5639 Personally I would check that out first and test it with available open source models. If you don‘t get relevant result you still can try to fine tune.
@@finnsteur5639 I've now created a new solution that maybe can help you in this regard. Learn more in my latest video ua-cam.com/video/n63SDeQzwHc/v-deo.html
Love it!!!!!!!!!!!!!!!
Thx!
Hello Robert thanks again for this good tutorial :). I tried to create such VM in the region you mention and in many other regions, but every time I get the error saying it's currently unavailable (GPU shortage). I also encounter shortage with Nvidia T4. Did you find easily resources? Or it's because you are a GCP partner so you access resources in priority?
Hi Kim, good to hear from you again :). Well, I did not encounter GPU shortages, but unavailability of the required „a2-highgpu-2g“ machine type. What I did was to reach out to Google at Twitter and the very next day it worked for me: x.com/robertschmidpmp/status/1696870241584775368?s=46&t=5SAiC-TXlqIYFkhMf8DAMg
Not sure it was by accident, however feel free to respond to my tweet. Alternatively you can provide feedback to Google from the Google Cloud Console. Or you can send me an email with your account details and I will reach out to my partner manager at Google directly.
Would it be able to just run llama
Yes, same approach
I'm trying to create 100 000 reliable tutorials for hundred complex software like photoshop, blender, da vinci resolve etc.. Llama and gpt don't give reliable answer unfortunately. Do you think finetuning llama 7b would be enough (compared to 70b)? Do you know how much time/data that would take?
I also heard about embedding but couldn't get it to work on large dataset. Would that be a better option? We have at least 40 000 pages of documentation I don't know what the better approach is.
Check out HuggingFaceEmbeddings (SentenceTransformers) together with a vector store like Chroma
@@BlueAntoinette So for you embedding is enough to answer complex question that rely on multiple part of a 800 page technical documentation? We don't have to finetune?
@@finnsteur5639 Personally I would check that out first and test it with available open source models. If you don‘t get relevant result you still can try to fine tune.
@@finnsteur5639 I've now created a new solution that maybe can help you in this regard. Learn more in my latest video ua-cam.com/video/n63SDeQzwHc/v-deo.html