The ALPACA Code explained: Self-instruct fine-tuning of LLMs

Поділитися
Вставка
  • Опубліковано 17 лис 2024

КОМЕНТАРІ • 19

  • @anshtanwar1813
    @anshtanwar1813 4 місяці тому

    Great series, really enjoyed it.

  • @carefir
    @carefir Рік тому

    You're basically just reading out the Stanford alpaca codes without any understanding. Great!

    • @code4AI
      @code4AI  Рік тому

      Thank you for leaving a GREAT on my videos! I know I am fantastic and since you need to attack me verbally, never mind, if it helps you, I am here for you. Just a hint: this is part three of the series, so do not look at the explanation in the first two videos. You might get humiliated by your comment. See you!

    • @carefir
      @carefir Рік тому

      @@code4AI I had to read the code myself even just to figure out how prompt.txt is used in conjunction with seed_tasks.jsonl (something which your video only made more confusing).
      But you want to believe what the other commenters wrote... that you're going a great job. What can I say? Great to be you!

    • @carefir
      @carefir Рік тому

      @@code4AI Oh, ya. I watched the other two parts as well. Needless to say, I am watching no more. Reading the code is so much more effective.

    • @code4AI
      @code4AI  Рік тому +1

      NO! You ...YOU had to read the code yourself!? I am shocked! If I would have known before, I would have send somebody over to do the reading for you! Now I understand your complaints so much better .... enjoy your day! And thanks for the great job compliment!

    • @carefir
      @carefir Рік тому

      @@code4AI Dude, your video is literally titled "The ALPACA Code explained"

  • @monx
    @monx Рік тому +1

    this is great. I think what's missing from the explanation 20:40 and forward is the detail about how the supervised labels (targets) are used during training. is it correct to say: the input contains the full string, source+target concatenated. but the source portion of the prompt is ignored when calculating the loss. in other words, the token prediction begins only after source_len tokens. seems reasonable but I'm new here.

  • @brunoesteves3367
    @brunoesteves3367 Рік тому

    Can you please make a "Hands On" video making your own fine-tuning based on their process?

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Рік тому

    awesome series

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Рік тому +4

    if Google can't place commercial restriction on the output of Google search, why can OpenAI place limits on their search results? Especially, given that their model is trained on the public web.

    • @code4AI
      @code4AI  Рік тому

      American law. ...and a corporate lawyer takes $1500 per hour.

    • @henkhbit5748
      @henkhbit5748 Рік тому

      @@code4AI Maybe the country regulators, for example the EU, should step in banned chatgpt if they dont open source it 🤔😁 and most probably they used the input use by the chatgpt to fine tune their model also by let us pay for that😆

    • @dinkusstinkus4396
      @dinkusstinkus4396 Рік тому

      It's not illegal to read a book, just because it's illegal to steal one

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Рік тому

    Are the words "input" and "output" treated as special words in LLM?

    • @code4AI
      @code4AI  Рік тому

      No. Take "in" and "out" consistently throughout your code sequences, if you prefer.

  • @maxgom4734
    @maxgom4734 Рік тому +1

    It uses openai api right? And it's not free