WebLLM: A high-performance in-browser LLM Inference engine

Поділитися
Вставка
  • Опубліковано 2 січ 2025

КОМЕНТАРІ •

  • @erkintek
    @erkintek Місяць тому +6

    3 years ago I run some models in tensor flow js, also on mobile. They were not performat but way cheaper than on server.

    • @jasonmayes_goog
      @jasonmayes_goog Місяць тому +1

      Humble beginnings back in 2018, but many of the TFJS Models are very fast now with WebGPU and such.

  • @ekolteenarp
    @ekolteenarp Місяць тому +1

    Excellent presentation! 👏

  • @dev-play
    @dev-play Місяць тому +1

    If models can be separated by purpose, many local on-device models are likely to emerge soon.

  • @SriniVenkata-my4uw
    @SriniVenkata-my4uw Місяць тому +2

    Excellent video

  • @RockyTheDog756
    @RockyTheDog756 Місяць тому +1

    When will it become the part of specification?

    • @Rusputin-eo6tv
      @Rusputin-eo6tv Місяць тому

      GenAi was not meant to be specialize. You need a set of quality data to fine-tune AI and make it specialize.

  • @jason77nhri
    @jason77nhri Місяць тому

    Thank you for your sharing
    However, I have a few questions about WebLLM that I’d like to ask:
    Which country or company/team developed WebLLM?
    Does WebLLM work based on the principle of running LLMs locally on a computer?
    However, it seems to allow the use of WebLLM's default models from website.
    In this case, is it considered local operation? What about the privacy risks of conversation data?
    If WebLLM uses default models from the internet, can I still use it completely offline after disconnecting from the network?
    Can WebLLM utilize locally downloaded GGUF models, similar to tools like Ollama or LM Studio?
    Thank you!

    • @jasonmayes_goog
      @jasonmayes_goog 3 дні тому +1

      Correct WebLLM works entirerly locally in the browser on your own hardware - no cloud.

  • @alexkizer639
    @alexkizer639 9 днів тому

    Awesome!

  • @khangvutien2538
    @khangvutien2538 Місяць тому +1

    Like with NotebookLM, Google is doing a great job for the AI community. We the public regret even more that the corporate marketing department is so lousy to promote the results. Will it be always like so? 😢

    • @chaomingli6428
      @chaomingli6428 Місяць тому +2

      They make easy money from search ads monopoly. That makes their whole sales and marketing very sloppy. It’s a culture problem that won’t be fixed easily.

    • @khangvutien2538
      @khangvutien2538 Місяць тому

      @@chaomingli6428 I can't resist comparing Google to Xerox that let their PARC (Palo Alto Research Center) invent the mouse and the WYSIWYG concept and did nothing with it. Who still remembers Xerox?

  • @nri_raj
    @nri_raj Місяць тому +2

    3rd like 😂

  • @mariancaikovski8638
    @mariancaikovski8638 Місяць тому

    4th