Thanks for your video Anton. Been watching ur vids. What do you think about K230 board? I'm still unsure whether to go with SG2002 or K230 for prod quantities. This board has double the RAM 512MB, and the one from 01 Studio has 1GB RAM. 😉 Can we run mistral 7b or too small. In that case I go with Orange Pi 5 Max.
I think that company that was produsing k210/K230/K510 switched from AI to cripto miners - www.canaan.io/ So, I prefer not to test platforms without perspective of use in production:)
@AntonMaltsev I see.. didnt know that. Good catch. Too bad.. that SoC has alot of stuff in it.. another thing I've been watching is Effinix FPGA, they're releasing the Topaz line, promising high vol applications/cost. Their selling point is high vol costing (which is unlike traditional FPGAs, Xilinx et al) I mean I used to design in FPGA only for quantities in the hundreds... for avionics system, now they're talking consumer products. And hard RISCV cores (just like microchip FPGA Polarfire) plus the ability for implementation of custom RISCV instruction in FPGA fabric for TinyML libs. Nice right... I emailed the company hoping they will release open source board like BeagleV-fire. Have you done matmul on FPGA fabric for LLM it will be crazy fast right..?
Anton, Very informative video! what kind of board do you suggest I am looking for a security camera based detection system in realtime .It has to be battery powered to detect friend or foe .I am looking at 1000-1500 quantity for production also ! Thank you
@@PavanSBV-y3u I recommend you consult an AI researcher who can choose the algorithm based on your dataset and algorithm. There is almost no difference between different YOLO networks, and YOLOv8 is not the best network from an export point. Also, there are many questions about inference speed, latency, etc. But if your input is correct, RockChip (3566,3568), some NXP boards, MediaTek, Sophon and Qualcomm are okay. But next, you need to go through my guide and consider the board based on your location, industry, type of development, etc.
I like x86 boards now way better than various ARM based. N100 very cheap now and faster than most ARM SBCs. Plus you can add GPUs easily with CUDA cores so you can run much greater variety of models such as vision transformers. Much better support than anything else. I just wish you could buy tiny mobile style GPUs like you get in laptops rather than gigantic pcie cards.
We did this in 2018. But nowadays it will be more expensive. The cheapest CUDA device today is Jetson. A suitable N100 with good cooling will cost around 150$, which is compatible with RK3588
Anton, your videos are fantastic. Keep up the good work!
nice video, thanks
Thanks for your video Anton. Been watching ur vids. What do you think about K230 board? I'm still unsure whether to go with SG2002 or K230 for prod quantities. This board has double the RAM 512MB, and the one from 01 Studio has 1GB RAM. 😉 Can we run mistral 7b or too small. In that case I go with Orange Pi 5 Max.
I think that company that was produsing k210/K230/K510 switched from AI to cripto miners - www.canaan.io/
So, I prefer not to test platforms without perspective of use in production:)
@AntonMaltsev I see.. didnt know that. Good catch. Too bad.. that SoC has alot of stuff in it.. another thing I've been watching is Effinix FPGA, they're releasing the Topaz line, promising high vol applications/cost. Their selling point is high vol costing (which is unlike traditional FPGAs, Xilinx et al) I mean I used to design in FPGA only for quantities in the hundreds... for avionics system, now they're talking consumer products. And hard RISCV cores (just like microchip FPGA Polarfire) plus the ability for implementation of custom RISCV instruction in FPGA fabric for TinyML libs. Nice right... I emailed the company hoping they will release open source board like BeagleV-fire. Have you done matmul on FPGA fabric for LLM it will be crazy fast right..?
Anton, Very informative video! what kind of board do you suggest I am looking for a security camera based detection system in realtime .It has to be battery powered to detect friend or foe .I am looking at 1000-1500 quantity for production also ! Thank you
Hi!
Depending on what kind of neural networks you have, how much battery life you need, etc.
@@AntonMaltsev We want to use YOLO v8 or above and battery life of approx 12 hours I am looking at
@@PavanSBV-y3u
I recommend you consult an AI researcher who can choose the algorithm based on your dataset and algorithm.
There is almost no difference between different YOLO networks, and YOLOv8 is not the best network from an export point. Also, there are many questions about inference speed, latency, etc.
But if your input is correct, RockChip (3566,3568), some NXP boards, MediaTek, Sophon and Qualcomm are okay.
But next, you need to go through my guide and consider the board based on your location, industry, type of development, etc.
I like x86 boards now way better than various ARM based. N100 very cheap now and faster than most ARM SBCs. Plus you can add GPUs easily with CUDA cores so you can run much greater variety of models such as vision transformers. Much better support than anything else. I just wish you could buy tiny mobile style GPUs like you get in laptops rather than gigantic pcie cards.
We did this in 2018. But nowadays it will be more expensive.
The cheapest CUDA device today is Jetson.
A suitable N100 with good cooling will cost around 150$, which is compatible with RK3588
So, it's one of the options, but not the single one that is possible.
Also, x86 is less efficient for the same power consumption.