Mind-bending new programming language for GPUs just dropped...
Вставка
- Опубліковано 15 чер 2024
- What is the Bend programming language for parallel computing? Let's take a first look at Bend and how it uses a Python-like syntax to write high performance code that can run on the GPU.
#programming #tech #thecodereport
💬 Chat with Me on Discord
/ discord
🔗 Resources
Bend Language GitHub github.com/HigherOrderCO/Bend
CUDA in 100 Seconds • Nvidia CUDA in 100 Sec...
Recursion in 100 Seconds • Recursion in 100 Seconds
🔥 Get More Content - Upgrade to PRO
Upgrade at fireship.io/pro
Use code YT25 for 25% off PRO access
🎨 My Editor Settings
- Atom One Dark
- vscode-icons
- Fira Code Font
🔖 Topics Covered
- What is Bend language?
- Parallelism vs Concurrency
- Rust programming language projects
- CUDA alternatives
- How to run code on a GPU - Наука та технологія
Time to become a code bender.
But what about non-benders? ;-)
@@igorthelight Folders you mean?
@@mishanya1162 is that with or without hard R
Everything changed when the Bug nation attacked
Finally, I have a chance to step into my father's shoes. Grow a pair. Live the life I was born to live.
Time to add 10 years of experience to my resume/CV
Can we stop making this joke every fucking video
My work experience is multithreaded, that's how
Probably 4 minutes in 1314000 threads
@@lionlike5856 We would if it wouldn't still be true
Recruiters be like: Do you have 10 years experience in bend?
That video was *extremely* well done. Seems like someone has read the docs! Thanks for making it. We're long-term fans of your content here at HOC. If you or any other content creator wants to reach out, we're available to talk about the technology and answer any questions.
We know there's a lot of work to do, but we're excited about it. You can expect Bend to become faster with every release. There are also many missing features (64-bit numbers, larger memory limit, etc.) that will be added very soon!
Let's do it. 🚀🚀
excited*
Thank you for commenting! Shouldn't it be possible to model for loops to work like the tree model that Jeff showed, or am I missing something?
This might legitimately make my life so much easier. It looks like I have some docs and playing around to do.
@@mysterry2000In the circles that would be using this language, the “fold” keyword and everything that comes along with it is actually more understandable that calling it a loop. When he said no loops I originally was confused but as soon as he said it’s replaced by “fold” I immediately got it. It’s just one of those things :/
Folds have been a mainstay of the functional programming world for a very long time.
I remember teaching students how to implement folds in Python a few years back and it blowing their minds. The basic concept that all loops are simply a subset of a wider class of iterative constructs called folds is transformational - I love the simplicity and inherent understanding of a fold.
This new concept of a bend is incredibly interesting, as like the fold it appears to be able to emulate any kind of loop or iterative function, but instead of going from many to one, goes from one to many. I will be playing around with this over the next few weeks I am sure!
Yeah, these concepts are really fascinating! The paper "Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire" from 1991 gives a deep-dive in the different kinds of recursion schemas you can work with, though unfortunately not in the most accessible way possible. In that paper a fold is called a catamorphism, while a bend/unfold is called an anamorphism. Having these concepts built into the language is very interesting, as it both gives them more directly to users and allows the compiler to optimize them directly.
Another interesting related concept is the Church-encoding of data-types, allows any algebraic data type to be encoded as a (higher order) function. The Church-encoding of a data type is literally just the data applied to its fold. This is actually used for list-fusions in Haskell, where the lists are temporarily converted church-encoding, which can allow skipping construction of intermediate lists that would've been immediately folded away. Afterwards, you can recover the original data by giving the fold the original constructors. As the video mentioned, a fold is basically just a search-and-replace, where each constructor is replaced by some function, so replacing them with the original constructors is obviously a no-op.
I'm not sure if there is a corresponding concept to the Church-encoding for bends, but it feels like there should be one
Looking at the Haskell module Data.Fix, I believe that what it calls the Least Fixpoint
data Mu f = Mu { unMu :: forall a. (f a -> a) -> a }
corresponds to the church-encoding, which would mean that the Greatest Fixpoint is the corresponding thing for a bend
data Nu f = forall a. Nu (a -> f a) a
both of these are equivalent to the basic recursive fixpoint
data Fix f = Fix { unFix :: f (Fix f) }
which allows you to write recursive data types without explicit recursion, which also means that you get the folds and bends for free
In the bend language, you wouldn't need the Fix datatype, since folds and bends are built into the language
BTW you also know that FOLDS can be done not only on lists and graphs, they can be done on AST of code, and the fold on the AST is what is reffered to as an interpreter (the LISP eval function) O_o. Also in Haskell Monads are cata mappables, where the cata operation of monoids (i.e: (0,+,sum),([],++,concat),(true,&&,and) is a fold operation. Monads(programmable semicolons; chainable functions) literally are calling evaluate then map on your embedded languages as ASTs, and they split the embedded languages into their semantics so that Monads are programmable semantics.
You talk like somebody who thinks IQ is an important number.
@@Tony-ow9bo huh, what? why?
Watched this video on 8000 Cores in parallel, added 500h of bend experience to my resume
I just knew you actually calculated the time x cores, instead of giving random numbers
5head 🧠
I will hire you for my job listing which requires 10 years of production bend programming experience
ACKCHYUALLY... 500h = 30,000m = 4m video x 7,500 cores 😋
500 ? You need to start with 50 000 hours for entry level positions, or no one will take you seriously.
“1 week of problem, instead 7 days with 7 computers” 🤣🤣🤣
doesn't make any sense, especially in the context of parallel computing
wooosh or no?
@@Qohist it's definitely a r/wooosh
@@deadlock107 The joke is that the time saved running parallel was spent on the extra complexity of developing a parallel solution.
Thanks it wasn't stated in seconds😂
We needed this 17 years ago.
It is happening 17 years too late.
Imagine what we could have done with those core2quads and Phenoms !!
🥺 My Phenom2 x4 955BE ran for over 10 years 💪& the Motherboard died before the CPU!
Was a really great Piece of Silicon!
People are rediscovering functional programming without knowing it.
I think it's more accurate to say that people are copying features from Haskell without giving it enough credit.
@@leftaroundabout the description for bend literally shouts out haskell
@@bearwynn bend is running on interaction combinators, that are optimal implementation of lambda calculus, which haskell is running on
@@leftaroundaboutyou have no idea what you’re talking about
@@keepmehomeplease you sure excel at phrasing criticism diplomatically, hm?
But, what's your point? There can't be much doubt that the designers of Bend were well aware of Haskell from the start, and so were the designers of, say, Rust. So who do you mean that supposedly didn't know about functional programming?
someone tomorrow will start recruiting people with 10 years of experience in this
This reminds me. Fuck the modern industry.
With salary being "valuable experience"
😂
yeah, lol. The only reasons that companies will ask for that is, either because they want the employee to do the work of making the company look better on statistics, or because they want the credentials to be impossible so they have an excuse to cut costs with dirt-cheap outsourced labor (or, with a textbook "do less with more effort" lack of experience that only an executive could have, with a GPT AI model)
@@AMan-xz7tx Problem is, that at a certain level this approach just won't work anymore. If a company demands unattainable experience , the few candidates that will have a somewhat relevant portfolio will demand a lot more than the company is willing to pay.
Saw this on Twitter/X and was like 'no way, this is a scam'. Then I saw their pfp was an anime avatar and ngl kinda trusted them immediately lmao
😂😂
Certificate of trust spotted
Same, I saw it from anime pfp account yesterday lol
Anime nerds single handedly pushing humanity 100s of years ahead just to make their waifus real in their lifetime 💪
why are weeaboos like that lol
That diagram showing the combinators being untangled is just so awesome.
You should do a video about Lean, it's a language that can understand mathematical logic and be used to prove theorems. It also has a very unique type system.
There’s also a tiny game where you have to prove some stuff about natural numbers online.
He should do a video about doing some actual lean
I LOVE LEAN, CHARLIE. I LOVE LEAN!
Next he should do a video about Gleam
But GPU guy said we will never have to code again.
he did say that didn't he? LOL
LMAO GPU guy
*the* GPU guy
all hail the lord mighty Jensen huang
@@Eleganttf2 shhh it's an insider joke people aren't supposed to know 😂
Yoooo new programming thing dropped that's not a new js library finally letsgoo
not a js library yet*
@@DeltaByte chatgpt how do I compile a rust based programming language for wasm
just started working on the JS binding to bend so we can write bend in js using bun
Its not AI!
I'm not super on top of JS world, but haven't new frameworks kinda stopped releasing lately?
Yoo!! The guy who wrote that paper you mentioned at 2:15 was one of my teachers at my first year of my maths bachelor, that's sick
Props for covering this project
7 days 7 computers joke was hilarious 🤣🤣🤣
Well, that's somewhat true for some cases. As this language heavily relies on recursion which takes a lot of resources & time for the same thing can be done with a normal for-loop but since it uses parallel computing it makes up for that. And that 7 computer analogy was used there because it can use all cores of cpu/gpu unlike what traditional languages do.
I initially thought he made a joke about crypto miners, both before and after I got the joke it was still funny
I liked his random numbers ❤
did it lol , buggy distributed node code
Was it a typo or a satirical analysis of how we are becoming frozen in a block of new good-intended time-savers?
0:30 So THAT'S why my CUDA code wasn't running close to optimally... it's not my sophomoric understanding of algebra, it was that I wasn't leveraging REGEX!!!
how else will you parse... large text files...
Simply ripgrep your way to Blazingly Fast Code™, young padawan.
"and now you have two problems"
you now have a sophomoric understanding of regex.
@@stephenkolostyak4087 how else will you parse (X)HTML?
I'm not in tech but I've been following this one after a few people on Haskell Twitter mentioned it, the first of whom exclaimed something along the lines of "this guy is turning GPUs into real modern LISP machines!" which is exciting in and of itself, and then I think Ed Kmett engaged with something indicating that this does reduction of something akin to the lambda calculus really efficiently with GPU acceleration which is REALLY exciting
oooooooooooooooh.
I just went down the lambda calculus rabbit hole again.
ooooooooooooooooooooooooooooooooooooooooooooooooooh. This is VERY interesting (if true).
the way you talk about code is so satisfying
Parallel Language: Check.
Supports GPU: Check.
Built with Rust(has thread safety, fast, beautiful, runs everywhere): What? ALL IN. let's integrate to Python and push to production now.
Python Integration is a must unfortunately. I don't like that it is necessary, but a lot of people are using it
rust is as thread ""safe"" as any language, the compiler is simply more restrictive about mutations
in what world rust is beautiful? lmao
@@marcs9451 thats why bend doesnt even allow mutations in the first place.
It's neat but putting it in production shouldn't happen. Your computer isnt made to run one python app, it runs an os and all sorts, if your app can run in one thread to the satisfaction of people do it that way, let the computer manage it.
Imagine running this on cloud servers. They'll hate you for maxing out their resources constantly lol
You could make them hate you in any language.
Bend is just a little bit easier ;-)
@@igorthelight A lot a bit easier my friend, a lot a bit. The amount of money my company has lost/wasted on parallelism issues is honestly insane. Time to present this to my team...
Cloud servers run virtual machines on hypervisors so each customer can only run on as many threads as the virtual machine is configured for, and virtual machines come in different sizes like small, medium and big depending on which one you pay for.
You mean the CFO or however set the cloud spend budget would hate you. CSP's would love you for this.
@@igorthelight you might even say that you can get them "bent" out of shape pretty easily
1:42 “two completely random numbers”
Mhmmmmmm
That quite comfy, that by modifying launching argument, you can specify on the fly, if you would like to use single or multithreaded and either CPU or GPU!
Thanks for the video!
1:20 “may even lead to conflicts with demons”😂
daemon, as apparently some dialects call demons daemons
@@paulstelian97 I think that was the whole joke and thats why he is laughing about it...
@@paulstelian97🤓
@@paulstelian97 I think it might be referencing the famous "nasal demons" meme from 1992 on comp.std.c noting how undefined behavior (which is common in incorrect parallel programs) can permit compilers to do arbitrarily strange things, giving "make demons fly out of your nose" as an example.
@@---..
Neat. Thanks.
takes the order, cleans the toilet and cooks the food, in that order lmao 😂😂
ngl that someone looks like you, respectfully.
@@mu3a.d215 delete this
In parallel now they can clean the toilet 3x before cooking the order.
@@mu3a.d215 that's really ignorant, if the toilet doesn't get cleaned what are we going to cook in?
Smh these damn customers can't even appreciate concurrency
I heard a lot of positive stuff about you and then I met this video. First video of yours i am watching... Totally hooked and I subscribed before I even finished watchng. Thanx you.
The parallelism promise sounds revolutionary for computing. The way you explained the workings makes it seem less complex. Looking forward to exploring more about this language.
Parallelism has been a thing since as early as computers had multiple CPUs, which is almost as soon as computers became a thing. The difference this makes, is that being a room temperature IQ chatGPT user does not preclude you from writing multithreaded code.
The multi-threaded example on Python isn't correct, the code shown in the video around 1:11 is creating Python threads which are executed under GIL meaning that only one thread is executed at a time under a single system thread (being Python interpreter itself). So while providing multi-threaded languages feeling, the threading module in Python is not suitable for CPU bound tasks and you'd have to mingle your software parts around to achieve parallel execution on multiple threads/cpus.
🤓 well ackchyually
Thank you, I learned something
Exactly what I wanted to say.
Also worth noting, there is a multiprocessing library in Python, which does get around the GIL.
@@kjgoebel7098yes but python multiprocessing cannot exploit hyperthreaded architectures in the same way multithreading does in C/C++ for example. Also threads are more lightweight and efficient than processes.
The first video on all of youtube about this language. Nice.
the picture used for the daemon conflict was perfect 🤣
You're in a league of your own bro, these videos are hella informative and have me rolling lmao
I’m just gonna stick with C, like God intended.
the Holy-C?
O yes God hates people.
@@RawFish2DChannel The Holy C.
Temple OS this is the way
God's Language
Babe wake up Fireship uploaded
Your babe is with me. she's had a good time.
@@karanr3ddy
Bad joke :/
@@karanr3ddybruh... that was way too cringe.
Your babe is sleeping with me 😴
The babe is woken. You, however, are sleeping on the couch tonight
I love that he still puts the “Hi Mom” references in. Gets me every time.
Love your humor, 1 week in serial or 7 days in parallel with 7 different pcs
That's actually so sick. Gonna check it out now
it's VERY early stages. It's probably not the best dev experience right now
4 minutes and one poo later. I now have 10 years experience of Bend. Thanks fireship 🔥
they won't hire you unless there's streetcam evidence
How do you poo so fast?
@@ivandenkov7446 Usually 60 years of training (accelerated at triple rate). You need to get your pooping game up mate.
Sorry but the industry is already ages ahead of you and left you behind, you're gonna have to find a new specialty
Holy moly you made this fast!
Does this also support multiple computers? In our CS department we use a lot MPI, which allows for massive parallelism, and bend would be pretty great as a higher level substitute...
0:40 I so wanted to see that program written as a bunch of threads each writing one character, with synchronization between them.
look into sleep sort
@@last8exile Hah, I think I already get it. Clever (but potentially a looong time until the last element gets appended).
Amazing to see this featuring on fireship
crazy stuff from the HOC guys
Nice knowledge about parallel programming
Time to fold 10 years of bend to my resume
"Hi mom!" ❤
0:40
rip for his mom
😢 I’m sure she’s proud
Good content bro!
Funilly enough, I just spent my afternoon reading about Bend, and then stumbled upon this video! It's definitelly very interesting, and it seems promising. Honestly, I wish this had come out 3 weeks ago, maybe it could have saved a project of mine...
Could whatever techniques used to implement fold/bend be ported over to other programming languages?
Will this help me build my todo app
Yes if it's a ToDo app for all your parallel universe personas
Brazil mentioned! Let’s gooo!!
France mentioned! Let’s gooo!!
This is the channel I always watch to feel an intellectual boost while knowing that I don’t understand a thing anyways. Thanks man
You dropped the mic and I dropped my jaw. Insane stuff.
APL was made with built-in compatibility with parallel algorithms and reasoning. It could do all of it on a modern interpreter.
Including Cuda? ;-)
This sounds like a nightmare for compiler bugs
This is it! Time to recode my portfolio that I haven't finished yet.
I just love it Jeff .... This is exactly the kind of content that I am interested in =)
I DID NOT UNDERSTAND I need a follow-up tutorial on folds and bends
Edit: I read the github page and I firmly believe that nobody will ever understand folds and bends.
@@sortysciaofiscia gotta ask gpt4 to summarize for us simpletons xD
1:14 But Python still runs a single thread at a time though. This isn't a good example, this will only help when you're limited by waiting for I/O. Compute workloads will be just as fast as without threading. The GIL (Global Interpreter Lock) only allows 1 thread to run at a time. There are ways around this, such as the multiprocessing library but each method has its own caveats.
Great presentation thanks 😊😊😊
Sounds fun when it works. Good luck debugging it.
1:02 "handling one instruction per cycle"
*** CISC has left the chat ***
Uhhh... x86 has been doing more than one instruction per cycle for decades and I'm pretty sure the only ISA more CISC that it was VAX...
DIV over in a corner ruining everything by taking multiple cycles on the majority of architectures; Unless it's an integer division by 2 cause then you can typically get away with a bit shift to the right
@@devnom9143 Haha, beat me to it. The Motorola 68HC11 microcontroller takes 41 cycles (!) for an integer divide.
@@devnom9143 Haha, beat me to it. IDIV takes a whopping 41 cycles on a M68HC11 microcontroller.
@@devnom9143 Haha, yes! I once worked on a microcontroller that took ~40 cycles for a DIV instruction.
2:18 "high level languages like python and haskell" two very similar languages on the same difficulty level XD
To be fair, I feel like much of Haskell's difficulty arises from its distance from imperative languages the majority of programmers are used to. Well, that and the relative lack of ide support.
@@traveller23e I think the current IDE support through Haskell Language Server is really good! I've had some issues with it refusing to start in the past and such, but when it works it's excellent! Installing it via the vscode extension is also really easy
"High level" has nothing to do with the difficultly of the language. It's referring to where the code is being applied. Low level languages would be languages which code instructions closer to the hardware of the machine, closer to the 1s and 0s which pass through the logic gates. This tends to be extremely specific by virtue of how information is processed at low levels. But, by virtue of said specificity, this makes them require more effort than a higher-level language. High level languages, on the other hand, are languages that are designed for humans to understand, and therefore are further abstracted from the low-level operations. This comes at the cost of less precise control, memory efficiency, and slower speed of operations, but at the same time allows programmers to complete more work than they would given the same amount of effort in a lower-level language.
@@traveller23e Also, high and low level isn't referring to the difficultly, it's referring to how abstracted it is from the hardware processes.
@@ParadymShiftVegan Yes, but one of the goals in abstraction (aside from cross-platform/architecture potential) is to make the language easier to learn or more convenient. The original commenter was laughing at the grouping together of such different languages, I don't think the goal was to teach people that high-level means "easy".
Threading package in python std lib is still GIL bound and just time slices a single thread, you must use multiprocessing to achieve true parallelism in python (unless you have compiled 3.12 using noGIL or have written some native library that uses it outside the GIL)
thank you for blessing simpletons like me to understand this. dont ever stop FireShip.
came after i spent 3 months learning cuda
It really targets a different domain than cuda. It's for things that you specifically don't want to implement in cuda: just normal, general purpose programming.
As the name implies it's a VM and that VM comes with some overhead - so if you implement something once in bare cuda and once in bend the cuda version will be faster. Bend / HVM(2) won't magically speed up your matrix multiplications and stuff like that. But since you already know cuda you'd surely agree that there's a ton of things you would never implement with it simple because it's fucking hard to do so or simply not practical. That's where bend comes in: putting algorithms on the GPU that were traditionally relegated to the CPU due to the limitation of the current implementations of high-level languages.
(There's apparently also some room for super new algorithms / making already known algorithms that have issues in current systems more feasible in practice, for example around symbolic computing)
(At least that's my current understanding - I'm not involved with the project I've just been following it for some time)
@@SVVV97 feels like GPU will replace CPU completely damn
If you can get an OS working like this
You know I was just about to write a comment along the lines of "Man I'm glad I didn't invest my time into studying GPU architecture & learning cuda"
@@NielsGx OS are fundamentally single threaded concepts at a low level, that's why when supercomputers run, the operating system the supercomputer runs doesn't boot once it boots thousands of times, once for each individual SoC node in the cluster, and the init system of Linux is a single process PID 1 that all other processes fork from, and the bootloader of an SoC initializes the processor through a series of mode changes and all of these occur in a single thread.
multithreading and multitasking is a _feature_ that gets enabled and active at a certain point in the SoC boot process but it can't be instantly active it takes a few milliseconds
@@NielsGx I don't, because the system still needs high hz cores, it's a dandem of GPU/CPU where CPU is the main head
I just started to learn coding a few days ago, admittedly i have no idea on what anything said in this video is, but it sounds good for me!
You should start with Go, do not touch this if you're new.
Start with a more simpler typed language like Go, that covers all these principles already and will probably teach you better, as all this would not be happening under the hood and you would have to write it manually.
And then move to this or Rust, if you want to explore more. I mean I love Rust, but don't start programming right away with this or Rust 💀
Go will teach you what rust does for the most part already, in an easier syntax.
Sry for the long para, btw still love rust
i really recommend you to start with C in cs50.
@@daphenomenalz4100 my only focus is game dev. And i assume AI will make me regret time investment soon, so im more so passively just trying to understand coding in a meta sense. for now i've settled on just learning python + unity or godot engine
Don't listen to either of them and just continue learning what you are currently learning. Both C and Rust will burn you out as a beginner unless you are pationate about the languages. You will learn them later, when you can appreciate what they bring to the table.
Nitpick: multithreading in python are still single threaded because of the GIL. You can spawn threads that are waiting on I/O, but number crunching on the CPU with python's GIL won't ever use more than one vcpu. To leverage more than one cpu in Python requires the multitasking library (spawn additional processes, each single-threaded).
Great video ! Just bear in mind that no matter the underlying framworks, not all algos are parallelizable :)
Episode 5476 of "coders" afraid of C++ looking for an easy way out.
C++ isn't *that* bad
Modifying your Python code to take advantage of multiple threads just adds a few gotchas. Nothing too crazy except, oh yeah, IT DOESN'T ACTUALLY RUN ANYTHING IN PARALLEL.
You can run multiple processes though, no?
I can't believe @Fireship overlooked the GIL
Might as well use async library.
exactly my thoughts xD
Weren't they getting rid of the global interpreter lock in a coming version?
I was literally talking about using the gpu for multithreading with a friend 2 days ago at work
Major question. Is it so slow like interpreted python on each core? or is it translated into native instructions for CPU / GPU like in good compiled languages?
The code kinda bends as I understood it 🤔
Finally. Procedural Haskell.
Wait, what? Can you elaborate, please?
2 questions: what’s procedural about this language? Folds and recursions associate with FP.
Haskell is said to be good for parallelism also. What’s different with this language that it’s better than Haskell?
Its not haskell and its not procedural at all
@@evergreen- the language is procedural but has folds and bends and parallelization like haskell. it's better because stuff like parallel arrays is really hard in haskell, but I don't think it claims ot be better than Haskell anyway. different use cases
@@brendanfay2017
:^)
We are looking for a code bender with 10 years experience in code bending.
I liked my own comment cuz I like it that's why I posted it. GigaChad
Data partitioning is the hard thing about parrel programming. If blocks have a morass of dependencies between each other then you get mostly sequential execution
Honestly, as someone who deals with parallelization pretty often (scientific computing), this is kinda huge
"Two completely random numbers". Yeah i believe you :)
Here's another two completely random numbers if you don't like those: 80085 + 7175
Happy?
Holy shit how did i missed this joke
@@zweitekonto9654 same lmao 😂😂
yeah, I believe him as well!
What a time to be alive!
This is actually groundbreaking and awesome.
I think that would be pretty big for hardware health as well!
But now I don't have to write .sType in vulkan 17,000 times. :(
Damn, It's like everyone apparently knows how to write progrmaming languages and I'm here trying to understand how it all works, despite all the resources available telling me, in complicated ways, how it all works...
I'm a simpleton.
skill issue
Take solace in knowing that even with a lot of learning and professional experience, that feeling will never go away. It's par for the course.
read sicp, the concept is really well explained there
if you cant get through sicp... my condolences
I have been coding for 5 years. I still don't feel I fully understand programming.
because you learn by doing
you wont learn shit just reading about it
same way you wont get good at drawing by reading van goghs biography or whatever
just sit down and CODE
interpreter written in rust is crazy
Should have used matrix's spoon bend as thumbnail and add mic hitting the floor sound at the end
We had a tool for that, it was called *compute shaders*
This new language appears to be a higher level of abstraction than compute shaders and seems to be used for the CPU in addition to the GPU.
@@matthewmoulton1 Feel free to correct me if you're a GPU expert, but I don't think abstraction is a particularly good idea in this space.
You're not going to be running complex business logic on the GPU. The logic you _do run_ should be as compact and optimized as possible.
People literally minimize divisions and multiplies in GPU code, let alone memory reads or branches.
If anything we should be making CPU langs look more like hlsl so we can actually make use of SIMD intrinsics without code looking ugly as sin
@@pokefreak2112 You are right- abstraction brings with it a number of inefficiencies. I was trying to point out how this new language is different from compute shaders, not necessarily claim that it would/should *replace* them.
The fact of the matter is that the average programmer likes writing in as high-level of a language as possible. I think it is cool that Bend handles the parallelism internally, which hopefully should lead to higher multithreading and/or GPU utilization across the board. For situations where performance is critical (like rendering or neural net execution), we should probably stick with the lower-level tools available.
I mean, it sounds great and all, but at least right now, I don't see how this is going to be much better than the solutions we already have for running code on the GPU like compute shaders (OpenGL/OpenCL/CUDA) or a library like tensorflow or pytorch, and the multiple solutions for multiple threads and async/await in all the major programming languages, all of which already abstract away everything quite a lot.
Trying to make hardware accelerated computing and general parallelism more accessible is great, but this is the sorta thing where you get into the nitty gritty bits of your algorithm's performance and with that explicit's almost always better than implicit especially with algorithms which require complex human thought and logic to optimize, something much harder to put in a compiler compared to say, the comparatively much more basic optimizations made by the C compiler, and when GPU cores can be easily compared to the CPUs of the yesterdecades: not exactly something you want something running high level code in an application worried about performance to this degree.
So you end up left in a weird limbo where most applications are fine with a single thread, most of the few applications left that aren't delve into async/await and stay in the same language, and then the applications that really care about performance like simulations end up going with the more performant options mentioned previously that are already the industry standard anyways, and for good reason too.
In my opinion, I love the project and its idea. It sounds great and I want it to succeed, but what I think it'll be great for and largely used for will probably be smaller scripts and data science computations and especially as an educational tool, but using it as something more than that, like to replace the industry standard solutions in an actual large project doesn't feel like it'd catch on nearly as much as just using the standard in the current tech landscape and most likely for years to come.
Edit: Yes, this has promise for doing the things industry standard solutions can do and yes it abstracts a lot away which especially helps with things like race conditions (the industry standard solutions I listed also help significantly with that), but what I'm saying is that even then, most of the tasks are ones that are better off done with those industry standard solutions. Also, as for rj7259a's comment I doubt using a GPU for compilation say, for languages like C/C++ is a good idea at least in most cases.
interaction combinators are extremely promising, its attainable in the near term
You write serial code and it runs in parallel. That is the greatest invention in programming since subroutines.
If you are not a average JS developer, you know how complicated it is to write parallel code, you will always need to deal with a deadlock or race condition, and they are non reproducible bugs, the worst kind of bug. (Heisenbug)
There is software that could run on a GPU, but would be pratically impossible to be coded by humans, like a compiler.
@@rj7250ayou don't get that sort of bug in (safe) Rust. Deadlocks can happen, sure, but they're probably the easiest bug to diagnose, they're their own breakpoint.
And what do you mean humans can't program GPUs? We've been doing that for decades? It's a little fiddly to shuffle data in and out, but honestly it's just not that bad.
@@rj7250a Yes, I understand that writing parallel code is difficult, but not only are most applications fine with a single thread, but those that truly need the extra threads are oftentimes better off with the more industry standard solutions I previously listed. They already have the developers with the skills needed to utilize those libraries as well.
Fireship. Immortal content, as always.
using this for shader programming sounds promising
Reminder: protogens go beep boop
Thank you
Very important reminder. Now it's time to make these reminders multithreaded.
Until I run memz.exe
ew a f*rry + who asked
1:21 wtf is that picture is that a devil . Chat is this real ?
Do not fuck around with the daemon, it will haunt your CPU forever looking for processes to keep alive
That's a 100% real photo, his name is Michael and he's a gentle giant.
The way Bend works reminds me of recursive functions in functional programming languages like Haskell
Nvm just got further into the video that's literally all it is lmao
This totally *bends* my mind
perfect name for a new "programming language"
0 days without AI
zero day 😹
Lua programmers are getting insulted
Hrm. This bend thing is a neat concept. I feel like the syntax is too indirect though. I can think of a couple of ways to implement it as simply a generic function that takes a starting value and a function that takes the current value and a function to recurse. It's really cool as a built in language feature though as I'm sure it was much easier to implement.
Saw you starred the repo, expected a video lol
GPU hackers incoming
no different than webgl
So glad to see my experience from a semester of multiprocessor architecture is now obsolete.
Haha 😄
I think you're now expected to write Bendscript or whatever the next amazing parallel programming language is.
Nah, this will never pick up.
Thank you very much for such interesting material, I will follow new posts.😂
But did you compare the same implementation with loops to the one with recursion? Is it faster? And how about the memory consumption from recursion? Complex recursive functions often cause stack overflow.