If LLMs Can Code, Why Are Apps Still Broken?

Поділитися
Вставка
  • Опубліковано 27 гру 2024

КОМЕНТАРІ • 88

  • @LowestofheDead
    @LowestofheDead 6 днів тому +4

    If an LLM is used as an natural-language interface between a developer and structural tools (e.g. refactoring, error-detection, system design) that already threatens a lot of jobs.
    Think of how the Rust compiler can catch whole classes of errors which used to require skilled developers to detect. While it's currently a pain to write and read in Rust, a fine-tuned LLM interface can help with that.
    I think we'll see more advanced versions of these structural tools built by very qualified engineers. And those could be put together like Lego blocks by a single developer paired with an LLM. In a way, those structures could replace the role that languages and IDEs filled in the past.
    I don't think this is better than what we have now, but it will be used by businesses if it's cheaper.
    And no, it's not reliable enough for critical systems or aerospace, but the vast majority of developers don't work in those fields.

  • @MichaelWilliams-lr4mb
    @MichaelWilliams-lr4mb 7 днів тому +7

    Why do people assume AI-written code is going to be better, especially when they're just trained off of humans anyway?

    • @dadlord689
      @dadlord689 6 днів тому

      Exactly)

    • @mandisaw
      @mandisaw 4 дні тому

      Even worse, they're trained off of public repos, blogs, and tutorials - it's why JS frameworks and beginner or cheap-contractor web-apps are overrepresented in the languages it's "good" at. When he said the models are good at SQL, I laughed in anything more complex than "SELECT * FROM table".

  • @j01237
    @j01237 6 днів тому +5

    Humans and AI have something in common...none if perfect 😂...drivers are still not replaced, so just keep living life bro..we need each other forever

  • @techsuvara
    @techsuvara 8 днів тому +8

    I tried to use these tools for something more than just a small site, and it doesn't even get small sites right. The moment I start adding things like Tailwind with Lottie and Rive... Falls over, doesn't even build... Tried all the tools,, ChatGPT, Claude etc... new tools and updated APIs with things like Rive Animation notoriously fails and breaks with these tools. I just reverted to building the whole thing myself, and I actually made progress much faster.

    • @ChangeNode
      @ChangeNode  8 днів тому

      Yeah, for some stuff like this esp w/smaller footprints it's more of a chatbot on the side sort of thing still.

    • @pondeify
      @pondeify 6 днів тому +1

      i think everyone is missing the bigger picture. these tools won't replace coders, they will just devalue the job. i'm seeing it at my work - users are just creating things themselves that they would have previously raised a JIRA for. core system updates are still done by my team but even then the level of knowledge required is going down.
      the end state is we won't have applications, we're just have models so there won't be as much 'coding' as such.

    • @ChangeNode
      @ChangeNode  6 днів тому

      @@pondeify Oh, absolutely - many of the comments boil down to "ai/llms are garbage bs" or "ai/llms will take over everything." I suspect the truth will be more like the combo of ui tools (eg bubble and similar) plus llms will "democratize" software dev kind of like how video game dev has already "democratized" - ie an absolute flood of stuff, incredibly hard to make any money due to the volume, but not a lot of sympathy for structural change because "you just have to grind harder" or whatever.
      Films are already a mess - my understanding is that marketing runs x2-x3 production costs. My guess is that if AI/LLMs/reverse diffusion systems get to where you can make an MCU class film for 20m-30m instead of 100m-200m, the bulk of the budgets will have to go to marketing just to break through.
      If everyone gets fired all at the same time I think we might have structural change, but the more likely is a slow roll over 5-10 years. :(

  • @Optimus6128
    @Optimus6128 8 днів тому +7

    Weird question.
    AI right now is not even near doing the mental work a human programmer does, like having a goal of what problem you want to solve, thinking of an algorithm, running the programming flow in your mind while typing it, noticing precisely things that can go wrong, it's a very precise conscious engineering work.
    AI will probabilistically produce something that is close to what seems to be the solution, but not consciously engineering it step by step. You run it and it doesn't even compile. Or it doesn't do what it's supposed to do. It works well with AI art or literature, because you don't need to be precisely engineering to the smallest detail to pass.
    Because of the hype of "we are livining in the golden age of AI and people talk about coding AI, so why is software not good?". It's like saying "we are living in the age of flight, the Wright brothers just did that and we have also ballons, why don't we already have bases on mars?".
    And in fact, the question "why is software still bad?" even applies to conventional tools that work precisely. A compiler will precisely take your high level language and produce assembly language that has to work and do exactly what you tell it to do. And many times will produce even optimized assembly. And we have modern developer tools, software methodologies and all that kind of jazz. And we have those for years since the 90s. So, the question here is, even with all that stuff and way faster computers, why software still sucks today? Both in performance and errors?
    With all our robust precise tools and "improved" methodologies, we can't go there. So how can we expect the primitive AI to do that, and isn't that the case that because we offloaded all that responsibility to automatic tools, we ended up not caring about good software as the new tech is always a "deus ex" that will magically solve the problems for us? AI is not even there but even what it will be in 10-20 years, I think it will produce crap, even worse than the human. We will just have computers 100-1000 times faster but still software that sucks.

    • @newvocabulary
      @newvocabulary 6 днів тому +1

      The Wright brothers analogy is accurate. We have only just crossed the chasm required to build AI systems, and they are as you said, primitive. Now comes a time of refinement, and implementation, standardization, and deployment. It will take a while before the robots eat us, but they are certainly hungry.

  • @huveja9799
    @huveja9799 8 днів тому +8

    As I said the other day to an arrogant startup founder in San Francisco, "I don't understand, what do you want a foundational engineer for?, hiring one you have to share ownership and pay a salary, but, if LLMs are so good, why don't you pay the top subscription of OpenAI, and that's it? .. one thing is the smoke, and another thing are the actions at the moment of truth ..

    • @ChangeNode
      @ChangeNode  8 днів тому

      Yeah, it's interesting to see these obvious contradictions. I'll be more impressed the day that, say, OpenAI can use ChatGPT/tooling on top to generate something like their native macOS desktop client from prompts only.
      In a lot of ways this whole video is just calling out all that BS but in a nice, constructive criticism sort of way...

    • @heyyrudyy404
      @heyyrudyy404 7 днів тому

      ⁠@@ChangeNodemy point is exactly what you just stated here, instead of Mac OS desktop client you mentioned, I only mentioned an IDE like IntelliJ. But you came up with your bias telling me "I don’t think big enough". English sure it’s not my primary language, but my point was just AI cannot create what it’s not intentionally train for. That’s it.
      But the moment I wrote my comment I suspect you will come up with that bias, and this says a lot about you. Good luck.

    • @errrzarrr
      @errrzarrr 7 днів тому +2

      He wants AI for the credits/bandwagon and a human for the blame.

    • @huveja9799
      @huveja9799 7 днів тому +3

      The other thing that should not be forgotten is that the model (to call it AI is to collaborate with the hiccups, sorry, the hype), is going to generate the patterns that it saw in its training, and that means that the vast majority of those patterns were generated by the average (mediocre)

    • @homomorphic
      @homomorphic 7 днів тому

      Who cares whether you can generate apps from prompts only. Of course you can't and of course that is irrelevant. Gpt is a tool, a very useful tool, that increases programmer productivity. I do some work that is beyond gpts capacity to assist and man, do I wish that were not the case and that it could assist in that work because when I am able to use it to assist (which is probably 80% of the time) it is fricken awesome.

  • @ripsky7586
    @ripsky7586 9 днів тому +7

    Thoughts on embedded systems as a career? I feel like knowing hardware + software is a good bet for the future.

    • @ChangeNode
      @ChangeNode  8 днів тому +2

      I did a few other vids including one on pivot to robotics you might find interesting. WRT embedded it's a smaller pool of devs and a smaller pool of jobs. ¯\_(ツ)_/¯

  • @thecollector6746
    @thecollector6746 7 днів тому +2

    It wouldn't have anything to do with AI being little more than probabilistinc models that can't actually reason nor understand problem sets, but are literally "guessing" what the solution is based on old solutions that may or may not have something to do with said problem, would it ?

  • @HaraldEngels
    @HaraldEngels 7 днів тому +2

    Many apps got quite bad due to the widespread switch to JS frameworks for most processing tasks (also on the server) or for recreating business logic on a client (based on the JSON data coming from a server).

    • @dszmaj
      @dszmaj 6 днів тому +1

      it’s not the fault of js, it can be very fast, but rather inefficient programming and carelessness

    • @mandisaw
      @mandisaw 4 дні тому

      Right answer, but I'd disagree on the reason. JavaScript became the all-in-one hammer to "everything is a nail" because there are so many (cheap) programmers who are only trained in cobbling together JS frameworks to build web-apps. Instead of hiring or training programmers to more suitable tech, which would eventually become expensive, they just try to rework every problem-domain as a matter of "add more JS frameworks".
      You are seeing similar now with Python, which is a decent enough language for what it started as (a more beginner-friendly interpreted wrapper for Fortran). But now you've got students learning Python and companies trying to rewrite everything to accept Python scripting, no matter the impact on performance or requirements mismatch.
      There's also the classic issue of poorly understood problems creating poor software - genAI cannot help with that.

  • @MysticCoder89
    @MysticCoder89 8 днів тому +13

    because it is LANGUAGE models for now.
    They can speak, but they can't architect, can't understand goals, can't engineer

    • @khrishp
      @khrishp 7 днів тому +6

      They fundamentally can't understand anything. AI is a perfect representation of the wisdom of crowds. No individual portion of an AI black box understands what it's being told. It just has so many different data points that can come to the most average representation of what an acceptable answer to what it's been given would be. But it has no understanding of any of the concepts that it's saying out loud.
      The fact that it learned how to sound sentient, far sooner than it actually has any form of understanding, means that things like the Turing test no longer work to represent actual understanding.

    • @pagetvido1850
      @pagetvido1850 6 днів тому +1

      However, they're currently building a dataset from people attempting to code with this that might give it the necessary data to replicate programmer reasoning. If a lemon can run an llm, I'm not sure why they're floating the excuse that compute power is the limiting factor. It's more likely the quality data that's holding them back from 'AGI'. Hence they jam AI into everything, so they can get the needed logic and learning data from people.

    • @MysticCoder89
      @MysticCoder89 6 днів тому

      @pagetvido1850 good luck;) most programmers live without reasoning)
      Most people doesn't have a clue how to live properly, if you think of it.
      Not a good source for learning

    • @pagetvido1850
      @pagetvido1850 6 днів тому

      @@MysticCoder89 Imo it'll be if/when they become tutors in schools that they'll get a big boost. They'll just have to filter for switched on students, then they'll get some good data for that vector moving around its hyperspace. I use local only as I'd rather not train my replacement.

  • @Kerojey
    @Kerojey 8 днів тому +3

    Software become bad in web before AI. Popularization of React, electron, etc. definitely did damage to quality of software, AI is just nail to the coffin.

    • @ChangeNode
      @ChangeNode  8 днів тому +2

      FWIW I have gotten much better results writing x-plat good quality stuff w/SvelteKit & Capacitor/Tauri than say the JDK or .NET over the last year. async/await, write native stuff in C or Rust as needed FTW. React is a foot gun.
      I've gotten decent perf out of pretty much everything from Unity to whatever, but ya have to profile regardless. I think that part of the problem is junior folks ("let's grab the front end JS guy and have him write backend") is the real issue, which in turn is non-technical management making stack decisions.

  • @TreeLuvBurdpu
    @TreeLuvBurdpu 8 днів тому +2

    The trend in programming over the last decade is a trend towards rules, policies, guidelines and TESTS. This is still true. Look how much mileage Rust is getting from making the compiler do the work and apply policies. I just updated my ESLintrc, with the help of free Gemini, and now the Cline checks ESLint after every edit.

    • @ChangeNode
      @ChangeNode  8 днів тому +2

      My two cents - you are, way, way, way ahead of most devs & esp most corporate dev. Hope you are documenting/putting out your own vids on setting this stuff up... :)

    • @TreeLuvBurdpu
      @TreeLuvBurdpu 8 днів тому

      @ChangeNode funnily enough, I'm heavily documenting, but I've only posted a couple vids. A few people have told me what you just said, so I'm going to ramp up my video deployment pipeline very shortly.

    • @ChangeNode
      @ChangeNode  8 днів тому +1

      @@TreeLuvBurdpu good luck! My two cents FWIW, Camtasia works pretty well for that kind of tech material, or just get a good screen recorder and use DaVinci Resolve. It's about lights, not the camera. Absolutely most important - get a decent mic. :)

    • @TreeLuvBurdpu
      @TreeLuvBurdpu 8 днів тому

      @ChangeNode I've just been using ClimpChamp and my stupid Audio Technica ATR2100-USB broke. Going to check out what you mentioned

    • @homomorphic
      @homomorphic 7 днів тому

      Rust is useless for me as I need to create UB in my code all the time and branching into UB under control of the developer is not a capability that rust possesses.
      It is funny how people have such tunnel vision and think that what is bad for their software is bad for everyone's software. Being able to create UB on demand is a critical capability in security (both offensive and defensive).

  • @malcomgreen4747
    @malcomgreen4747 8 днів тому +5

    I have an app that took me 4 years to build and architect the database and the backend and the apps, i believe AI will not be able to understand the entire architect. it's huge just the backend controllers are so many, not talking about database tables, and hundreds of pages in moblie apps, Claude ai struggled with just one controller and it hallucinate, but its very helpful in creating chunks of codes, it can refacor an entire controller easily, but i have to revise the code, the ai is very helpful solving some problems that takes me hours like calculations, and it also writes better code than mine 😅 but it does apply the separation of concern too much

    • @ChangeNode
      @ChangeNode  8 днів тому +1

      Do you use anything to help visual/understand the app? eg modeling tools for the schema, refactoring tools, structural analysis, etc? That's part of what I was touching on in the vid, that at some point the AI/LLM tooling should be able to talk to that kind of code comprehension tooling. Going to be interesting to see how that evolves...

  • @RichardBenoit-p7k
    @RichardBenoit-p7k 8 днів тому +2

    Engineering, Programming, and Coding all have slightly different processes. Now this might sound simple but there is a LOT of nuance. When you ask AI to design something from the software engineering you are not asking for code or the logic (programming) nor the code (whatever language). Instead the broader aspects of what are the needs etc... Think of it like building a business plan. The aspects of the business plan needs logic but the code needs to support the overall plan and logic.
    When you ask AI you will get code spit out right away and the differences are mostly ignored, UNLESS you ask VERY direct questions.
    Programming is useless if you do not have an understanding of engineering because such is the systems, resources, etc... Programming (logic) is suppose to be a means to offering a solution but the solution needs to be comparable or do something useful and expressed in a language (code).
    AI is GREAT for learning but creating something useful you NEED to be solving problems. What are problems that can be worked on to create a general plan of action, create the pseudo code (word logic), bring such into a more developed programming framework and find a compatible language.
    AI will simply keep re writing everything and also has a limit to what it can do. Perhaps if it treated the programming like how business plans work and does a feasibility study internally and ask for tuns of parameters then build something requests could perhaps be useful.
    Later,

    • @ChangeNode
      @ChangeNode  8 днів тому +4

      It's a very weird process trying to "force" myself to use AIs more and more. eg I just used ChatGPT's native mobile client to take pics to help me solve a wiring issue for a dead dimmer, and also swap out the cartridge for a leaky shower head. It actually did a pretty good job at helping me troubleshoot both. UA-cam videos helped me get oriented, and then the feedback on the pics helped me trouble shoot a few specific things.
      Same with dev, beats the heck out of Google + StackOverflow, but nowhere near the hype curve.

  • @xobk
    @xobk 8 днів тому +3

    Always enjoy your perspective, thanks.

  • @JohnKerbaugh
    @JohnKerbaugh 8 днів тому +1

    My LG TV Amazon Prime TV app, requires that I unplug and plug in the TV every couple of days so that it can be used.

    • @ChangeNode
      @ChangeNode  8 днів тому +1

      >_

    • @newvocabulary
      @newvocabulary 6 днів тому

      My toaster coils shorted out therefore no airliners can take off.

    • @ChangeNode
      @ChangeNode  6 днів тому

      @@newvocabulary ua-cam.com/video/vLm6oTCFcxQ/v-deo.html

  • @m12652
    @m12652 7 днів тому +2

    People forget.... being "intelligent" and producing good results in an intelligent way is something that eludes 95% of humanity at least. Why would an artificial intelligence trained on us even be a step forward, let alone accurate or even an improvement. You get out what you put in so given most AI contain the effective sum of all human knowledge, its hardly a surprise they produce such 💩 answers.

  • @jzero1579
    @jzero1579 7 днів тому +1

    What about 10 years from now

    • @ChangeNode
      @ChangeNode  7 днів тому +1

      That's the trick, ain't it? I mean, try working out how you might explain the 40s to someone in 1939, etc etc etc.
      I'm thinking my next few videos are going to be book reviews of some of the books that explain the world... maybe also kick up my filming game a bit, get out of my office ftw... :)

  • @Relaxingpeacefulcalming
    @Relaxingpeacefulcalming 6 днів тому

    Google promotes only the websites that can pay and they surely don't have the best coders automatically.

  • @deletedaxiom6057
    @deletedaxiom6057 7 днів тому

    My analogy is programmers move from being a violinist to being a conductor.

  • @rexxpowercolt
    @rexxpowercolt 2 дні тому +1

    Most based UA-cam title award

  • @brightokoro7073
    @brightokoro7073 8 днів тому +2

    Nice points but what people are not exploring how to use AI effectively as software engineer or a developer with best practice it powerful,since you do the thinking as a human and pass specific instruction of what you want exactly meaning you must know your basic as a programmer❤❤❤❤❤❤

    • @ChangeNode
      @ChangeNode  8 днів тому +1

      Totally. I keep coming back to “you don’t know what you don’t know” as a big limiter.
      💯❤️

  • @ToastyCubes
    @ToastyCubes 7 днів тому +2

    The title of this video goes so hard

    • @ChangeNode
      @ChangeNode  7 днів тому +1

      The original title idea I had was "If AI/LLMs are so great, why does so much software suck?" but didn't want to annoy YT etc lol

  • @vitalyl1327
    @vitalyl1327 7 днів тому +1

    The modern software is so bad because of self-taughts, bootcamp "graduates" and the world being infested by WWW. Developing apps in Electron, really? No matter how smart the developer is, the result will be an utter trash.

  • @kamertonaudiophileplayer847
    @kamertonaudiophileplayer847 6 днів тому

    Real software companies didn't adopt AI yet. Give couple years, and the situation will be improved. I'm writing an article about that, so I will share with you when it's ready.

  • @Andrew-rc3vh
    @Andrew-rc3vh 6 днів тому

    Some guy bought a new car for a dollar using an AI chatbot which created the contract on behalf of the company.

  • @ViralKiller
    @ViralKiller 7 днів тому

    Exactly well said... because it's shite

  • @ahooenergy
    @ahooenergy 8 днів тому +1

    Maybe they don't need to be!

  • @rwlurk
    @rwlurk 8 днів тому

    fundamentally LLMs are GIGO

  • @justinbaker84
    @justinbaker84 8 днів тому +2

    The most interesting part of this video is that it sounds like you don't see any technical limitation to LLMs being able to do the vast majority of coding as long as a talented prompter is leading them in the right direction.

    • @ChangeNode
      @ChangeNode  8 днів тому +2

      I could do a +/- for the theoretical limits. Limits incoming: lack of data, something fundamental in LLM design, processing limits (some kind of geometric increase needed), energy limits, etc. Lack of limits: tons of low hanging optimizations still in progress, dedicated chip designs, new ways of collecting data (eg user observation), etc.
      Short answer is could go either way and so smart move is the (admittedly exhausting) have to consider both valid. ¯\_(ツ)_/¯

    • @justinbaker84
      @justinbaker84 8 днів тому +2

      @@ChangeNode Great analysis!

    • @shawn_bullock
      @shawn_bullock 4 дні тому

      I've been doing some wild things zero/one shotting over 1000's of prompts lately and getting exactly what I'm asking for. The following prompt just adds a detail. From making my own programming language -- even one that I can add new syntax to as its running, to a NES game, to a database migration tool, to things I do at work (a real time analytics dashboard), a gsheets like spreadsheet, and I'm not really struggling with any aspect of it other than the coding style it gives me isn't anything I resonate with. I have however figured out how to get it to generate code that exactly matches our project guidelines and coding style.
      There are really serious limits but there are ways to game it and get what you're asking.

  • @doesthingswithcomputers
    @doesthingswithcomputers 6 днів тому

    Software was bad before ai…

  • @gtrguy17
    @gtrguy17 6 днів тому

    Touche’

  • @mojolotz
    @mojolotz 7 днів тому

    AI is at best as good as the average dev.
    Think about that

  • @noway8233
    @noway8233 8 днів тому +1

    Good luck with that😅

  • @heyyrudyy404
    @heyyrudyy404 8 днів тому

    If AI 🤖 is this good, why can’t I build from scratch my own full clone of IntelliJ IDEA IDE for free ? 😂😂😂

    • @ChangeNode
      @ChangeNode  8 днів тому +2

      You don't think big enough - did you see the prompt I gave it at 1:17
      ua-cam.com/video/yoz9C2a30rM/v-deo.html
      Might need to watch it on a desktop / 4k to see it lol

    • @heyyrudyy404
      @heyyrudyy404 7 днів тому

      @@ChangeNode my point is simple :
      AI is conceptually a dead end.
      AI only works when the actual work is done before and publicly available to be fed to the model. But if AI is fed by the code people copy/paste from AI tool, what will be the value of AI then ? None.
      Sure you can bootstrap things with AI, but it’s only because AI databases was fed up and warm up in advance.
      To finish, in domain of pure functional programming languages where common programming patterns keep appearing, do you really need AI to bootstrap anything for you juste because you are lazy ? No. I’ve programming for a while now, and then come the promise of AI that anything can be built by AI (it’s not me who bring the hype but the Actual AI companies).
      With less lazy and more smart people, AI hype will wane.

  • @uncletimo6059
    @uncletimo6059 7 днів тому

    boomer gramps fancies himself an expert in tech and AI

    • @strigoiu13
      @strigoiu13 7 днів тому

      so, why is genZ is still bad at everything, though?

    • @uncletimo6059
      @uncletimo6059 7 днів тому

      @@strigoiu13 genZ does not want to become slaves to corpos for laughable money.
      also, learn to english.

    • @ChangeNode
      @ChangeNode  День тому

      lol
      I’m Gen X, not a boomer
      I’ve been in tech for decades, written several books, worked for big tech from Apple to Symantec, ran a consulting business which I sold, etc etc etc…
      But you do you 😂