This one was close to my heart - if you enjoyed it, please consider sharing it with a colleague! I also am very close to finally releasing the beta of my git cheatsheet, subscribe here go get it: philomatics.com/git-cheatsheet
I've been really enjoying the more "out there" content that you have been recently posting to your channel! I haven't seen anything like this anywhere else (even outside of UA-cam), so to me it's been a blast. Keep at it :)
recursive feedback easily overfloods any datacenter, so ai could be quite good only for semi-automatics + we must keep in mind malware which can be too empowered w/ ai intrinsic flaws :)
I played with Google's ai while writing some code that uses opelssl... NONE of its recommendations worked but for almost impossible reasons to discover if you're not already super familiar with openssl or have the time and skills to build n debug INSIDE openssl libraries and not just link to them (i didn't do this of course but if i wasn't already familiar with the project i probably would have had to). I found it quicker to look at long depricated stack overflow answers as well as the flawed openssl documentation, and finally ... just read the darn source code (their documentation is so wrong and if their example code works... it's probably been depricated for a few years). But that's the problem with ai... it's only as good as the errors in the data used to train it... some knowledgeable competent human must write down the technical information before an ai can lean it... ai can't tell who's the expert if two statements contradict each other. I could glitch the ai into generating contradictory recommendations just by rephrasing a query. Ai may be useful somewhere in programming but I haven't seen a convincing example yet.
As a uni student, its refreshing to hear someone say the exact things im thinking in regards to AI. I've been thinking about all of this abstraction and came to the conclusion that the best way to work around this problem might be to get AI to create its own system of "coding". We have defined many parameters to create things digitally, can an AI one day take those parameters and reimagine a way to use computers/electricity/hardware that we perhaps havent thought of? After all, we are expecting AI to make scientific break throughs - to what extent can the concept of computers be advanced and changed with their help? Coding at its core is just a way for a human to tell a machine how to create something based on the physical attributes of the system. We taught the machine how to communicate with us to achieve a goal with coding, i think AI will reverse this relationship. The AI will tell humans how it should be created physically to accomodate the AI "coding language(what is the most efficient way to create or show all of the things that encompass computing/coding/art/video/games.. etc.)", and because we communication with language - thats how the AI will communicate with us. If we get AI researchers i thing we start down a path of hyper optimising the virtual world, id say nothing is off limits.
I've been passionate about this topic for decades, and it's a breath of fresh air to see it discussed with so much sincerity and intelligence. For what it's worth, to my mind there is no doubt that the core issue is indeed managing complexity. I also agree with your point about the 'abstraction tower' and needing to be able to navigate between different levels of abstraction and ideally being bidirectional - well done - brilliant video
The "software crisis" is manufactured. Software isn't buggy because of developers or development practices, software is buggy because software companies value stock market tricks more than they value their own product. When you are hiring and firing developers every 2 years, you're not going to have good code. The average dev turnover rate needs to be 5 years or longer. You need devs who stick around for as long as 10 years for large products. Without this, the process of churning through devs will ruin the quality of your code base.
Incorrect. You can use hundreds of software quality assurance tools to guide LLMs to writing high quality code. If you follow very strict standards, you can have high quality code that is extremely easy to maintain.
The practice of ”write as much code as fast as possible" often leads to more than a few problems. What happened to learning how to do it right the first time even if it is a bit slower?
@@caLLLendar No you cannot, LLM can help you learn concepts, it can't effectively design or architect software. No amount of prompting will get it to do it correctly either, even if it excels at algorithms.
Microsoft paint: A few megabytes of hd space and ram, good basic features, startup time < 0.5 seconds. Microsoft whiteboard: a hundreds of MB of hd space and a gig of ram (it is just a web browser and web app) -- startup time > 8 seconds. Very few features. This is the issue we have. Web apps are bad. Period. New microsoft outlook? A complete joke
@@caLLLendar Yeah, right. Standards and quality. You, my friend, have no real-world exposure. :) I've seen companies invest hundreds of thousands of dollars in something that no one needs and be unwilling or unable to fix critical bugs on their front pages. Standards and quality demands just lead to more bureaucracy and internal politics in large corporations.
There's a huge difference between a compiler and an AI code generator: the former is deterministic. Same input => same output. With AI, this guarantee is out of the window. Let's get ready for your code compiling differently on each run. Have fun!
Yeah totally. It's kind of like a compilation step where we only ever save the compiled code. I actually find AI generated code interesting when doing code review because the thought process that generated the code happened in the natural language space, not the programming language level. The code becomes harder to describe just like compiled output of a traditional compiler is harder to describe than the high level code used to produce it.
This isn't true, AI is deterministic. AI tools may not seem to be because they are programmed to use random seeds on every input, but that's not a required part of their design.
@@mstefanwalker3654 Yea facts. Litterally nobody understands shit. But it works so who tf cares. Its even worse when you consider all the syntax slop modern languages like rust like to have like let x = object.map().hash().convert().destroy(param1, param2).memoize() Litterally just greenlights the AI into coding all cryptic.
@@mstefanwalker3654, actually there is no thought process and neither does it happen in natural language space. There is a tendency to anthropomorphize gigantic matrices with multipliers. I would compare an AI to an almost infinitely complicated pinball machine. The output is made of probabilistic resemblance of thought and language. Improving an AI is improving probabilities, never thought or language directly.
The "software crisis" is sales problem, not engineering. It's about how to arrange market forces so higher quality software is valued over quickly produced one.
They haven’t hired senior Web developers anymore, I had to keep myself afloat with freelancing for the past 3 years. Full stack Web developer since 2008.
@@albertoarmando6711 That original meaning is not really the issue today. The crisis with a almost evrything, not just software, is surface vs volume difference. You can only see surface details before you buy something while volume details are what you really want. Sales got too good with getting away with Potemkin villages and empty calories.
I have been programming since the late 1950s. Regarding reading the output of compilers: there were many times when I was hired to fix bugs or add new features and the only thing the company had was the executable object code. After examining a few bytes of object code (I learned to read and write object code directly without tools like a debugger for any of the commonly used processors) I could tell exactly which programming programming language had been used for the original source code and if there were multiple compilers available for that language and processor combination I could tell which compiler. In some cases I could even figure out which programmer had written which part of a compiler. Writing in assembly language (or even writing in object code and hand entering it with front panel switches) was mostly about speeding up the running of the code or squeezing the size of code (so many times where the program was useless unless it would run in 128 or 256 bytes because of the cost of hardware part counts). That is bytes, not KB or MB or GB, We even went to extremes of writing code specific to certain configurations of hardware and timing idiosyncrasies to get it to run faster, even coordinating with the hardware engineers to swap out capacitors and resistors and other parts to change the timing. We even went to extremes of writing self modifying code so that the code could do many different things at different phases of the program life to squeeze it into a limited amount of memory. One example would be a jump table (a fast way to implement the case structure or lookup table that was reused for different purposes during the life of the program. The memory addresses for the start of functions (jump table) or the data in memory (lookup tables) would be changed on the fly to reuse the same memory space for different things at different times. Another example would be modifying the behavior of a function because multiple different functions had some of the same boiler plate and we could change the actual working code portion to save memory. There were so many techniques for using self-modifying code, including what would now be called metaprogramming or reflection.
Congratulations on your long experience. I thought my 45 years was a long time, and I'm still coding. I remember looking at disassembled code in the 80s (In Circuit Emulators) and knowing exactly which source code produced it, and that was a huge project, 35 engineers. I remember thinking it might be the first sign of madness 😅
"A9 03" -- ah yes, 6502 code. I still remember watching that first Terminator movie and I was like "wait, the Terminator runs on a 6502-compatibe CPU, like the Apple 2 or the C64?" Born in 1968, I can't compete with your level of expedience though. Still good enough to hear "we won't hire you, you're overqualified" far too often.
Yeah, AI making programs that waste ressources but are cheaper to make than manual programming seems reasonable to assume. We are in a time where every app with 3 text fields and a button is a webapp that takes 400MB of RAM and 5% GPU for stuff that technically could have been a 4kb program by using the native api.
@@ScorgRus Yeah it's the Java syndrome, let's code in a shit environment using bad syntax and performance, arguably, just to have it "run anywhere", just in case we need it (like we don't know where we want it run in the first place)
The issue is not the loss of control, which can be mitigated with asm and ffi. The issue is the compounding complexity with each layer. Add the loss of knowledge and remove the pressure to ship good software and you go full speed into a wall. AI will ship code that is way worse, checked by dev that never programmed more than few toys exemples. In 5 years the dommage will be astronomical
No; Learn about companies like Snyk and SonarQube. I am developing a platform that will combine hundreds of quality assurance tools that will be run to check the code generated by the LLM. The tools will micromanage the LLM and the result will be much higher quality code than what humans currently write. It will also be completed in a shorter amount of time. The most tedious SDLCAs can be automated.
Programming through communicating with an AI is like programming by drawing flowcharts. It seems like a great idea on the surface, but in reality it is only limited to very narrow domains which can be efficiently described that way (say, plugging together premade shader components in a game engine). For most other domains, flow charts are just awkward to use and a bad idea. That is also why domain-specific programming languages are invented and replace more general ones (let's say, SQL as the universal query language regardless of db implementation). But what all these tools have in common is that they balance power of expression with precision and performance required for a particular domain. Programming by telling an AI to fix something 100 times iteratively, is kinda like doing it with a developer and claiming that the 100 emails you sent them are the "source code" or "documentation". Whereas in reality they are just a messy history of your attempts at formulating what you want, quite inferior to an actual source code which captures the eventually resulting ideas in every imaginable way.
Taken from an even higher level of abstraction, software engineering is the task of converting business needs and business domain models into software models, and then into software implementation. Each transition step or abstraction in that process is leaky. By definition, the most possibly efficient thing is for the AI to go directly from business problem to solution, and for the AI to figure out all the details through generalization. Essentially, full automation. Everything else is necessarily inefficient
@@xt-89907 Not really. Business needs (and any other needs) do evolve over time, sometimes without much awareness from those who expressed the needs originally (they may even no longer be in business). It is not sufficient to translate them into a "solution" once. That solution has to be continuously adjusted to new needs, and the adjustments also have to be explained and justified to those who have those needs. Furthermore, long-term trust relationships and checks and balances in power must be established between the needy and their service providers (professional software engineers) to support that process of need fulfillment. This is possible with humans because we are, more or less, aware how humans operate and which goals they pursue. It would be hardly possible with a black box which does "something" which at first glance seems to satisfy your needs. Unless you want AI to literally become human and replace all human relationships, we are never going to achieve what you envision that way.
I think they are very useful for giving you syntax you don't remember fast, like how do I do check if a substring is in another string JavaScript? It's string.includes(substring), but I rarely remember that syntax, and it's much faster to ask an AI (specially one that answers fast) than to Google it, manually filter a good result, check the answer, not get distracted by the other crap around like comments about little niche things you don't care about (like it happens a lot in Stack Overflow)... But if you ask anything more involved, it risks send you on a loop of useless crap it can't get you out of
@@lowlink534 Yes, they are very useful as glorified canned search engines, trained to imitate and filter results from actual search engines. Ultimately, the usefulness of a ML algorithm boils down to two factors: the training data and the loss (or fit) function to be optimized, given the user input and the training data. Both are of course predefined by designers of the ML models. And in many cases it is as hard or more difficult to determine the scoring for an output as to directly spell out the algorithm which transforms the input into the output. ML thrives and astonishes people in cases where the scoring is easier and can be accomplished.
Dude. Seriously do you even pair program with claude. I’ve seen way more junk code from scratch by humans than ai. Ive been at it for 20 years you cannot even fathom the amount of beginner code ive had to wade. No one talks about the tide being raised. It’s all ‘im so much better than ai’ smh. Get real, no you’re not.
I would argue that anything these LLMs generate is a hallucination. Not just when something is clearly factually false. Correctness can be more nuanced than just black and white. Which is exactly why it will be so hard to replace good engineers.
I looked up the word. An AI can never, by definition, have a hallucination, any more than an Excel spreadsheet or a pinball machine can have a hallucination. The user has a hallucination if he thinks (or when he thinks) that the AI can think or experience things.
@@nielskersic328 This is the silly semantical argument. For AI use, Hallucination has a very specific meaning. It's like Theory in Science... You are essentially using "But it's just a theory!" level of argumentation here. You want to be one of those people?
@@seriouscat2231 This is the silly semantical argument. For AI use, Hallucination has a very specific meaning. It's like Theory in Science... You are essentially using "But it's just a theory!" level of argumentation here. You want to be one of those people?
Exactly. The entire point of abstraction is getting rid of the 'details' you don't 'need' to simplify things. Which is great, until you need them, which is exactly why all abstractions 'leak'.
@@mwwhited The opening chapter of Gareth Morgan's seminal business book Images of Organization says that the metaphors for business structure described therein, as abstractions of complex systems inherently obscure the true nature of what thet describe. Business majors are famously clowned on for being crayon eaters, and they *still* know better than you on this subject. Thats just embarrassing. Im sorry for you
@9:30 I mean, C supports inline assembly, which I think is the expected way to manually patch the compiler output which automatically pushes forward those changes whenever you change the code elsewhere.
Yup stuff like that is really cool, it's like "portals" into the lower layer abstraction. But I think we need something even more fundamental than that (still researching what this could look like).
@@philomatics The problem I see with being able to "move up and down freely" like this is that you're extremely limited in your abstractions if you require that they be able to represent literally everything. I think rather than trying to accomplish something like being able to just edit the assembly and have that reflected back in normal code, it may be more realistic to have something like the ability to tell your IDE/compiler to replace this function with its compiled version which you can then modify. Less reversible compilation and more partial compilation. Same with stuff like being able to easily expand macros/generics in Rust.
A lot of the C that people write does not get turned into the assembly they might be intending as well. For example, ++i and i++ end up getting turned into the same assembly code after most modern compilers hit them because of compiler optimizations.
@@redoktopus3047 Why would people be "intending" a specific order of the assembly instructions? Though there are far more egregious cases, e.g., the compiler entirely removing loops by solving recurrences and putting in a closed form instead.
13:31 - Double agree. One of the saddest moments I think all of us devs go through is when you find the perfect function that solves a very difficult and niche problem your app has, but then realize that, like pulling on the threads of a tightly bundled ball of yarn, you would have to understand the entire monolithic host library to surgically extract all the highly interdependent supporting code the function requires.
This is why software should be written to be reusable from the beginning. Extracting code written to be application specific after the fact rarely works out. But code can be written with reusability in mind which makes reuse very easy.
The business side will talk about quality and innovation but then demand the code be written in the shortest possible time - often too little time. Most managers don’t like hypotheticals, so they won’t think things through, and if you question their ever changing requirements they’ll just say you’re “getting into the weeds” or “not being a team player”. IMHO the problem isn’t the technical people - it’s the yes-man managers and clueless executives running wild with requirements and deadlines.
While I like thinking that abstractions many times reduce the level of control. There are some things that we need to clarify. The whole notion of abstraction is to reduce the level of complexity of a program, not to increase it. When you need to open the black box due to bugs, it means either the absraction layer is not written correctly, or you did not take your time to understand it fully. Math uses abstractions in much of the same way that software does - The fundumental idea of abstraction is not flawed, but in fact what drives us forward. Without the abstractions software development would be too slow or even impossible. An example would be that current day C compilers create much more optimized code than any human could ever write. Those compilers were written also in C - produced by compilers which were originally written in assembly. So in this case not only the abstraction did not reduce control, it allowed actual better performance. Much of modern software development is about prototyping ideas fast, and optimizing when needed. These higher level abstractions allow fast prototyping, and you always have the option of rewriting a more custom optimized solution once you know which solutions are worth optimizing. The problem arrises that inexperienced developers can start working fast and have some results but they might hit a wall that they cannot deal with because they are inexperienced in dealing with problems that might take a long time - such as messing with a foreign code base. - ultimately I think everyone should choose which solutions should be custom, and which ones should be imported from a library. and a developer should know how to gain the knoweldge he lacks in domains he lacks experience in, once the need for that arrises. While i aggree that you should have some understanding in writing lower level code, it is only in the cases you are absolutely certain you know what you want to make. Using modern UI libraries, or just the newest version of CSS available, allows you to explore a much higher space of possible designs and UX functionality than you would normally would be able to if you had to implement everything by yourself. Opinionated abstractions such as Css overall reduce the mental tax on your mind rather than increase it, I do not think it is a fundumental issue with programming. While a two way interaction with the abstraction is a nice idea, it might not solve the modern issues - at least you did not present a compelling case. In fact - I think a perfect solution that would allow a two way paradigm which will bypass the issues we have with functions and function definition, such a paradigm will require its own abstraction that would reduce the amount of control you have over language/algorithems themselves - hence you reduce the amout of control you have over how you define the language in the first place - it may be worth it but it brings us full circle - creating an abstraction over the abstraction methodology just to solve the "fundumental issue" presented, only to prove that abstractions are sometimes necessary and disproving in the same time that abstractions which reduce control are bad.
All non trivial abstractions are leaky - complete reversibility either violates this or requires the abstraction to not be more abstract than the target. Ie. The problem with reversibility as the constraint is that it means that both the high level and low level languages need to be able to describe the same things, and that necessarily means a horrifically complicated syntax and vocabulary for the high level language, since it must address many low level assembly dialects, which in turn vary according to their hardware. It's pushing against what appear to be fundamental rules of information theory and thermodynamics, unless you relax the constraint and do things like introduce intermediate languages (ie even more abstraction). Edit: To add, it's very common to encounter this exact problem in decompiled binaries. Not all machine instructions have a direct high level equivalent. The selection of specific machine instructions is made based on not only on what the high level abstraction described, but also on the target hardware, desired optimisations and other external constraints placed on the compiler. As a result, many decompiled functions end up with decompiler-specific language additions and inline assembly that are necessary to describe what the function does, since the meaning is ambiguous or untranslateable in the high level language. The thing is, this isn't a bug or an implementation limitation, it's a fundamental feature of universal programming languages and specific hardware targets.
What if you did introduce intermediate languages, as well as corresponding decompilers? So between any neighboring languages, you still have leaky abstractions, but the leakiness is kept to a minimum. You could make changes to lower-level code, and have the decompilers back-propagate the changes to the higher levels, except you could correct the results at any level.
You got it right here. There is one more problem with the reversibility, that lower levels have higher entropy. High level languages have guardrails that don't exist on the low level. Not every change you make on the lower level is going to find an equivalent on the higher level. In a more tame scenario, a small change on the lower level, can generate a spaghetti explosion on the higher level.
having inlined code in object binaries is the most general counterexample to any faithful decompilation - it's done for very good performance reasons and renders the output opaque to "cutting" it into the original functions/procedures, especially as the number of compilation stages or intermediary representations grows - so equivalently editing the source and object code in both directions is in itself a worthless goal (as too many layers of abstraction exist between source and target for practically all relevant cases OR because it would infringe someone's rights on the source and binary forms, i.e. "one does not optimize shader binaries or proprietary compilers/libraries/applications by hand for redistribution") it would also be useless in practice as the expressive power differs tremendously between almost all source languages and most flavors of machine code or ILs; there are very few situations where hand-picked optimizations would even matter, and even those do not leave a dent in software running expenses outside of e.g. datacenters or equivalent hardware -- and nontrivial optimizations, if strong enough, as a rule are opaque to decompilation (a practical form of this point is that hashing functions are "one-way"), and even worse is that runtimes (execution environment + machine state + thing to execute + any data input or output handled by the full system) are not exchangeable: the CPUs, the GPUs, the (say) JVM, the browser, the network, and the UI are never the same thing, and programming for each of them is severely constrained by what these environments allow or prohibit one to do (e.g. one would rather not use assembly to parse and transform text, or use JavaScript to change ring 0 state on the processor) having a transparent build chain (any language + some compiler + platform-specific machine code) would require compiler and language and hardware design set back by more than half a century (as even hardware execution units have nasty issues with things that one takes for granted, e.g. synchronization, availability, locality, coherence etc. - and these are not (yet) available as robust or usable primitive notions in any higher-level language) and the issues do not stay put within the "mere" realm of compilers (or languages) -- at runtime the execution itself has very few constraints that are actually properly known for any running program (including the whole operating system, firmware, and active hardware); barring the not-so-decreasing frequency of pure hardware faults and network/wiring failures which application software is never privy to, the software is assumed - not guaranteed - to work as intended practically all of the time (e.g. when did one last check how much available memory is available to an application? what if an allocation fails due to resource contention or takes too much time?) just like the intention behind programming (e.g. I want to draw a circle on the screen) differs from the source code (I want X to happen to Y, i.e. "Y.X(parameters)", e.g. setting some pixels to some color -- but what's a color, what are pixels, and how can those be accessed?) the latter differs from the result (object code and executed machine code -- a level at which all is made out of integers/bits, and their interpretation is left fully to the hardware), and neither is precise enough to represent the reality during execution (as there are too many constraints to be aware of to guarantee complete correctness, too many possible states of a system to expect total reliability, and too many levels of abstraction from one end to the other to concoct a "slider" that continuously varies the "level of abstraction I'm looking at to make changes")
@@mihailamarie4318 But like with huffman coding, you can redistribute your entropic cost - a high level language could simply include more general, less abstract, lower level concepts by "isolating" them (like inline asm in C). This obviously isn't a perfect solution, but I find it likely that some method of combining different levels of abstraction would be significantly superior to both asm and current high level language paradigms.
The compiler is quite deterministic. It gets better and better with upgrades, and people are maintaining compilers to make it more efficient over time. LLM code has some randomness due to temperature. But if we remove the randomness, it still does not resolve the randomness nature of LLM because we don't have 100% control over how it generates the code.
You DO have 100% control. Simply run many static software quality assurance tools on the code the LLM generates. Use the output of the tools as a prompt for the LLM to fix the flaws in the code. See how easy it is to combine deterministic tools with a non-deterministic LLM?
@@nousquest Think outside the LLM-box you're in. Instead, think about how you would create a system that automatically manages the LLM. Roughly: . prompt the LLM to guess . immediately run thousands of tests to check the code . use the warnings from the tests as prompts to improve the code . ignore the bad code, loop and let the LLM try again . save the good code by pushing it to a repository If you like, I can show you a demo of this working. Otherwise, you can do a little bit of searching and see that there are plenty of companies and open source projects succeeding.
C is another example that allows to go down the abstraction. You can write assembly code directly in C, so, if you have something to optimize, you can do it
yes, but the point here is to allow and enable going back from assembly to equivalent C which is not immediately trivial, but I can think of special cases where it's possible
I thought I was crazy, but apparently there are other people out there too that want programming systems where you can see multiple levels of abstraction at once. Good sign since I'm working on building something like that.
I actually had an idea like this: how can one view a codebase or part of the codebase on a macro (architect) and micro (dev) scale depending on the needs. I hope your idea comes to fruition.
I'm working on a system to simply remove as much abstraction as possible within a program. Think: Without the abstraction, you don't need to "see multiple levels of abstraction at once". There will likely be a paradigm shift in programming; Abstraction may be discouraged. The reason is because there are a LOT of benefits to reducing abstraction in the code.
@caLLLendar for-loops are a bit too abstract for my tastes, so now I do: i := 0; if i = list.Count then Exit; list[i].Foo; I++; if i = list.Count then Exit; list[i].Foo; i++; etc. And of course, I try to avoid adding classes or functions, everything in Main(). as God intended.
This sounds fascinating! I'm very curious what this entails. Are you comfortable privately sharing technical specifications? Concretely depicting abstractions is something I've been drawn to (looking at you, design patterns) and what you described sounds ideal! Admittedly, I must step away for work before I could get through the rest.
I think if you're generating and maintaining code using an llm, then i think we need to treat the prompt and spec as the source code, and treat the programming language representation as an ephemeral artifact.
13 днів тому+1
Not necessarily. The previous source code can be also input for the LLM, then you only need to specify the new or to-be-changed parts. The previous source code acts as part of the spec in this case.
13 днів тому+1
Also it's impractical to regenerate whole modules by LLMs due to the flakiness. If you have working and well-tested code already, it's more productive to keep the working parts and generate more code and tests for new functions.
@karolstopinski8350 possible. I think it comes to how specific you are. If it does something different that's not what you want, make the prompt more specific. Treat the prompt as a system specification.
Gosh, how I wish I could *actually* descend up and down abstraction layers. It has bothered me for years that I can't, and that the industry seems to be actively trying to delude itself into thinking lower levels don't exist. Have you tried Forth? It seems strange that it wouldn't be mentioned in this video, being one of those classical programming languages which directly contradict the idea that there should be any separation between abstraction layers. It's maligned (with good reason) to be a difficult language to write and read, but it has an incredibly creative solution to the function problem: the base building block of the language isn't functions or expressions: it's words. Words directly map to some sequence of assembly instructions, and "function calls", if they can be called that, always map directly to some known set of assembly instructions. All of RAM is treated as a single block containing both code and data, and the interpreter is designed to execute directly from RAM through careful use of data headers and such. Of course, that's all interesting in theory, but in practice, it's used almost nowhere because of how alien its interfaces are. "Standard" Forth doesn't have a natural way of handling text streams, but more importantly: it's practically impossible to "compile" in a meaningful sense. Since classic Forths rely on writing data directly into RAM, the Forth "interpreter" is essentially a just-in-time compiler. The closest to compilation that's available are "Forth images", snapshots of memory at known points in runtime... which contain interpreter code themselves, meaning that, say, a GNU Forth (Gforth) image *must* be GPL'ed, by design. The incompatibility with compiling runs deep. Calling C code involves statically building a wrapper library, and each implementation does it totally differently. It's actually *impossible* to write a classic Forth in ANSI C, on account of memory block management and register usage, and Gforth accomplishes it by using dirty inline assembly tricks to force register allocation, since GCC has totally ignored the register keyword for over a decade. I've heard it said that Forth isn't the best programming language out there, but if there *was* a "best", it would look more like Forth than C. I think the branch of languages where code maps to known binary directly is unexplored.
This whole conversation seems a bit over my head, as I understand that you can make basic logical gates (and, or, latches, etc). What I don't understand is how higher level programming is essentially wrong in some way. Seems to be like if there is some data loss by higher level abstraction, that is simply due to a lack of understanding by the programmer? In mechanical systems, you can use a power tool without understanding what the tool is doing. This person is more a mechanic than an engineer. Furthermore, things are not really understood until they are interacted with first hand. Hard to say anything about assembly until you've interacted with it first hand. This is akin to the allegory of the cave where people are learning from the shadows on the wall vs actually interacting with the electrical engineering. An employer has a much higher demand for code mechanics than code engineers. Not every mechanic needs to know how a torque wrench works, just how to use one correctly. In sum, I think that programming might have a lack of people with zero-abstraction knowledge bases (full truth understanding). It's true that these people exist (like people who write their own coding languages), it's just that code mechanics get a much higher ROI on their time investment.
@ You're absolutely right, there's nothing wrong with high level programming and being a "code mechanic", and you've identified correctly that our economy needs more mechanics than engineers. The issue is that tech employers have absolutely no clue what the difference is. It's infuriating. The issue that arises then is that our infrastructure (that is, the internet's) ends up being designed by mechanics rather than engineers. As a concrete example: PHP, a programming language which was originally designed by a hobbyist for use in his "Personal Home Page". Its inventor said that an intentional design goal was to flaunt common engineering sensibilities and be "easy to use"... at the cost of being a nightmare to debug. As a result, in the early 90s, it was adopted by Facebook, Wordpress, and most online shops at the time. It remains the most popular platform for eCommerce today, and Facebook has yet to remove it from their system. Times are, thankfully, changing- thanks to understandings by bigger companies that there *is* a difference, and now agree that new technology that's meant to be used by others, going forward, should have some committee of engineers discussing it. But smaller employers are still practically blind to the difference and hide "mechanic" jobs behind engineer level qualifications. Hence, my desire for a programming language that's more "honest" about the systems behind it, rather than hiding behind abstractions and insisting "Hey look, I'm just as good as programming at the low level! Anyone who says the low level matters? They just want to be paid more!"
you can totally go up and down abstraction layers if you want to, you can learn how the programming language and the compiler, assembly work if you want to, maybe bothersome to some but thats what makes programming fun to me
@@MommysGoodPuppy I decided to quit comp sci and go mechanical engineering. As much as I liked manipulating numbers and letters, I feel that mechanical processes are more interesting to build for me. I just could never think of something I actually wanted to build just using code.
I think a big thing people are forgetting is that AI can reflect. The AI isn't just going to generate new code to replace the old one. It can look at the old code to predict what the inputs and outputs are going to be and then update it from there. Manual changes should be a lot more preservable here, or the AI can make its tweaks directly to those changes.
I wish we would take a more "start low, to learn how things work from the ground up, only then introduce more high level tools" approach to IT education. In pretty much every metal related job, at least here in Germany, formal job education starts by giving you a block of iron, a file, and a bench vise, only. Your task for the first weeks is just "make this flat on one side", "now make this flat on another side", "now make sure the angle between these two sides is 90°", to give you a feel for the material you are working with. Only then you are gradually introduced to the various power tools, up to almost fully automated CNC machines eventually. I think software engineering education should take the same "get a feel for the material first" approach ... But maybe I'm just a boomer who had to learn these things that hard way back then, and now just envies the current young generation for the tools they have now while I slowly start to become too old to learn new tricks?
You are correct, and as I see it, this is how universities teach programming. But there are a ton of lower education courses that go straight to programming websites. So in those courses you essentially only learn the programming syntax and no real computer science.
I learned with 8086 that had LED lights to show the 8 bit word you instructed it to do, the abstraction layer were so called mnonics - you could advance the bus clock with a trigger, or switch to quarz clock to start the automatism. It helped tremendously doing this for the rest of my life. (Even if i dont do code anymore.) Its valuable to know how its done. I had to grind that metal block too, it sucked back then, but surly it teached.
@@hartmutholzgraefe Maybe so. 😅 At least in my case it was like that. We designed a RISC processor, built it in a simulator, and then made it run some code we wrote for it. Then they made us program an OS with multitasking, and also a Candy Crush clone for the Nintendo DS. All written on straight ARM. And that thing had a devilish architecture with 2 processors that you had to synchronize because one had access to the graphics chip, and the other had access to the buttons and audio. Not to mention all the discrete math classes. We went pretty low level. And in fact they never taught any programming language, just programming in general. It was a given that you would learn the syntax by yourself to any language you had to use.
I agree. Same with any other basic education, let's say mathematics. You have to work your way up from 1+1 to integrals and differential over YEARS! Just giving a calculator to a 4th grader to type in some numbers and get a result is dead wrong! You need REAL understanding of the problem and principles! Even if you are going to use very advanced machines afterwards! Sure, we have to talk about some details, but this is not the problem.
There is nothing bad with abstraction, the core problem is the integration of the components. Programming is inconsistent, solving it is to make it consistent, and this is already done, but we call it by another name: math. A consistent program is equivalent to a mathematical proof, so the solution should be rebuilding all as mathematical proofs. This, indeed, is incredible more difficult: there are programs (theorems) that haven't been implemented for years nor their counterpart (that the program is not possible).
@@Me__Myself__and__I And it can't because such ability to answer everything, will imply it is inconsistent. Maybe a better name, instead of math, is logic.
@@danielmurcia6755 okay, good software and being a good software engineer/architect requires logic. Math is optional 99% of the time because the vast quantity of software systems only require rather basic math. Exceptions would be niche areas like science, academics, game engines, etc.
@@danielmurcia6755 this is the best interpretation so far, I think. It really comes down to the fact that abstraction should work fine because it's describing logic. The problem with AI is, is that the AI doesn't know logic well enough yet, and how logic is fundamentally different from real systems and objects. Like how theoretical calculations lead to lesser actual yields, which create error margins, and then designing systems with the error margins accounted for. Eventually it should be able to do these things. We'll see though
6:10 I actually kinda disagree? A lot of higher skilled programmers in the backend have likely looked at raw x86 or arm assembly. Even more rarely wrote some. So not your average javascript dev. This is a sneeze away from the punch card example brought up, just adjusted to the current day with the environment of compilers, +10k instruction large programs, modern pointer logic, etc... Programming then and programming now is still just kinda about crafting a set of instructions the CPU executes. (or gpu or embedded chip, what ever it is) It's just the scale and the tools that have changed a lot. Before you would remember an instruction address so you can jump to it. Aka an if statement. Today we write if statements without a second thought.
I think this comes down to the fact that programmers think they are engineers, when they may be mechanics. An engineer designs the torque wrench (high level function), and a mechanic uses it. AI might not do a good job at programming because there simply isn't enough good data. If you understood how LLM's had a breakthrough because of more data more compute, you could extrapolate this to coding and thus its plausible that there just needs to be more good data for coding to become a solved problem. For example, if you had to reinvent the car, but all you had to go off of were mechanic's service manuals, listing the torque specs for each bolt, you would be SOL. But the AI can still become a pretty good mechanic when it has enough information.
@@williamtburt Yeah I do agree but I feel like there are some fundamental challenges that need to be addressed and I don't see talked about a lot. The current LLM technology is not a good fit for code or genuine intelligence. Something that is highly needed in this field. By LLM technology I mean the next token generation LLMs like chatgpt. Sure some emergent behavior can happen but AI testing has been incredibly flawed and is in my opinion at least to some degree investor fraud. Simply creating never seen before questions or adding irrelevant information in to a question will very significantly degrade behavior we perceive as reasoning. So I am not convinced MORE training data is the way to go. Big companies already scraped the entirety of the internet at the expense of everyone and it is still not enough. At what point is it just beating a dead horse? Secondly, a more vague problem. We can only rely on AI so much because it is done trough a natural language. The language we speak is short, highly contextual and most importantly vague. That vague part is important because code is EXTREMELY exact so when you try to translate something vague in to an exact set of instructions, you can only get out at best the most common, average result that fits what you described. While it may be correct, I do not wish to live in a world where everything is just kinda fine. Progress is made by putting in the effort and making a leap so impactful, the average of the past is no longer acceptable. This is basically the same problem as the collapse of training data where AI content is being fed back in to the training sets.
There are a few industries that still even require peer review of object code. I hear a lot of grumbling around the office when it comes up, but yeah, they have to review the complier output. And just this last cycle, we had a bug come through that came down to the complier not doing _quite_ what you wanted, leading to crashes. It wasn't an actual compiler bug, but did depend on specific behavior that it didn't do.
@@williamtburt I never got a clear answer since it was hardware, language, and compiler I was not familiar with.. but it was something to do with how the complier handled range checking and throwing hardware exceptions, with two functions which from a language perspective you would expect to behave the same, well, not doing the same thing... and some consumer of both functions that expected them to behave the same way when supplied with out of range data.'
If I’m understanding the argument right, it’s best to have abstractions, as they increase workflow speed, but maintain your ability to write and understand them yourself, with the ability to mentally climb up and down the ladder. I wholeheartedly agree with this. It’s kind of baffling to me that the norm is to just accept errors as normal. I had to patch a type in a form validation library we were using, and my coworkers were baffled that I just *edited the library code* and stored the patch in the project, but allowing errors on the master branch is not acceptable to me, because it devalues the ones that have larger impact. We should be taking responsibility for the product we release, not the code we write
There are some interesting points in the video, but at the same time I think you're working a bit up from a conclusion, perhaps out of some ignorance for some details that are not immediately obvious. For one, abstractions aren't really meant to hide a layer below because it is more complex, in fact it is quite the opposite! Most low level languages are excruciatingly simple, but also very tedious, and some programming environments allow you to mix levels freely, which contradicts your main point! You can inline assembly in Rust, C and C++ and more system languages easy as pie! Rust allows you to switch from the managed memory context to the raw one with unsafe{} even! Even OSes allow you to directly talk to hardware if you so desire and the user sudo's your programs. You know why people keep dreaming of doing this but don't use those system in place except when they are maximally effective? Because acting low level is fun for 5 minutes and then becomes tedious and annoying. Most places where abstraction sucks are usually historic piece of garbage where you have stratification of some historic context and proper lack of standardization like the graphic pipeline, but this is not the fault of abstractions themselves. Finally the crisis in programming is due to the industry culture of "move fast and break things" (like it spells right there. Doesn´t even mention abstractions, and indeed making *proper* abstractions is something against moving fast), alongside with the big money flying around the industry that require thousand of lines of codes shipped but doesn't have enough skilled programmers going around for it. Having thousand of code monkeys doing GUIs in react that needs to be shipped with short dev cycles will result in bad code quality, no matter how many abstractions you torture. Like if we had to build a new bridge every day over 10000 rivers you would run out of decent welders and you would see more accidents, no matter how many osha memorandums you sent out
@fluffy_tail4365 wrote, "Having thousand of code monkeys doing GUIs in react that needs to be shipped with short dev cycles will result in bad code quality, no matter how many abstractions you torture." Incorrect. It is easy to implement hundreds of software quality assurance tools and use them to guide the LLM. You can pass code written by humans to through such a system and automatically fix flaws and bugs.
have similar comment about "most fast and break things" tl;dr leads to us all in a sunk-cost fallacy boat with no personal time for higher and higher productivity requirements fewer and fewer benefit from.
YES PLEASE! I’d love to see a video on the research you had been doing . I love the way you explain things. My second favorite channel after fireship. But better in a way because it feels more personal.
As a SE, I’m happy people are talking out about this. My general points when explaining why AI is not going to solve everything are: garbage in garbage out(the basis of the models could have a large portion of junk in its learned data), the art of prompts(between two humans we can ask for the same program but if one knows actual programming theirs will be closer to usable code), the output is usually incomplete/broken(this is why copy and pasting results is a huge risk). I personally use it a lot to fill bits I’m not so great at, which actually helps me complete an idea but also understand what I don’t before. I would never and will never trust “make me a component that does x” and just drop it on and trust it.
I think a huge problem is the way many coders learn. There’s a just make it work mentality and who cares how. It’s gotten a bit better lately but I remember when looking up stuff for c# especially on stackexchange it seemed like all they did was scream premature optimization and just make it work no matter what. All that matters is how fast you can spit out a product and move onto the next. The result of all that is a lot of people who largely only know how to call functions from some library with absolutely no understanding of how the computer works, what’s going on in the library, execution speed, or even error handling. Who cares about errors when you can just try/catch everything. Well when that’s the mentality of many coders and corporations it’s unsurprising we have a lot of bad code. A big part of my education in school was working with assembly, c, building data structures from nothing, memory management, and cpu architecture. That made a big difference in my understanding of programs even when I’m using a higher level language.
You can also end up with a very insecure project with premature optimization. Especially if optimization is your main goal. Optimizations are often little hacks to utilize stuff better, and optimization for speed often neglects security. And yeah, you kind of need the code relatively quickly. You cannot have every program that a company needs 2-3 years later down the line.
@ well my point is taking an extreme stance on either side leads to problems. We have such buggy and exploitable programs specifically because people took an extreme stance that all that mattered was how fast you could deliver a program. Sometimes taking the time to do it right saves time down the road when you don’t have to refactor or fix a billion problems and crashes. Of course you can take too extreme a stance in the other direction and take too long to get anything done. There’s also the problem of the more you tell someone something is safe the more likely they are to do something unsafe. They’ll ignore warning signs because there’s so many safety nets. It’s very possible to write exploitable code in any language. So when the goal is screw quality control and forget learning how a computer works at all just pump out code as fast as possible what you’ll get is low quality buggy and exploitable programs. The only solution to this is giving well trained programmers time to do the code right.
@@ryandodrill6904 Extremism, regardless of where, is terrible. It also comes down to the extreme OOP for example, often leads to terrible and awfully unreadable code. And sure, it is very possible to write exploitable code in any language, but that regardless, writing programs requiring modern complexity in Assembly is very VERY!!! likely to lead to memory management. The problem is that even if you know how a computer works, that doesn't save you from it, because there will always be the tiny error, hence why higher level languages tends towards being more safe overall. I'd rather have a programmer who has no knowledge of the silicon what so ever, no real understanding of the CPU architecture, but aware of the pitfalls in programming that can lead to security exploits in a high level programming language, than I'd prefer a very educated programmer, that has learned all the intricacies, and tries to do the same in assembly. Very unlikely that the assembly coder will be able to produce safer code. And yeah, sure, the mentality of pump out too quickly is bad, but the mentality of making way too secure is also bad, because it'll never be done within a reasonable time frame. A little simplistic point here, but you can essentially fix any problem given enough time... but we do not have endless time. We will never have endless time... and regardless of how secure you make the system, the most vulnerable entry point will always be humans. It'll be that one Wi-Fi spot, that one procedure, that little loop-hope in security permissions, etc. And as you mention in the first paragraph "Sometimes taking the time to do it right saves time down the road when you don’t have to refactor or fix a billion problems and crashes." is a great sentiment, until you hit the problem of scope, features, or other stuff coming in. The better option is frankly giving the programmers enough time to do their job at the end, when they have a functional product in many cases, because at that point, you know what it'll need to do, and stop feature requests at that time. Otherwise you do end up with the development hell of constantly refactoring. Something that would take a month, takes half a year, or suddenly it takes a year, etc.
13 днів тому+1
I think the "get it done" mentality is actually rooted in wisdom. A competent developer will choose the right algorithm or data structure by default when writing a function. This is different from optimization, I'm talking about basic things, such as using a set instead of a list (in Python terms). Optimization should always come after the correctness and completeness of the program is verified by testing (preferably automated tests). It's unwise to get a program 10% faster by optimizing if doesn't fulfill its purpose yet. Also programs are vulnerable to constant external change: new business requirements, new hardware, new programming languages. Software is called "soft" because of the fact that it can get obsolete extremely fast, so getting things done to be able to start using it is the highest priority. If the program is too slow to fulfill its purpose, then yeah, optimization is important, but it should be guided by benchmarking, not random little hacks. I want to emphasize though that good initial architecture has a big impact on large software projects and reevaluating the architecture is important for growing software, but it's automated tests which support this continuous refactoring, not optimization.
Having fluency in a language is the exact skill which allows the fluent user to achieve mastery, unlike a non-native speaker. Consider the task of modeling 3d environments. A master modeler, familiar with various tools and techniques, can translate a text description into a beautiful scene. Ok, so can a clueless novice equipped with an AI, you might claim. But the reality is, no matter how much the novice keeps talking to the AI, they will not get the result at a level of a master. For the simple reason that they are lacking the "words" and "phrases" that are needed to formulate what they want to achieve or change, and if they try it, it will be in a dramatically more inefficient way than the master grabbing the 3d brush or material or light editor and adjusting a few faces in the exact location that needs adjusting. You could also say that the direct translation of "wanted quality" into "necessary action" already happens in the brain of the master, it is not externalized or communicated in any form, apart from the result of the (proper) action. And how do you want to train AIs to mimic this in domains where there is no such external written down training data? And why would anyone who values their skill even want to transfer it to an AI?
I'm not dare to say that I achieved mastery in creating music, but I have spent a lot of time doing it, and this is my exact thoughts on AI generated music. It's great for AI covers and memes, but it will take infinite time to guide an AI to create a song you want. It's fast for people who lack the skills of a music producer, but for music producer it's slow and imprecise.
Currently, AI image generation is purposefully being stifled so that the TRL (Technology Readiness Level) is lowered. Even still, the market is trying to utilize AI image generation. It will spread like a virus. I've been an artist before, and your brain always wishes it could bring the magnificent emotional detail from your mind to the art piece. this is the climax of art for the artist. finishing a piece is just work a lot of the times. this is why sketching is so popular and fun for the artist
@@williamtburt nono, the birds haven't been revealed as spy cameras yet so that the BRL (bird readiness level) is lowered. Even still, the people are trying to reveal birds as spy cameras. The knowledge will spread like a virus. I've been a bird spy catcher before, and your brain always gives an emotional high from revealing a bird. This is what its like to climax for a bird spy catcher. Catching a bird is just a lot of work a lot of the times. This is why using security cameras ti catch the birds is so popular and fun for the catchers.
2:20 This is by far the greatest use for the current generation of AI imho. You cannot rely on these models to actually write code for you, nor should we want to. and they're also shit for learning entirely new concepts as they lack the structured approach real educational content can provide. But if you know the right questions to ask about something niche, can feed it data about something very specific, or are looking to break into a topic related to something you're already somewhat proficient in, they can be an insanely powerful starting point. For me, the number of projects made possible or vastly more simple just by asking something "Are there any tools or libraries that do X and work with Y?" is amazing, So often, no knowing what you don't know is the biggest barrier to doing interesting things, and these models are some of the first tools I've ever found that can solve that problem an order of magnitude better than a search engine. I wish these AI companies would focus more on what the models are already pretty good at instead of trying to make them replace in any every job that could exist.
Current generation of LLMS is: bad questions get bad answers. With most people being incapable of realizing how bad their questions are; let alone not being misled by the "answers" that are generated wordsoup from the most complicated adlibs systems ever devised.
I can’t tell you how great it is to find someone else who sees this! I’ve not been able to dig as deep, but I’ve become convinced that abstraction has the unintended effect of _increasing_ complexity. This is super counter intuitive, completely antithetical to what they claim to do, and makes people I try to explain it to look at me like I’m nuts, but I’m convinced it is true.
This exploration of the merits of compilers leaves out assemblers as a concept. In low level assembly languages, you get 100% of the control that you got from writing directly in machine code. Every assembly statement maps directly to a known sequence of 1s and 0s. It is even entirely possible to write critical sections of your C or other high level language program in Assembly as long as you carefully maintain an API for the C and Assembly to communicate with. Rollercoaster Tycoon is a great example of commercial software written entirely in Assembly so that the developer did not give up any control at all at any point, yet got at least some of the benefits of a higher level language by using an assembler instead of a compiler. By adding some syntactic sugar on top of an assembly language (macros mostly) you can even get a lot more power at your fingertips in a custom assembly language or an assembly / high level hybrid dialect. I think the real problem might be the hardware getting more complex, not the software.
@@markgacoka9704 Yes and no. First of all, computers sometimes *do* make mistakes. From the FDIV-bug in the Pentium to random events like high energy particles or failing components due to aging/overheating (see the the Intel 13th/14th disaster). Secondly, the AI algorithms themselves are flawed - they are basically stochastic parrots on Speed and would, even given perfect training data, only replicate (i.e. overfitting/recall) or still hallucinate plausible looking BS. They do *not* incorporate theorem provers or even just syntax checkers. This means that the algorithm itself is - by design - very error prone, too. It has to be, otherwise it wouldn't be able to generate novel solutions and just be a code snippet DB that can be queried using natural language.
Wow, you’re hitting right on something I’ve discovered while working on my software project. Most of my struggle has been determining what dependencies to use, and I’ve found that the decision has a lot more than just how much the bundle grows. A big part for me is considering learning the library. For example, I decided not to use lowdash, mainly because it would expand the size of documentation to research when trying to solve a problem, which would actually slow me down. I have to constantly keep my perfectionism in check so I can balance between an acceptable pace of progress without leading the project into a big ball of mud.
The problem created by abstraction is the same problem that is created by symbolism. Translating semantic content between languages still has linguists stumped too. We know all the phonemes: the way a mouth can make a sound, and we can diagram how they all fit together to make meaningful utterances in a given language, but we can't *translate* the meaning.
The bidirectional mapping between layers of abstraction was a really interesting idea. I enjoyed the video, especially the idea of replacing functions with some other fundamental building block.
imo, as you are talking about this is that Julia is actually pretty close to this. I don't think it'll happen with Julia simply because the architecture still relies on LLVM, and, at lowest, C API's to lower level. But the way it handles functions and packages allows a lot of interactivity through the entire library.
You unintentionally have the answer in your post, assuming you accept the universal reversibility constaint as correct. LLVM IR, except with much more infrastructure in the VM for carrying out the operations in both directions, as both a compiler and decompiler, and some kind of requirement for extended debug symbol inclusion. I don't accept the constraint as correct though, because it necessarily requires universality of language concepts, which it seems would result in really bad languages.
I love how this video gives me time to think for myself. When you finally got to explaining reversibility, I was pleased to discover that what you were talking about was exactly what I had been thinking about for a portion of the video. Thank you for inspiring me in this way (and making me feel smart).
Not is not that complicated to understand Code or Compiler => deterministic system AI => indeterministic system > but but we should be chill to have shittier programs > but but we should allow ourselves to continue in the decline trend After 20 years in the industry, it is quite an awful take since we all refuse to be accountable for the decline of quality in software across the entire industry. We didn't had a REAL breakthrough in more than 20 years everything else added more bloat and mediocre solutions. Most of the good solutions came from people programming ON THEIR SPARE TIME while big corpos are leveraging that code without contributing back. Even your example at 13:33: - Cross platform framework that we have sucks -> Nothing beats good old C/C++ - DSL always sucked - ORM introduce such a bloated code and queries that you will be forced to re-write it anyway, at best use a code generation tool that way better - Frontend frameworks are a distraction and bloated -> look what solidjs did with a very simple library OR look how fast we can be with just server-side rendering and jquery - Most games these days sucks, big corpos are switching to UE because of incompetency in their teams -> using a game framework or game libraries is still better than a game engine - Website builder are just the most awful thing ever for a real corporation, if you're a solopreneur yea sure it is good and WP works for most cases It is not a "Low level vs High Level" It's how can we build better technologies that enhance the human potential and it's creativity without creating unnecessary constraints It’s all hype-driven nonsense in the software industry now. People aren’t learning programming out of a genuine passion for creating or innovating anymore; they’re chasing the latest job trends. And who’s behind this? The same big corporations that treat developers like disposable cogs in their profit machines. These companies flood the market with buzzwords, shilling Rust, C#, Swift, Java-whatever language lines their pockets best-and the masses lap it up like obedient sheep. Most of the jobs they offer are glorified "web shitter" roles anyway, building yet another bloated, ad-infested app or a soulless SaaS product. But that’s not even the worst part. The new wave of developers, indoctrinated by corpo propaganda, parrot their talking points: “C is useless. C isn’t memory safe. C is old. C is bad.” What an absolute farce. They don’t see the bigger picture: C is the foundation, the backbone of everything they rely on, from operating systems to embedded devices. But no, big corpos can’t profit from a language that empowers individuals to think critically and build efficiently, so they smear it instead. It’s laughable. It’s tragic. It’s exactly what you’d expect from an industry hijacked by marketing and mediocrity.
When I started to code, I coded in C#. A pretty high-level language, and I managed to get visible results in Unity and WPF. Just after that it was necessary (or beneficial) to dig deeper into it, to understand underlying processes, to optimize functions, to pursue good project and code structures, to go low level and write code that accesses memory directly. It doesn't help to dissuade people from coding by putting them in front of C and write menial and simple things which require a lot of effort. I remember what joy I had when I wrote my first program, a calculator in C#. Terrible code, but the end-result was quite good. I also remember when I managed to have 3D objects move in Unity. It was great to see your code actually *do* something. Well, today I work on a game that spans the observable universe in meters, and is riddled with systems and math that are of extreme difficulty (already done).
That's why my favorite language since around 1999 when I started programming, is C. It has just enough abstraction, but not too much. If only there were as many jobs for C developers as for Java, etc.
That! I don't understand when people put C as low level as almost assembly. It's so far easier to understand and write code than assembly but still very very good for speed that I don't see any reason for anyone to ask for more abstractions than this.
i think you are underestimating on how much abstraction gives C gives you, it is a LOT more over raw asm the jump from asm to C is much bigger than C to the most insane boilerplate oop language out there.
@@khhnator Agree with that. That was the point. Assembly to C is big breath of fresh air. Code can be read as a high level algorithm. C to C++ is tiny. Sometimes I start a C++ project but a lot of my code looks like pure C. And creating too many classes and hierachies doesn't make it easier to read in my view. Sometimes it adds more complexity. C was enough for everytyhing.
C is fine for smaller programs, but extra features in C++ like namespaces, function overloading, class functions and class constructors/destructors, operator overloading, the STL is massively helpful in larger projects. I'm not a pro, but all the syntax rules in C++ often allows people to make code that is easily misunderstood. "const" for example is a very context sensitive keyword and often poorly explained to beginners. The C/C++ code styles with excessive C macros are a nightmare for new people who were only taught C++ and basic syntax rules for programming concepts only.
@JohnDoe11VII Yes, objective oriented programming is very helpful for large projects. The issue with C++ is that the language is already huge and keeps growing... C++11 was a useful step forward since C++98, but then the updates accelerated rapidly. There is now a new revision every 3 years. It's hard to keep up, and the language is becoming very complicated, if it wasn't already.
In order to fix some issues and to implement new features, last year I (partially) reverse engineered the firmware of synth. The insights you gain, when you try to decompile such a small piece of code are invaluable. There are so many inefficiencies, stupid conversions, bugs, code copying, security issues. Most of these, I assume, stem from using libraries, that in-turn use libraries, i.e. abstractions built on abstractions. Nobody ever looks at the end-result.
My fundamental problem with AI programming is being able to interpret the prompt as a programmer would. The AI doesn't push back and say "Actually that's not a secure pattern" or ask for more information it just "does it". More of my time is spent unfucking things that should not have been done, then it is doing things. It's also not determinative. Compilers will at least give you the same answer twice. That with the hallucinations makes me think I'll spend more time understanding what was just written by AI, versus writing it myself.
We can't get quantitative pushback in good LLM tools because people already scream to high heaven when companies tune the AI response to moderate it from becoming toxic. They are being built for corporate midwestern fake helpful unhelpfulness. Like a glossy tier0 support rep that fakes an empathetic: "oh that must be frustration". As they have absolutely no real experience, or power, to actually solve the problem; or even at bare minimum recontextualize the problem into a better problem statement for someone _else_ to solve. The support playbook end result is output that leans towards emotionally manipulating someone into feeling better about their NEW problems.
Yes. I'm a QA and often biggest part of engineers job is to figure out what exactly client/management wants. Making exactly what is asked is a surefire way to get ungodly poor UX which often fails to resolve actual needs of users. You need to figure out the intention and than offer back solutions. Current generative neural networks are incredibly bad an getting intention and maintaining context.
Exactly. Lack of determinism in AI programming is what will prevent it from being widely used. Even if we can get to a point that you can write whole huge programs just from a prompt i don`t think any banking system will allow for that. Users probably won`t be happy when their account balance is subject to hallucinations :D
This is very good. I see so many ignorant people (including myself) trying to make "simple" decisions with big consequences without proper analysis and trade-off awareness. Usually following current "industry standards" or previous experience while not taking into consideration project needs and specifics. Nobody talks or thinks about complexity, long term maintenance or control loss when using different libraries, abstractions etc.
This feels like a very academic view of programming. In engineering terms with stakeholders and deadlines, you will use whatever is the best tool available to make the thing work that you need. If your codebase is already written in a language, you will likely extend it in the same language, and if a black box library solves a problem, you will use it. Despite some idealistic views about how we "should" program, most people are just programming to complete a task, and so the black box method will continue to survive.
Very interesting video. One point on your diagram @5:45 : when AI generates code in a high level language, it is because it has data under this form to get trained on. If there was no high level code data, it won't be able to generate it. If AI was trained only on binary code data, it could probably do it pretty well, but we humans would have a lot of difficulty to undertand the output. So in a way, the AI should possibly trained on both, high level and low level code, so that they can translate inbetween both levels, similar to english-german translators. But ultimately it relates to human supervision, and implicitely the reason why we humans prefer high level code to binary opcodes is because it is easier to supervise (aka control?) at scale than binary opcodes at scale. If AI does not need humans in the loop, then it can go full binary code, but then who would write the prompts...?
Interesting video, well done! Couple of observations: - Compilers are deterministic, while LLMs of today have a random component (temperature setting). - We can write sections of ASM code in our C programs (either by linking them or by using a pragma section, depending on the compiler), so we can still have control over the critical parts even with a "one-way" development flow. Same thing for Python and NodeJS if you go through the trouble of writing a native module, which is a bit more work
Software guy with 24 years of experience here. Started coding with C/C++, then C#, JavaScript and Python. Now I code in English too :) Enjoyed watching this! Made me look at abstraction from a completely different perspective.
I don't agree, the vast majority of problems are how to get your programs logic right. Better tools should ideally give you more time for the analysis and verification, but in practice are just used to churn out more code at a faster rate. "We'll fix the bugs later...Yeah, right...." AI will make this worse. The bigger problem is the jenga tree of unmanagable dependencies. To solve a meaningful problem you need to manage the abstractions yourself, else you're borrowing someone elses abstractions. It's a bit like trying navigating Paris with a city map of London.
I work on a PhD whose main message is how security issues in hardware break down abstractions and that's why we can't easily compile security countermeasures into programs. So this is close to my heart as well-I spend most of my time pondering such questions. I think the expectation of a one-to-one mapping through abstraction is a red herring: it fundamentally contradicts the point of abstraction. Connecting the lower levels back to the higher levels already exists in the form of debug information, and we already know how this goes: badly. If you higher-level language captured all the subtlety of the lower-level one it wouldn't have the benefits that make higher-level languages interesting in the first place. Creating a connection between levels, though, *that* is a different piece of meat that I couldn't agree with more. We already have some of that, too: with ABIs and FFIs. I can write an assembly function in a C program to have performance-critical code in assembly and performance-flexible code in C. Which brings me to a pet peeve of mine: expanding a function call into a function body is not a drop in abstraction level; you do that without changing languages. Functions are by far the most successful abstraction in programming, but they don't constitute a lowering. It's the opposite, in fact: it's the one abstraction so successful that we implement it in every language. Which makes it the perfect bridge for connecting C code and assembly code and code written in other languages with other abstractions. I'm willing to take a gamble and claim that we already have the tools to fulfill your vision: we know how to make binary interfaces to connect pieces of code from different languages (we just need to standardize them), and we have the function interface as a near-universal abstraction to give semantic meaning to that interface.
"Anymore"? I started programming in 1984 and I don't feel that there were less bugs then.. I believe that you have a romantic view of the past, maybe just to make a point? Now I do agree that AI does not solve that issue: it just repeats the same errors, since it learns from existing software well known for being buggy..
These were early days - check how in 90s and early 2000s everything "Just worked" without Product Management team constantly messing features back and forth and teams compromised mostly of technical people.
@@piotrd.4850 no, it wasn't. I still remember those times. I already had a PC in the 00s, and every time I bought a game or software, I prayed to all gods that it would boot up on my PC. Because sometimes, it just wouldn’t. And that was it. No patches, no internet, no forums, just returning the CD or floppy disk back to the store. It's just people tend to forget things and wear nostalgia tinted glasses.
@@alexm9104 oversimplifying is rose tinted, but they have a point regardless. I think what has really happened is that the ratio of well built software and rapidly built cheap stuff has changed. There have always been such types, but the later was generally relegated to games or low cost apps, while stuff that cost real money to by also cost real money to make. I think what has happened is that the later, cheap fast software, has not only become the norm, but the expectation, so all the companies that used to invest in development are under pressure to adopt the 'fast' model. I work at a company that makes avionics, so the kind of software that really can not afford mistakes. our new VP apparently had a stint at Google and believes those are the deisng practices we should be using... fast sprints, fast testing, AI assistants, etc. Our quality has plummeted, ...
great video! it's funny, in your last video on programming as theory-building got me thinking about AI - one of the tradeoffs of having LLMs writing our code is that no one has the theory of the program in their head (unless you familiarize yourself with the AI generated code enough you might as well have written it yourself). looking forward to more videos on this theme!
You can migrate assembly in C and C++, because you can write Assembly in those languages. That's like one of the major selling points. There's still hickups but not your "barrier". I worked with such libraries in those languages.
First time here and I really enjoyed the video, subscribed instantly. I'm not too hopeful about the last philosophical portion about learning something in programming and transferring the knowledge elsewhere - we have a very bad track record of using the things we've learned on managing complexity so far.
@@MelroyvandenBerg Wait till that shit start reviewing loan applications, healthcare procedures, and government regulations. Will be shitshow, we will need years to fix the reversability problem.
... AI is a marketing term that means nothing. There is no definition that everyone agrees on. And machine learning... is not really what these AI pruducts is... The large language model is simply a fuzzy API to use tools we already have, like compiler errors and static analysis. Like, here is a fact: Largw language models cannot do math. They are terrible at it. Because you cannot do word prediction so well that you can do math. It can never work. But chatGBT can do math! Yes... programs, that are NOT large language models, tries their best to spot math question, and then it uses a normal calculator to do that math. Meaning chatGBT is just a REALLY environmentally unfriendly API. And that is what Google and Windows wants. Fuzzy API's. Because how are you going to compare or replace programs that you do not know what is? You will be forced to use the Windows or Google AI. And just like always, they are currently working to change the law so no one else is allowed to make those large models. This is great for them. Monopolies that cannot be challenged, even via law. No competition! And once the user cannot leave, you can treat them as bad as you want it. We are ignoring here that the models are build with stolen code.
The split between library and your own code sounds like the effect of matching the organizational structure: the libraries are written by domain experts who communicate via documentation.
10:08 I think it is easy and comfortable to sit in a chair and criticize the software industry as problematic because it "don't start from low level", but they never really showed any concrete evidences on their conclusions, it is just yapping for the most part.
Abstraction is like putting a mountain between you and a low level concept, you can build a rail, or a road to go over that Mountain but it still has its own problems
Abstractions are probably one of the biggest problems in software. Developers create abstractions of problems they don’t fully understand. It makes them feel good. It’s easier than investing the time to understand the problem space. Spend less time abstracting and more time writing code and talking to people.
0:25 if we accept the definition on the screen ("Software crisis is a term used in the early days of computing science for the difficulty of writing useful and efficient computer programs in the required time."), then it's obvious what causes the "crisis". Since we include the word TIME in the definition, there lies the problem. This is artificial. It's not that we could not write "useful and efficient programs", it is that we add an artificial constraint, which is - more often than not - added by non-programmers, like managers, the board, etc. They are obsessed about the 80/20 principle for example. I can just hope they don't really dare to use it to program nuclear reactors and such.
This is an interesting video but considering AI something that could ever be an abstraction in the first place is strange to me. The stuff we have right now is inherently random, with no way to turn that off? And even if we could, the slightest change in the prompt text could affect the result substantially. In the same way that a senior dev telling a junior dev to do something is not abstraction, AI can’t be abstraction either. I don’t really have much to comment on the rest of this though. I’m in the position of using typescript primarily and that only transpiles down to js, which is already human readable, and sourcemapping makes debugging easier, even if the js gets uglified. I very rarely if ever need to look at that layer. Maybe things are different in other languages though? The one note I have about abstractions via a compiled language is that oftentimes the abstraction language just has different goals in mind. Usually simplification. If you could go both ways the abstraction language would need to support literally every bit of syntax in the base language in its own way, which in many cases sounds pretty pointless. It’s like you have an abstracted language with gc compiling to a language without gc where you have to manually free memory. It would be _intended_ to not have the syntax to manually free in the abstracted language, but it would be _required_ if it’s two-way. Any abstraction in your paradigm would therefore just be syntactic changes and well-known code being turned into what’s functionally macros via abstraction syntax, and it would otherwise be a superset. Abstractions would be far less useful, no? Maybe I’m not quite understanding though
Sure abstractions can be non-deterministic, it's just usually not stochastic. A non-stochastic abstraction can still be non-deterministic in the way that it must account for all reasonable implementations of the interface of the module that has been abstracted over, such as the final implementation of your program on all its viable hardware targets. The final deployed machine code is not determined by the programmer, but they have at least specified some constraints on the behaviour of the final system, so what's different when the final system can be stochastic but still stays within the boundaries of those constraints?
@@MagicGonads While I agree with what you're saying, I think my core argument is kinda unrelated. I just don't think language for communication could be an abstraction. There's just too many ways to say things, and too many ways to interpret each of those different ways. In the case of LLMs I think it's even worse. And I don't think all the training data in the world could fix that, either. Even if there's an LLM you can turn randomness/fuzz down to 0% on, I think the sheer variance in possible outputs you could have just by changing your phrasing a bit is enough that you need constraints expressed in ways other than english language just to verify it's doing what you want. In which case you're desperately using a leaky wooden bucket, covering over all the holes you can spot with duct tape, just hoping you've caught them all. You basically need to have a language with rules so there's no possible confusion of intent, and to have the LLM be trained on that instead. Why aren't we just using a normal programming language? And none of this is even taking into account AI alignment as a concept! It's a black box, who knows if the weights interpret things the way we do. Who knows if there's hidden stuff in those weights? If AI generated code includes a backdoor is it a valid abstraction to you? I guess it comes down to whether you believe something is still an abstraction if using it causes it to fail for no discernible reason sometimes
@@ChiriVulpes You're close! I developed a free open source platform that is designed to: . Convert pseudocode to whatever language you like (ABSTRACTION) . Use extremely strict quality assurance tools to scan the code for flaws. . Automatically guide a human developer or LLM to fix all of the flaws. . Automatically run reports on the entire code base and improve it. . Automatically write and run unit, mutation, and integration tests. The result is much higher stability, security, simplicity, and speed.
@ChiriVulpes wrote: "You basically need to have a language with rules so there's no possible confusion of intent, and to have the LLM be trained on that instead." Incorrect. The code just needs to be able to pass very strict tests. Intent is proven with unit, mutation, and integration tests. @ChiriVulpes wrote: "Why aren't we just using a normal programming language?" Pseudocode is much easier to read by NON-programmers, Quality assurance tools, and LLMs. Write pseudocode to CLEARLY COMMUNICATE INTENT to non-programmers, QA tools and LLMs. Use a platform to convert the pseudocode to real code and fix all the flaws in the code. You can also use code written in "a normal programming language" as the prompt to the LLM. It is more challenging to refactor code written by experienced developers to a more simple style. Developers often use OOP and other techniques that make the code harder to comprehend. The programming paradigm will shift when developers revert back to writing lots of boilerplate. Procedural boilerplate code encapsulated in < 3000 token sized functions is much more clear. Correctly structuring the code while avoiding abstractions allows us to automate quality assurance.
There is an important difference between current AI and compilers. Compilers are determenistic, and it is possilbe to explain what exactly each part of a compiler does and why is it necessary. With current AI, we do not know what will it exactly do to the code. There is some interpretability research, but it's just not enough (yet?). Will this problem prevent "good enough" code or not - that's a separate question though
The "2 hours coding + 6 hour debugging" vs "5 min ChatGPT + 24 hours debugging" is simply not right. If you know how to use AI, it's more like "2 hours failing to role your own + 6 hours researching hot to do it right + 2 hours implementing it + 1 hour making it compile and "work" + 12 hours debugging it" vs "30 minutes AI + 1 hour code adjustments + 1 hour unit tests and it works".
or just use a search engine and get the original stack overflow answers with code the AI screws up by trying to combine several answers instead of just being able to find the correct one for your case that likely has the most votes.
The non-AI case only applies if you learn nothing from project to project. If you're working intentionally and not just copy+pasting other people's code, you'll learn how to execute a project with minimal or no research at all.
@@michaelcummings7246 Laughable. Stackoverflow is full of crap. And the most voted answer is usually "Use boost" followed by "you're doing it wrong!!!"
@@zoepentaleri If you can "execute a project with minimal or no research at all" you're probably a web-developer code monkey, not an actual software engineer.
This fatal flaw is what essayist Peter Welch was talking about in his essay "programming sucks". It is the same flaw that prompted Bill Bryson to make his famous "They are, in short, a perfect match" observation. It is this flaw which I devote my life to, if not eradicating, at least mitigating. To put it bluntly: If software I use does not meet its contract for usability, it will not be used for much longer.
Something that comes to mind immediately is the idea of training and LLM to generate code on a byte by byte level. Impractical today, but like you said, if if the speed and volume of these models increase, it could theoretically solve this one to one problem.
To be fair they were probably right about the performance of code produced by early compilers. I mean just the fact that "optimising compiler" used to be a term that was in use, but now people just say "compiler", because the base assumption is that it can do a lot of optimisation, should tell you something about the optimisation level of code produced by early compilers.
I like your thoughts about working on all levels of abstractions (16:00 etc). However, there already is a simple solution for that, which is also widely adopted: Dependency Inversion. A simple example: My functionality depends on an interface, that abstracts all complexity. There is a "standard implementation" (think about the library from your earlier examples), that implements this interface and that is used throughout the code. Now this is what I would usually don't touch (the equivalent to using the compiler standard output). But in certain cases, I need to override this module. This is where I would inject low-level code (or even: optimized byte-code) into my module that accesses it via the same abstraction. We also do this in UI, where standard components that render some buttons have "slots", where we can inject our custom implementations in case we need more control over the "header", "footer", "action buttons" or "body" of a certain element. Very simple, highly effective. And it allows you to ignore the lower level if you want to, but you can always jump to the level, when you need it.
i don't know about other programming languages, but this is basically python. You can take any function/class and add onto their functionality for a custom implementation (or completely override it)
in terms of UI at least for browsers and other things a user would enjoy. I've often felt that allowing us some level of control would be really helpful. hum, While a user could redownload their software if they screw something up it'll be neat to have some sort of "Revert to Default Conditions" button too. Like what a motherboard has in their BIOS. Also also. Thanks for not making this click bait. Your video was well put togther and well thoguht out. Count me as a new subscriber.
Your comment about looking for reversibility in abstractions reminded me of two things: reversible computing, and how a dual object is represented differently in lambda calculus and in sequent calculus. I'm not certain that the category theory concept of a dual object is exactly what you're looking for, but it may be in that direction. A dual object in a Hindley-Milner type system (based on the lambda calculus) may exist for a specific abstraction, but they are generally non-trivial. A dual object in a Gentzen type system (based on the sequent calculus) may exist, and the logical inversions are generally structurally symmetrical. To my knowledge, this type of reversibility cannot be generalized, see cellular automata such as Conway's game of life, but would likely help in the expression of these reversible abstractions.
Wow, that was very refreshing, it's the first time I've agreed with essentially everything a creator has said. I think reversible abstractions are only currently out of reach for performance reasons, with enough compute it should be possible to alter every high level code character (individually) in every possible, or maybe conceivable way, and observe the resulting low level code. Those changes could provide an extremely large amount of minutia to train a potentially revolutionary LLM. Reversible abstractions. There's no reason I can think of why every abstraction level could not cascade together to allow prompt to compiled code via any number of interchangeable language abstractions - with editing possible at any level, in any language. I think AI will surprise us in ways we can cannot possibly understand without layers of abstraction. Reversible abstraction in this realm might unlock a far deeper comprehension and mutually beneficial communication. Really good video thanks :)
@@MagicGonads Isn't that just another level of abstraction? If the model is trained on high level, readable code and how alterations of that code affect the low level code, we know that AI can run anything in reverse from language to diffusion. Essentially changing the correct single low level bit would generate the high level code - including code comments. At the rate of current advancement, 5-10 years seems like a reasonable expectation timeline.
@@74Gee the high level code lacks the syntax to specify the low level semantics, no matter what, and even if you find a sly way to trick the compiler it's gonna be an unstable hack
@@MagicGonads Yes I agree it lacks the syntax, however given the correlation a trillion minuscule source code edits and the resulting binary, and the notion of how AI can correlate the sound of a swiping finger to a fingerprint, or the flickering of a door entry LED to a code, or a prompt plus noise to an image, do you think that the reverse correlation of bit swap to source is always going to be impossible? I don't.
But you have to keep in mind how it took for us to trust compilers enough to actually use them on a daily basis! Nintendo still prefered to code in ASM most of the time even in the N64 era. They turned off compiler optimizations because they didn't trust it and rightfully so, since much of the early code produced had bugs introduced by the compiler and not the logic. This becomes even more frustrating if you want to use less-deterministic software like AI. Debugging an AI could be a nightmare even for compiler programmers and everyone who took a step into this field knows what kind of nightmare compilers already are :D
The N64 is a weird system. The bottlenecks in the N64 made it so that normal all purpose compiler optimizations could create slower code on the console. The N64 also had expensive carts, so getting code as small as possible for smaller ROM chips was more valuable than slightly faster code, compiler optimizations can dramatically expand the program size. Nintendo also wasn't very experienced (like most) at the time in a 3D space, huge performance issues and wonky collision (tick tock clock the worst map) exists in Mario 64 indicating inexperience, rushed design, and a lack of communication between creators.
Is not a fundamental "flaw" of programming. It's a defining characteristic. Even though I don't like the guy, the famous quote of Grady Booch is true: ""The entire history of software engineering is one of rising levels of abstraction". AI is adding yet another level
Another effect of increasingly capable tools and abstractions is with the increasing speed and lowering cost of iteration (which is obviously good), we become ever more tempted to think we can get away with not fully understanding the problem we're trying to solve, because we can "just iterate our way to it," (while possibly planting the seeds of catastrophic failure.) A positive effect of iteration friction is that it makes it increasingly profitable to take the time to make sure we understand the problem at hand first. Similarly to how high interest rates tend to cull less-disciplined real estate deals and business formation, and expensive recording studio time used to cull unoriginal music recordings.
Really enjoyed these topics and your coverage of them, excellent video! I like your idea of “two-way” abstractions, allowing software engineers to cognitively navigate from top to bottom of a multilevel abstraction architecture. I’ve had a similar idea, but I was unable to come up with abstractions that can easily organized like that. The problem lies in that between some levels of abstraction, the translation between the two is not algorithmic, but heuristic. For example, if I have a system comprised of interacting state machines, and I wish to instead implement them with a petri nets to increase concurrency and distribution, the conversion/refactor will be heuristic, since there will be different solutions to the most effective way to implement the same behavior as the state machines. Maybe that’s not the best example, it’s just relevant to the levels of abstraction in the architecture i’ve been envisioning. Your example of the user interface editor that directly updates the styles applied in the browser does show your concept working, but the abstraction barriers between the color of a UI element and the actual variable that sets it is probably one of the most direct abstraction boundaries to cross.
I’ve always been someone who cares about the how and why of things. I’m in school studying data science and I’ve noticed that the farther I get into my degree the less people care. There’s almost a resistance to understanding among students and professors. Don’t ask questions that challenge someone else’s understanding. Just copy paste and act like you know what’s going on. I’ve walked in on conversations about programming where people are very passionate about this language or that but really it seems they are expressing their love for something to make it seem like they really understand that thing. “What’s your favorite programming language?” Is a silly question for someone who’s only been in school for a couple years. The level of appreciation you have for something should come through deep understanding and time spent with that thing. I would love to more rigorously study things but it seems no one has the time and I’m too lazy to do so individually. I almost want to switch to a pure math degree where understanding is the main focus.
When I first used Copilot I was actually excited --- Because AI can essentially REPLACE a lot of janky abstractions that developers would shim and work around to enjoy the time savings of not writing it themselves. Ideally this should increase the vigor and energy of building sideways into new ways of code execution and developer thinking, vs. just building another layer overtop of the pre-established pile of software abstractions. Imagine a future where we just learn assembly and work with an AI to build the best assembly for our target systems, that's entirely possible. We could even create a bunch of fancy visualizations for 'grok'ing the code, but ultimately just be creating and massaging the simplest kind of code we can understand and the computer can interpret. This also opens us up to stop worrying (as much) about typing comfort, so more verbose but explicit languages like Zig could be less painful (once the training data includes all the recent updates to the language and standard library) I'd say we keep an open mind, and definitely worry less about building 'middle layers' that don't do anything except restrict programmers into stupid paradigms.
Awesome video, That's exactly what I'm thinking about for few loong years. I can see so much abstraction. And trying to find ways to reduce it but I can see so much resistance mainly in inevitable questioning status quo. Fundamental problem is that most of the people find thinking painful and prefer to offset thinking and architecture decision to "experts" and frameworks. Which drags them deeper into abstraction well. And there is just not enough of those who would be willing to change stuff fundamentally and go through little pain and experimentation to become stronger on the other end.
C# It has an intermediate language, you can use something like ILSpy to decompile it into source code. Doesn't always work, usually is impossible to recompile (very good example of the software crisis) but you can read the source code. And then edit the intermediate language, which is almost assembly and it will tell you right away how a source code for that would look like. Very useful for modding if you don't care about publishing.
As someone who works as a software developer I think this problem lies mostly on the sales side of the product. A lot of the times we have to make quick fixes that doesn't solve the underlying issue it just covers it only because there is "no budget" for the correct change, since it usually takes much more time than just covering it.
2:50 - I used to be that meme; but actually debugging comes a lot easier now. A pro tip is to know which LLM to use for what certain tasks. Kind-of like a carpenter will have multiple hammers - One for framing, one for general nailing, one for pounding stakes.
This one was close to my heart - if you enjoyed it, please consider sharing it with a colleague! I also am very close to finally releasing the beta of my git cheatsheet, subscribe here go get it: philomatics.com/git-cheatsheet
I've been really enjoying the more "out there" content that you have been recently posting to your channel! I haven't seen anything like this anywhere else (even outside of UA-cam), so to me it's been a blast. Keep at it :)
recursive feedback easily overfloods any datacenter, so ai could be quite good only for semi-automatics + we must keep in mind malware which can be too empowered w/ ai intrinsic flaws :)
I played with Google's ai while writing some code that uses opelssl... NONE of its recommendations worked but for almost impossible reasons to discover if you're not already super familiar with openssl or have the time and skills to build n debug INSIDE openssl libraries and not just link to them (i didn't do this of course but if i wasn't already familiar with the project i probably would have had to). I found it quicker to look at long depricated stack overflow answers as well as the flawed openssl documentation, and finally ... just read the darn source code (their documentation is so wrong and if their example code works... it's probably been depricated for a few years). But that's the problem with ai... it's only as good as the errors in the data used to train it... some knowledgeable competent human must write down the technical information before an ai can lean it... ai can't tell who's the expert if two statements contradict each other. I could glitch the ai into generating contradictory recommendations just by rephrasing a query. Ai may be useful somewhere in programming but I haven't seen a convincing example yet.
As a uni student, its refreshing to hear someone say the exact things im thinking in regards to AI.
I've been thinking about all of this abstraction and came to the conclusion that the best way to work around this problem might be to get AI to create its own system of "coding". We have defined many parameters to create things digitally, can an AI one day take those parameters and reimagine a way to use computers/electricity/hardware that we perhaps havent thought of? After all, we are expecting AI to make scientific break throughs - to what extent can the concept of computers be advanced and changed with their help?
Coding at its core is just a way for a human to tell a machine how to create something based on the physical attributes of the system. We taught the machine how to communicate with us to achieve a goal with coding, i think AI will reverse this relationship. The AI will tell humans how it should be created physically to accomodate the AI "coding language(what is the most efficient way to create or show all of the things that encompass computing/coding/art/video/games.. etc.)", and because we communication with language - thats how the AI will communicate with us.
If we get AI researchers i thing we start down a path of hyper optimising the virtual world, id say nothing is off limits.
I've been passionate about this topic for decades, and it's a breath of fresh air to see it discussed with so much sincerity and intelligence. For what it's worth, to my mind there is no doubt that the core issue is indeed managing complexity. I also agree with your point about the 'abstraction tower' and needing to be able to navigate between different levels of abstraction and ideally being bidirectional - well done - brilliant video
The "software crisis" is manufactured. Software isn't buggy because of developers or development practices, software is buggy because software companies value stock market tricks more than they value their own product. When you are hiring and firing developers every 2 years, you're not going to have good code. The average dev turnover rate needs to be 5 years or longer. You need devs who stick around for as long as 10 years for large products. Without this, the process of churning through devs will ruin the quality of your code base.
Incorrect.
You can use hundreds of software quality assurance tools to guide LLMs to writing high quality code.
If you follow very strict standards, you can have high quality code that is extremely easy to maintain.
The practice of ”write as much code as fast as possible" often leads to more than a few problems. What happened to learning how to do it right the first time even if it is a bit slower?
@@caLLLendar No you cannot, LLM can help you learn concepts, it can't effectively design or architect software. No amount of prompting will get it to do it correctly either, even if it excels at algorithms.
Microsoft paint: A few megabytes of hd space and ram, good basic features, startup time < 0.5 seconds. Microsoft whiteboard: a hundreds of MB of hd space and a gig of ram (it is just a web browser and web app) -- startup time > 8 seconds. Very few features.
This is the issue we have. Web apps are bad. Period. New microsoft outlook? A complete joke
@@caLLLendar Yeah, right. Standards and quality. You, my friend, have no real-world exposure. :) I've seen companies invest hundreds of thousands of dollars in something that no one needs and be unwilling or unable to fix critical bugs on their front pages. Standards and quality demands just lead to more bureaucracy and internal politics in large corporations.
There's a huge difference between a compiler and an AI code generator: the former is deterministic. Same input => same output. With AI, this guarantee is out of the window. Let's get ready for your code compiling differently on each run. Have fun!
tbf if you have the same seed then it is deterministic and compiles the exact same.
Yeah totally. It's kind of like a compilation step where we only ever save the compiled code.
I actually find AI generated code interesting when doing code review because the thought process that generated the code happened in the natural language space, not the programming language level. The code becomes harder to describe just like compiled output of a traditional compiler is harder to describe than the high level code used to produce it.
This isn't true, AI is deterministic. AI tools may not seem to be because they are programmed to use random seeds on every input, but that's not a required part of their design.
@@mstefanwalker3654 Yea facts. Litterally nobody understands shit. But it works so who tf cares.
Its even worse when you consider all the syntax slop modern languages like rust like to have like let
x = object.map().hash().convert().destroy(param1, param2).memoize()
Litterally just greenlights the AI into coding all cryptic.
@@mstefanwalker3654, actually there is no thought process and neither does it happen in natural language space. There is a tendency to anthropomorphize gigantic matrices with multipliers. I would compare an AI to an almost infinitely complicated pinball machine. The output is made of probabilistic resemblance of thought and language. Improving an AI is improving probabilities, never thought or language directly.
The "software crisis" is sales problem, not engineering. It's about how to arrange market forces so higher quality software is valued over quickly produced one.
Exactly that!
I disagree with your take. There is a quote by Dijkstra in the wikipedia entry about Software Crisis. That is original meaning.
Absolutely. Subscription model + rolling updates is the recipe for disaster.
They haven’t hired senior Web developers anymore, I had to keep myself afloat with freelancing for the past 3 years. Full stack Web developer since 2008.
@@albertoarmando6711 That original meaning is not really the issue today. The crisis with a almost evrything, not just software, is surface vs volume difference. You can only see surface details before you buy something while volume details are what you really want. Sales got too good with getting away with Potemkin villages and empty calories.
I have been programming since the late 1950s.
Regarding reading the output of compilers: there were many times when I was hired to fix bugs or add new features and the only thing the company had was the executable object code.
After examining a few bytes of object code (I learned to read and write object code directly without tools like a debugger for any of the commonly used processors) I could tell exactly which programming programming language had been used for the original source code and if there were multiple compilers available for that language and processor combination I could tell which compiler. In some cases I could even figure out which programmer had written which part of a compiler.
Writing in assembly language (or even writing in object code and hand entering it with front panel switches) was mostly about speeding up the running of the code or squeezing the size of code (so many times where the program was useless unless it would run in 128 or 256 bytes because of the cost of hardware part counts). That is bytes, not KB or MB or GB,
We even went to extremes of writing code specific to certain configurations of hardware and timing idiosyncrasies to get it to run faster, even coordinating with the hardware engineers to swap out capacitors and resistors and other parts to change the timing.
We even went to extremes of writing self modifying code so that the code could do many different things at different phases of the program life to squeeze it into a limited amount of memory.
One example would be a jump table (a fast way to implement the case structure or lookup table that was reused for different purposes during the life of the program. The memory addresses for the start of functions (jump table) or the data in memory (lookup tables) would be changed on the fly to reuse the same memory space for different things at different times.
Another example would be modifying the behavior of a function because multiple different functions had some of the same boiler plate and we could change the actual working code portion to save memory.
There were so many techniques for using self-modifying code, including what would now be called metaprogramming or reflection.
Wow, programming since the 50s, it's an honor that you watched my video! Thank you so much for your insightful comment, super interesting read!
@@philomatics I think you have a worthy goal. I've looked into that same thing in the past and not found an answer. I hope you find a solution.
Congratulations on your long experience. I thought my 45 years was a long time, and I'm still coding.
I remember looking at disassembled code in the 80s (In Circuit Emulators) and knowing exactly which source code produced it, and that was a huge project, 35 engineers. I remember thinking it might be the first sign of madness 😅
You are amazing, i am also a developer, but your level is WOW!
"A9 03" -- ah yes, 6502 code. I still remember watching that first Terminator movie and I was like "wait, the Terminator runs on a 6502-compatibe CPU, like the Apple 2 or the C64?"
Born in 1968, I can't compete with your level of expedience though. Still good enough to hear "we won't hire you, you're overqualified" far too often.
Yeah, AI making programs that waste ressources but are cheaper to make than manual programming seems reasonable to assume.
We are in a time where every app with 3 text fields and a button is a webapp that takes 400MB of RAM and 5% GPU for stuff that technically could have been a 4kb program by using the native api.
But that web app will run everywhere
@@LetrixAR And a trained monkey can write it.
We are in the era of fast pace softwares. And they suck, to be honest
@@LetrixAR No, it won't. And also it is not needed 'everywhere'
@@ScorgRus Yeah it's the Java syndrome, let's code in a shit environment using bad syntax and performance, arguably, just to have it "run anywhere", just in case we need it (like we don't know where we want it run in the first place)
The issue is not the loss of control, which can be mitigated with asm and ffi.
The issue is the compounding complexity with each layer.
Add the loss of knowledge and remove the pressure to ship good software and you go full speed into a wall.
AI will ship code that is way worse, checked by dev that never programmed more than few toys exemples.
In 5 years the dommage will be astronomical
No; Learn about companies like Snyk and SonarQube.
I am developing a platform that will combine hundreds of quality assurance tools that will be run to check the code generated by the LLM.
The tools will micromanage the LLM and the result will be much higher quality code than what humans currently write. It will also be completed in a shorter amount of time. The most tedious SDLCAs can be automated.
@@caLLLendar to be honest, it is not hard to write stuff better than the average programmer. And that is part of the real programming crisis.
@@caLLLendar Nice, you're going to be a billionaire soon, then, writing all possible software in no time.
100% this
Programming through communicating with an AI is like programming by drawing flowcharts. It seems like a great idea on the surface, but in reality it is only limited to very narrow domains which can be efficiently described that way (say, plugging together premade shader components in a game engine). For most other domains, flow charts are just awkward to use and a bad idea. That is also why domain-specific programming languages are invented and replace more general ones (let's say, SQL as the universal query language regardless of db implementation). But what all these tools have in common is that they balance power of expression with precision and performance required for a particular domain. Programming by telling an AI to fix something 100 times iteratively, is kinda like doing it with a developer and claiming that the 100 emails you sent them are the "source code" or "documentation". Whereas in reality they are just a messy history of your attempts at formulating what you want, quite inferior to an actual source code which captures the eventually resulting ideas in every imaginable way.
Taken from an even higher level of abstraction, software engineering is the task of converting business needs and business domain models into software models, and then into software implementation. Each transition step or abstraction in that process is leaky. By definition, the most possibly efficient thing is for the AI to go directly from business problem to solution, and for the AI to figure out all the details through generalization. Essentially, full automation. Everything else is necessarily inefficient
@@xt-89907 Not really. Business needs (and any other needs) do evolve over time, sometimes without much awareness from those who expressed the needs originally (they may even no longer be in business). It is not sufficient to translate them into a "solution" once. That solution has to be continuously adjusted to new needs, and the adjustments also have to be explained and justified to those who have those needs. Furthermore, long-term trust relationships and checks and balances in power must be established between the needy and their service providers (professional software engineers) to support that process of need fulfillment. This is possible with humans because we are, more or less, aware how humans operate and which goals they pursue. It would be hardly possible with a black box which does "something" which at first glance seems to satisfy your needs. Unless you want AI to literally become human and replace all human relationships, we are never going to achieve what you envision that way.
I think they are very useful for giving you syntax you don't remember fast, like how do I do check if a substring is in another string JavaScript? It's string.includes(substring), but I rarely remember that syntax, and it's much faster to ask an AI (specially one that answers fast) than to Google it, manually filter a good result, check the answer, not get distracted by the other crap around like comments about little niche things you don't care about (like it happens a lot in Stack Overflow)... But if you ask anything more involved, it risks send you on a loop of useless crap it can't get you out of
@@lowlink534 Yes, they are very useful as glorified canned search engines, trained to imitate and filter results from actual search engines. Ultimately, the usefulness of a ML algorithm boils down to two factors: the training data and the loss (or fit) function to be optimized, given the user input and the training data. Both are of course predefined by designers of the ML models. And in many cases it is as hard or more difficult to determine the scoring for an output as to directly spell out the algorithm which transforms the input into the output. ML thrives and astonishes people in cases where the scoring is easier and can be accomplished.
Dude. Seriously do you even pair program with claude. I’ve seen way more junk code from scratch by humans than ai. Ive been at it for 20 years you cannot even fathom the amount of beginner code ive had to wade. No one talks about the tide being raised. It’s all ‘im so much better than ai’ smh. Get real, no you’re not.
When AI uses a slower method it's not a "hallucination". When AI makes up a function call that doesn't exist, that's a hallucination.
I would argue that anything these LLMs generate is a hallucination. Not just when something is clearly factually false. Correctness can be more nuanced than just black and white. Which is exactly why it will be so hard to replace good engineers.
I looked up the word. An AI can never, by definition, have a hallucination, any more than an Excel spreadsheet or a pinball machine can have a hallucination. The user has a hallucination if he thinks (or when he thinks) that the AI can think or experience things.
@@nielskersic328 This is the silly semantical argument.
For AI use, Hallucination has a very specific meaning. It's like Theory in Science... You are essentially using "But it's just a theory!" level of argumentation here. You want to be one of those people?
@@seriouscat2231 This is the silly semantical argument.
For AI use, Hallucination has a very specific meaning. It's like Theory in Science... You are essentially using "But it's just a theory!" level of argumentation here. You want to be one of those people?
@@seriouscat2231 hallucination is a technical term in the field
Here's the thing: Abstraction is simplification. It is lossy compression. It is necessarily not reversible.
I think so too! Especially on the upcomming abstraction step, natural language to code - there is no way this would be reversible in a 1:1 mapping
If you don’t know what you are doing maybe.
@@mwwhited Sorry, but he's right.
Exactly. The entire point of abstraction is getting rid of the 'details' you don't 'need' to simplify things. Which is great, until you need them, which is exactly why all abstractions 'leak'.
@@mwwhited The opening chapter of Gareth Morgan's seminal business book Images of Organization says that the metaphors for business structure described therein, as abstractions of complex systems inherently obscure the true nature of what thet describe.
Business majors are famously clowned on for being crayon eaters, and they *still* know better than you on this subject. Thats just embarrassing. Im sorry for you
@9:30 I mean, C supports inline assembly, which I think is the expected way to manually patch the compiler output which automatically pushes forward those changes whenever you change the code elsewhere.
Yup stuff like that is really cool, it's like "portals" into the lower layer abstraction. But I think we need something even more fundamental than that (still researching what this could look like).
@@philomatics The problem I see with being able to "move up and down freely" like this is that you're extremely limited in your abstractions if you require that they be able to represent literally everything.
I think rather than trying to accomplish something like being able to just edit the assembly and have that reflected back in normal code, it may be more realistic to have something like the ability to tell your IDE/compiler to replace this function with its compiled version which you can then modify.
Less reversible compilation and more partial compilation. Same with stuff like being able to easily expand macros/generics in Rust.
A lot of the C that people write does not get turned into the assembly they might be intending as well. For example, ++i and i++ end up getting turned into the same assembly code after most modern compilers hit them because of compiler optimizations.
@@redoktopus3047 Why would people be "intending" a specific order of the assembly instructions?
Though there are far more egregious cases, e.g., the compiler entirely removing loops by solving recurrences and putting in a closed form instead.
@yaksher some people still use high level languages as a shorthand for assembly. I'm not one of them but it's not unheard of at low level stuff.
13:31 - Double agree. One of the saddest moments I think all of us devs go through is when you find the perfect function that solves a very difficult and niche problem your app has, but then realize that, like pulling on the threads of a tightly bundled ball of yarn, you would have to understand the entire monolithic host library to surgically extract all the highly interdependent supporting code the function requires.
I love well made single file header only C libraries for this reason .
Haha. Dude it’s all digital sandcastles…
@@leversofpower What time scale are we talking here ? Sometimes I wonder why I create when the sun is just going to destroy all of creation one day .
"you wanted a banana but what you got was a gorilla holding the banana and the entire jungle".
This is why software should be written to be reusable from the beginning. Extracting code written to be application specific after the fact rarely works out. But code can be written with reusability in mind which makes reuse very easy.
The business side will talk about quality and innovation but then demand the code be written in the shortest possible time - often too little time. Most managers don’t like hypotheticals, so they won’t think things through, and if you question their ever changing requirements they’ll just say you’re “getting into the weeds” or “not being a team player”. IMHO the problem isn’t the technical people - it’s the yes-man managers and clueless executives running wild with requirements and deadlines.
Unrealistic expectations + insufficient resources = poor quality software
THIS ❤
@Me__Myself__and__I exactly 💯
While I like thinking that abstractions many times reduce the level of control. There are some things that we need to clarify.
The whole notion of abstraction is to reduce the level of complexity of a program, not to increase it. When you need to open the black box due to bugs, it means either the absraction layer is not written correctly, or you did not take your time to understand it fully. Math uses abstractions in much of the same way that software does - The fundumental idea of abstraction is not flawed, but in fact what drives us forward.
Without the abstractions software development would be too slow or even impossible. An example would be that current day C compilers create much more optimized code than any human could ever write. Those compilers were written also in C - produced by compilers which were originally written in assembly. So in this case not only the abstraction did not reduce control, it allowed actual better performance.
Much of modern software development is about prototyping ideas fast, and optimizing when needed. These higher level abstractions allow fast prototyping, and you always have the option of rewriting a more custom optimized solution once you know which solutions are worth optimizing.
The problem arrises that inexperienced developers can start working fast and have some results but they might hit a wall that they cannot deal with because they are inexperienced in dealing with problems that might take a long time - such as messing with a foreign code base.
- ultimately I think everyone should choose which solutions should be custom, and which ones should be imported from a library. and a developer should know how to gain the knoweldge he lacks in domains he lacks experience in, once the need for that arrises.
While i aggree that you should have some understanding in writing lower level code, it is only in the cases you are absolutely certain you know what you want to make. Using modern UI libraries, or just the newest version of CSS available, allows you to explore a much higher space of possible designs and UX functionality than you would normally would be able to if you had to implement everything by yourself.
Opinionated abstractions such as Css overall reduce the mental tax on your mind rather than increase it, I do not think it is a fundumental issue with programming. While a two way interaction with the abstraction is a nice idea, it might not solve the modern issues - at least you did not present a compelling case. In fact - I think a perfect solution that would allow a two way paradigm which will bypass the issues we have with functions and function definition, such a paradigm will require its own abstraction that would reduce the amount of control you have over language/algorithems themselves - hence you reduce the amout of control you have over how you define the language in the first place - it may be worth it but it brings us full circle - creating an abstraction over the abstraction methodology just to solve the "fundumental issue" presented, only to prove that abstractions are sometimes necessary and disproving in the same time that abstractions which reduce control are bad.
All non trivial abstractions are leaky - complete reversibility either violates this or requires the abstraction to not be more abstract than the target.
Ie. The problem with reversibility as the constraint is that it means that both the high level and low level languages need to be able to describe the same things, and that necessarily means a horrifically complicated syntax and vocabulary for the high level language, since it must address many low level assembly dialects, which in turn vary according to their hardware.
It's pushing against what appear to be fundamental rules of information theory and thermodynamics, unless you relax the constraint and do things like introduce intermediate languages (ie even more abstraction).
Edit: To add, it's very common to encounter this exact problem in decompiled binaries. Not all machine instructions have a direct high level equivalent. The selection of specific machine instructions is made based on not only on what the high level abstraction described, but also on the target hardware, desired optimisations and other external constraints placed on the compiler. As a result, many decompiled functions end up with decompiler-specific language additions and inline assembly that are necessary to describe what the function does, since the meaning is ambiguous or untranslateable in the high level language. The thing is, this isn't a bug or an implementation limitation, it's a fundamental feature of universal programming languages and specific hardware targets.
What if you did introduce intermediate languages, as well as corresponding decompilers? So between any neighboring languages, you still have leaky abstractions, but the leakiness is kept to a minimum. You could make changes to lower-level code, and have the decompilers back-propagate the changes to the higher levels, except you could correct the results at any level.
You got it right here. There is one more problem with the reversibility, that lower levels have higher entropy. High level languages have guardrails that don't exist on the low level. Not every change you make on the lower level is going to find an equivalent on the higher level.
In a more tame scenario, a small change on the lower level, can generate a spaghetti explosion on the higher level.
having inlined code in object binaries is the most general counterexample to any faithful decompilation - it's done for very good performance reasons and renders the output opaque to "cutting" it into the original functions/procedures, especially as the number of compilation stages or intermediary representations grows - so equivalently editing the source and object code in both directions is in itself a worthless goal (as too many layers of abstraction exist between source and target for practically all relevant cases OR because it would infringe someone's rights on the source and binary forms, i.e. "one does not optimize shader binaries or proprietary compilers/libraries/applications by hand for redistribution")
it would also be useless in practice as the expressive power differs tremendously between almost all source languages and most flavors of machine code or ILs; there are very few situations where hand-picked optimizations would even matter, and even those do not leave a dent in software running expenses outside of e.g. datacenters or equivalent hardware -- and nontrivial optimizations, if strong enough, as a rule are opaque to decompilation (a practical form of this point is that hashing functions are "one-way"), and even worse is that runtimes (execution environment + machine state + thing to execute + any data input or output handled by the full system) are not exchangeable: the CPUs, the GPUs, the (say) JVM, the browser, the network, and the UI are never the same thing, and programming for each of them is severely constrained by what these environments allow or prohibit one to do (e.g. one would rather not use assembly to parse and transform text, or use JavaScript to change ring 0 state on the processor)
having a transparent build chain (any language + some compiler + platform-specific machine code) would require compiler and language and hardware design set back by more than half a century (as even hardware execution units have nasty issues with things that one takes for granted, e.g. synchronization, availability, locality, coherence etc. - and these are not (yet) available as robust or usable primitive notions in any higher-level language)
and the issues do not stay put within the "mere" realm of compilers (or languages) -- at runtime the execution itself has very few constraints that are actually properly known for any running program (including the whole operating system, firmware, and active hardware); barring the not-so-decreasing frequency of pure hardware faults and network/wiring failures which application software is never privy to, the software is assumed - not guaranteed - to work as intended practically all of the time (e.g. when did one last check how much available memory is available to an application? what if an allocation fails due to resource contention or takes too much time?)
just like the intention behind programming (e.g. I want to draw a circle on the screen) differs from the source code (I want X to happen to Y, i.e. "Y.X(parameters)", e.g. setting some pixels to some color -- but what's a color, what are pixels, and how can those be accessed?) the latter differs from the result (object code and executed machine code -- a level at which all is made out of integers/bits, and their interpretation is left fully to the hardware), and neither is precise enough to represent the reality during execution (as there are too many constraints to be aware of to guarantee complete correctness, too many possible states of a system to expect total reliability, and too many levels of abstraction from one end to the other to concoct a "slider" that continuously varies the "level of abstraction I'm looking at to make changes")
@@mihailamarie4318 But like with huffman coding, you can redistribute your entropic cost - a high level language could simply include more general, less abstract, lower level concepts by "isolating" them (like inline asm in C). This obviously isn't a perfect solution, but I find it likely that some method of combining different levels of abstraction would be significantly superior to both asm and current high level language paradigms.
The compiler is quite deterministic. It gets better and better with upgrades, and people are maintaining compilers to make it more efficient over time. LLM code has some randomness due to temperature. But if we remove the randomness, it still does not resolve the randomness nature of LLM because we don't have 100% control over how it generates the code.
You DO have 100% control.
Simply run many static software quality assurance tools on the code the LLM generates.
Use the output of the tools as a prompt for the LLM to fix the flaws in the code.
See how easy it is to combine deterministic tools with a non-deterministic LLM?
This sounds good on paper until the AI creates countless bugs it can't fix and recommends going back to the prior version
@@nousquest
Think outside the LLM-box you're in.
Instead, think about how you would create a system that automatically manages the LLM.
Roughly:
. prompt the LLM to guess
. immediately run thousands of tests to check the code
. use the warnings from the tests as prompts to improve the code
. ignore the bad code, loop and let the LLM try again
. save the good code by pushing it to a repository
If you like, I can show you a demo of this working. Otherwise, you can do a little bit of searching and see that there are plenty of companies and open source projects succeeding.
@@caLLLendar please link a demo, I would be interested in how the system you are describing would actually work
C is another example that allows to go down the abstraction. You can write assembly code directly in C, so, if you have something to optimize, you can do it
yes, but the point here is to allow and enable going back from assembly to equivalent C
which is not immediately trivial, but I can think of special cases where it's possible
I thought I was crazy, but apparently there are other people out there too that want programming systems where you can see multiple levels of abstraction at once. Good sign since I'm working on building something like that.
I actually had an idea like this: how can one view a codebase or part of the codebase on a macro (architect) and micro (dev) scale depending on the needs.
I hope your idea comes to fruition.
I'm working on a system to simply remove as much abstraction as possible within a program.
Think: Without the abstraction, you don't need to "see multiple levels of abstraction at once".
There will likely be a paradigm shift in programming; Abstraction may be discouraged.
The reason is because there are a LOT of benefits to reducing abstraction in the code.
@caLLLendar for-loops are a bit too abstract for my tastes, so now I do:
i := 0;
if i = list.Count then Exit;
list[i].Foo;
I++;
if i = list.Count then Exit;
list[i].Foo;
i++;
etc.
And of course, I try to avoid adding classes or functions, everything in Main(). as God intended.
@@PhilAndOr epic 😆
This sounds fascinating! I'm very curious what this entails. Are you comfortable privately sharing technical specifications? Concretely depicting abstractions is something I've been drawn to (looking at you, design patterns) and what you described sounds ideal!
Admittedly, I must step away for work before I could get through the rest.
I think if you're generating and maintaining code using an llm, then i think we need to treat the prompt and spec as the source code, and treat the programming language representation as an ephemeral artifact.
Not necessarily. The previous source code can be also input for the LLM, then you only need to specify the new or to-be-changed parts. The previous source code acts as part of the spec in this case.
Also it's impractical to regenerate whole modules by LLMs due to the flakiness. If you have working and well-tested code already, it's more productive to keep the working parts and generate more code and tests for new functions.
And when the LLM you`re using gets an update the results may be completely different from what you had even using the exact same prompt :)
@karolstopinski8350 possible. I think it comes to how specific you are. If it does something different that's not what you want, make the prompt more specific. Treat the prompt as a system specification.
Gosh, how I wish I could *actually* descend up and down abstraction layers. It has bothered me for years that I can't, and that the industry seems to be actively trying to delude itself into thinking lower levels don't exist.
Have you tried Forth? It seems strange that it wouldn't be mentioned in this video, being one of those classical programming languages which directly contradict the idea that there should be any separation between abstraction layers. It's maligned (with good reason) to be a difficult language to write and read, but it has an incredibly creative solution to the function problem: the base building block of the language isn't functions or expressions: it's words. Words directly map to some sequence of assembly instructions, and "function calls", if they can be called that, always map directly to some known set of assembly instructions. All of RAM is treated as a single block containing both code and data, and the interpreter is designed to execute directly from RAM through careful use of data headers and such.
Of course, that's all interesting in theory, but in practice, it's used almost nowhere because of how alien its interfaces are. "Standard" Forth doesn't have a natural way of handling text streams, but more importantly: it's practically impossible to "compile" in a meaningful sense. Since classic Forths rely on writing data directly into RAM, the Forth "interpreter" is essentially a just-in-time compiler. The closest to compilation that's available are "Forth images", snapshots of memory at known points in runtime... which contain interpreter code themselves, meaning that, say, a GNU Forth (Gforth) image *must* be GPL'ed, by design. The incompatibility with compiling runs deep. Calling C code involves statically building a wrapper library, and each implementation does it totally differently. It's actually *impossible* to write a classic Forth in ANSI C, on account of memory block management and register usage, and Gforth accomplishes it by using dirty inline assembly tricks to force register allocation, since GCC has totally ignored the register keyword for over a decade.
I've heard it said that Forth isn't the best programming language out there, but if there *was* a "best", it would look more like Forth than C. I think the branch of languages where code maps to known binary directly is unexplored.
This whole conversation seems a bit over my head, as I understand that you can make basic logical gates (and, or, latches, etc). What I don't understand is how higher level programming is essentially wrong in some way. Seems to be like if there is some data loss by higher level abstraction, that is simply due to a lack of understanding by the programmer?
In mechanical systems, you can use a power tool without understanding what the tool is doing. This person is more a mechanic than an engineer. Furthermore, things are not really understood until they are interacted with first hand. Hard to say anything about assembly until you've interacted with it first hand. This is akin to the allegory of the cave where people are learning from the shadows on the wall vs actually interacting with the electrical engineering.
An employer has a much higher demand for code mechanics than code engineers. Not every mechanic needs to know how a torque wrench works, just how to use one correctly.
In sum, I think that programming might have a lack of people with zero-abstraction knowledge bases (full truth understanding). It's true that these people exist (like people who write their own coding languages), it's just that code mechanics get a much higher ROI on their time investment.
@ You're absolutely right, there's nothing wrong with high level programming and being a "code mechanic", and you've identified correctly that our economy needs more mechanics than engineers. The issue is that tech employers have absolutely no clue what the difference is. It's infuriating. The issue that arises then is that our infrastructure (that is, the internet's) ends up being designed by mechanics rather than engineers.
As a concrete example: PHP, a programming language which was originally designed by a hobbyist for use in his "Personal Home Page". Its inventor said that an intentional design goal was to flaunt common engineering sensibilities and be "easy to use"... at the cost of being a nightmare to debug. As a result, in the early 90s, it was adopted by Facebook, Wordpress, and most online shops at the time. It remains the most popular platform for eCommerce today, and Facebook has yet to remove it from their system. Times are, thankfully, changing- thanks to understandings by bigger companies that there *is* a difference, and now agree that new technology that's meant to be used by others, going forward, should have some committee of engineers discussing it.
But smaller employers are still practically blind to the difference and hide "mechanic" jobs behind engineer level qualifications. Hence, my desire for a programming language that's more "honest" about the systems behind it, rather than hiding behind abstractions and insisting "Hey look, I'm just as good as programming at the low level! Anyone who says the low level matters? They just want to be paid more!"
you can totally go up and down abstraction layers if you want to, you can learn how the programming language and the compiler, assembly work if you want to, maybe bothersome to some but thats what makes programming fun to me
@@MommysGoodPuppy I decided to quit comp sci and go mechanical engineering. As much as I liked manipulating numbers and letters, I feel that mechanical processes are more interesting to build for me. I just could never think of something I actually wanted to build just using code.
@@williamtburt what are you building in mechanical engineering?
I think a big thing people are forgetting is that AI can reflect. The AI isn't just going to generate new code to replace the old one. It can look at the old code to predict what the inputs and outputs are going to be and then update it from there. Manual changes should be a lot more preservable here, or the AI can make its tweaks directly to those changes.
I wish we would take a more "start low, to learn how things work from the ground up, only then introduce more high level tools" approach to IT education.
In pretty much every metal related job, at least here in Germany, formal job education starts by giving you a block of iron, a file, and a bench vise, only. Your task for the first weeks is just "make this flat on one side", "now make this flat on another side", "now make sure the angle between these two sides is 90°", to give you a feel for the material you are working with. Only then you are gradually introduced to the various power tools, up to almost fully automated CNC machines eventually.
I think software engineering education should take the same "get a feel for the material first" approach ...
But maybe I'm just a boomer who had to learn these things that hard way back then, and now just envies the current young generation for the tools they have now while I slowly start to become too old to learn new tricks?
You are correct, and as I see it, this is how universities teach programming. But there are a ton of lower education courses that go straight to programming websites. So in those courses you essentially only learn the programming syntax and no real computer science.
I learned with 8086 that had LED lights to show the 8 bit word you instructed it to do, the abstraction layer were so called mnonics - you could advance the bus clock with a trigger, or switch to quarz clock to start the automatism. It helped tremendously doing this for the rest of my life. (Even if i dont do code anymore.) Its valuable to know how its done. I had to grind that metal block too, it sucked back then, but surly it teached.
@@DrBernon depends on the actual university unfortunately
@@hartmutholzgraefe Maybe so. 😅 At least in my case it was like that. We designed a RISC processor, built it in a simulator, and then made it run some code we wrote for it. Then they made us program an OS with multitasking, and also a Candy Crush clone for the Nintendo DS. All written on straight ARM. And that thing had a devilish architecture with 2 processors that you had to synchronize because one had access to the graphics chip, and the other had access to the buttons and audio.
Not to mention all the discrete math classes. We went pretty low level. And in fact they never taught any programming language, just programming in general. It was a given that you would learn the syntax by yourself to any language you had to use.
I agree. Same with any other basic education, let's say mathematics. You have to work your way up from 1+1 to integrals and differential over YEARS! Just giving a calculator to a 4th grader to type in some numbers and get a result is dead wrong! You need REAL understanding of the problem and principles! Even if you are going to use very advanced machines afterwards! Sure, we have to talk about some details, but this is not the problem.
There is nothing bad with abstraction, the core problem is the integration of the components. Programming is inconsistent, solving it is to make it consistent, and this is already done, but we call it by another name: math. A consistent program is equivalent to a mathematical proof, so the solution should be rebuilding all as mathematical proofs. This, indeed, is incredible more difficult: there are programs (theorems) that haven't been implemented for years nor their counterpart (that the program is not possible).
Math is useful, but it is not the universal answer to everything.
@@Me__Myself__and__I And it can't because such ability to answer everything, will imply it is inconsistent. Maybe a better name, instead of math, is logic.
@@danielmurcia6755 okay, good software and being a good software engineer/architect requires logic. Math is optional 99% of the time because the vast quantity of software systems only require rather basic math. Exceptions would be niche areas like science, academics, game engines, etc.
@@danielmurcia6755 this is the best interpretation so far, I think. It really comes down to the fact that abstraction should work fine because it's describing logic.
The problem with AI is, is that the AI doesn't know logic well enough yet, and how logic is fundamentally different from real systems and objects. Like how theoretical calculations lead to lesser actual yields, which create error margins, and then designing systems with the error margins accounted for.
Eventually it should be able to do these things. We'll see though
6:10 I actually kinda disagree? A lot of higher skilled programmers in the backend have likely looked at raw x86 or arm assembly. Even more rarely wrote some. So not your average javascript dev. This is a sneeze away from the punch card example brought up, just adjusted to the current day with the environment of compilers, +10k instruction large programs, modern pointer logic, etc...
Programming then and programming now is still just kinda about crafting a set of instructions the CPU executes. (or gpu or embedded chip, what ever it is) It's just the scale and the tools that have changed a lot. Before you would remember an instruction address so you can jump to it. Aka an if statement. Today we write if statements without a second thought.
I think this comes down to the fact that programmers think they are engineers, when they may be mechanics.
An engineer designs the torque wrench (high level function), and a mechanic uses it.
AI might not do a good job at programming because there simply isn't enough good data. If you understood how LLM's had a breakthrough because of more data more compute, you could extrapolate this to coding and thus its plausible that there just needs to be more good data for coding to become a solved problem.
For example, if you had to reinvent the car, but all you had to go off of were mechanic's service manuals, listing the torque specs for each bolt, you would be SOL. But the AI can still become a pretty good mechanic when it has enough information.
@@williamtburt Yeah I do agree but I feel like there are some fundamental challenges that need to be addressed and I don't see talked about a lot.
The current LLM technology is not a good fit for code or genuine intelligence. Something that is highly needed in this field. By LLM technology I mean the next token generation LLMs like chatgpt. Sure some emergent behavior can happen but AI testing has been incredibly flawed and is in my opinion at least to some degree investor fraud. Simply creating never seen before questions or adding irrelevant information in to a question will very significantly degrade behavior we perceive as reasoning.
So I am not convinced MORE training data is the way to go. Big companies already scraped the entirety of the internet at the expense of everyone and it is still not enough. At what point is it just beating a dead horse?
Secondly, a more vague problem. We can only rely on AI so much because it is done trough a natural language. The language we speak is short, highly contextual and most importantly vague. That vague part is important because code is EXTREMELY exact so when you try to translate something vague in to an exact set of instructions, you can only get out at best the most common, average result that fits what you described. While it may be correct, I do not wish to live in a world where everything is just kinda fine. Progress is made by putting in the effort and making a leap so impactful, the average of the past is no longer acceptable. This is basically the same problem as the collapse of training data where AI content is being fed back in to the training sets.
There are a few industries that still even require peer review of object code. I hear a lot of grumbling around the office when it comes up, but yeah, they have to review the complier output.
And just this last cycle, we had a bug come through that came down to the complier not doing _quite_ what you wanted, leading to crashes. It wasn't an actual compiler bug, but did depend on specific behavior that it didn't do.
@@neeneko was it a conflict between the language and the hardware
@@williamtburt I never got a clear answer since it was hardware, language, and compiler I was not familiar with.. but it was something to do with how the complier handled range checking and throwing hardware exceptions, with two functions which from a language perspective you would expect to behave the same, well, not doing the same thing... and some consumer of both functions that expected them to behave the same way when supplied with out of range data.'
If I’m understanding the argument right, it’s best to have abstractions, as they increase workflow speed, but maintain your ability to write and understand them yourself, with the ability to mentally climb up and down the ladder. I wholeheartedly agree with this. It’s kind of baffling to me that the norm is to just accept errors as normal. I had to patch a type in a form validation library we were using, and my coworkers were baffled that I just *edited the library code* and stored the patch in the project, but allowing errors on the master branch is not acceptable to me, because it devalues the ones that have larger impact.
We should be taking responsibility for the product we release, not the code we write
There are some interesting points in the video, but at the same time I think you're working a bit up from a conclusion, perhaps out of some ignorance for some details that are not immediately obvious. For one, abstractions aren't really meant to hide a layer below because it is more complex, in fact it is quite the opposite! Most low level languages are excruciatingly simple, but also very tedious, and some programming environments allow you to mix levels freely, which contradicts your main point! You can inline assembly in Rust, C and C++ and more system languages easy as pie! Rust allows you to switch from the managed memory context to the raw one with unsafe{} even! Even OSes allow you to directly talk to hardware if you so desire and the user sudo's your programs. You know why people keep dreaming of doing this but don't use those system in place except when they are maximally effective? Because acting low level is fun for 5 minutes and then becomes tedious and annoying. Most places where abstraction sucks are usually historic piece of garbage where you have stratification of some historic context and proper lack of standardization like the graphic pipeline, but this is not the fault of abstractions themselves.
Finally the crisis in programming is due to the industry culture of "move fast and break things" (like it spells right there. Doesn´t even mention abstractions, and indeed making *proper* abstractions is something against moving fast), alongside with the big money flying around the industry that require thousand of lines of codes shipped but doesn't have enough skilled programmers going around for it. Having thousand of code monkeys doing GUIs in react that needs to be shipped with short dev cycles will result in bad code quality, no matter how many abstractions you torture. Like if we had to build a new bridge every day over 10000 rivers you would run out of decent welders and you would see more accidents, no matter how many osha memorandums you sent out
Agreed. Abstraction is not the root cause.
The author should have paid more attention to the market dynamics.
@fluffy_tail4365 wrote,
"Having thousand of code monkeys doing GUIs in react that needs to be shipped with short dev cycles will result in bad code quality, no matter how many abstractions you torture."
Incorrect.
It is easy to implement hundreds of software quality assurance tools and use them to guide the LLM.
You can pass code written by humans to through such a system and automatically fix flaws and bugs.
have similar comment about "most fast and break things" tl;dr leads to us all in a sunk-cost fallacy boat with no personal time for higher and higher productivity requirements fewer and fewer benefit from.
@@TheNewton I agree!
"Move fast and break things" results in technical debt and daily stress.
YES PLEASE! I’d love to see a video on the research you had been doing . I love the way you explain things. My second favorite channel after fireship. But better in a way because it feels more personal.
As a SE, I’m happy people are talking out about this. My general points when explaining why AI is not going to solve everything are: garbage in garbage out(the basis of the models could have a large portion of junk in its learned data), the art of prompts(between two humans we can ask for the same program but if one knows actual programming theirs will be closer to usable code), the output is usually incomplete/broken(this is why copy and pasting results is a huge risk). I personally use it a lot to fill bits I’m not so great at, which actually helps me complete an idea but also understand what I don’t before. I would never and will never trust “make me a component that does x” and just drop it on and trust it.
I will bookmark this video so i can come back to it whenever i feel bad for wasting time at work and feeling like i could be replaced
I think a huge problem is the way many coders learn. There’s a just make it work mentality and who cares how. It’s gotten a bit better lately but I remember when looking up stuff for c# especially on stackexchange it seemed like all they did was scream premature optimization and just make it work no matter what. All that matters is how fast you can spit out a product and move onto the next.
The result of all that is a lot of people who largely only know how to call functions from some library with absolutely no understanding of how the computer works, what’s going on in the library, execution speed, or even error handling. Who cares about errors when you can just try/catch everything.
Well when that’s the mentality of many coders and corporations it’s unsurprising we have a lot of bad code. A big part of my education in school was working with assembly, c, building data structures from nothing, memory management, and cpu architecture. That made a big difference in my understanding of programs even when I’m using a higher level language.
You can also end up with a very insecure project with premature optimization. Especially if optimization is your main goal. Optimizations are often little hacks to utilize stuff better, and optimization for speed often neglects security.
And yeah, you kind of need the code relatively quickly. You cannot have every program that a company needs 2-3 years later down the line.
@ well my point is taking an extreme stance on either side leads to problems. We have such buggy and exploitable programs specifically because people took an extreme stance that all that mattered was how fast you could deliver a program. Sometimes taking the time to do it right saves time down the road when you don’t have to refactor or fix a billion problems and crashes. Of course you can take too extreme a stance in the other direction and take too long to get anything done.
There’s also the problem of the more you tell someone something is safe the more likely they are to do something unsafe. They’ll ignore warning signs because there’s so many safety nets. It’s very possible to write exploitable code in any language.
So when the goal is screw quality control and forget learning how a computer works at all just pump out code as fast as possible what you’ll get is low quality buggy and exploitable programs. The only solution to this is giving well trained programmers time to do the code right.
@@ryandodrill6904 Extremism, regardless of where, is terrible. It also comes down to the extreme OOP for example, often leads to terrible and awfully unreadable code.
And sure, it is very possible to write exploitable code in any language, but that regardless, writing programs requiring modern complexity in Assembly is very VERY!!! likely to lead to memory management.
The problem is that even if you know how a computer works, that doesn't save you from it, because there will always be the tiny error, hence why higher level languages tends towards being more safe overall.
I'd rather have a programmer who has no knowledge of the silicon what so ever, no real understanding of the CPU architecture, but aware of the pitfalls in programming that can lead to security exploits in a high level programming language, than I'd prefer a very educated programmer, that has learned all the intricacies, and tries to do the same in assembly. Very unlikely that the assembly coder will be able to produce safer code.
And yeah, sure, the mentality of pump out too quickly is bad, but the mentality of making way too secure is also bad, because it'll never be done within a reasonable time frame.
A little simplistic point here, but you can essentially fix any problem given enough time... but we do not have endless time. We will never have endless time... and regardless of how secure you make the system, the most vulnerable entry point will always be humans. It'll be that one Wi-Fi spot, that one procedure, that little loop-hope in security permissions, etc.
And as you mention in the first paragraph "Sometimes taking the time to do it right saves time down the road when you don’t have to refactor or fix a billion problems and crashes." is a great sentiment, until you hit the problem of scope, features, or other stuff coming in. The better option is frankly giving the programmers enough time to do their job at the end, when they have a functional product in many cases, because at that point, you know what it'll need to do, and stop feature requests at that time.
Otherwise you do end up with the development hell of constantly refactoring. Something that would take a month, takes half a year, or suddenly it takes a year, etc.
I think the "get it done" mentality is actually rooted in wisdom. A competent developer will choose the right algorithm or data structure by default when writing a function. This is different from optimization, I'm talking about basic things, such as using a set instead of a list (in Python terms). Optimization should always come after the correctness and completeness of the program is verified by testing (preferably automated tests). It's unwise to get a program 10% faster by optimizing if doesn't fulfill its purpose yet. Also programs are vulnerable to constant external change: new business requirements, new hardware, new programming languages. Software is called "soft" because of the fact that it can get obsolete extremely fast, so getting things done to be able to start using it is the highest priority. If the program is too slow to fulfill its purpose, then yeah, optimization is important, but it should be guided by benchmarking, not random little hacks.
I want to emphasize though that good initial architecture has a big impact on large software projects and reevaluating the architecture is important for growing software, but it's automated tests which support this continuous refactoring, not optimization.
I feel like it’s hard to say this paradigm is grounded in wisdom while also saying software is getting worse all the time.
Having fluency in a language is the exact skill which allows the fluent user to achieve mastery, unlike a non-native speaker. Consider the task of modeling 3d environments. A master modeler, familiar with various tools and techniques, can translate a text description into a beautiful scene. Ok, so can a clueless novice equipped with an AI, you might claim. But the reality is, no matter how much the novice keeps talking to the AI, they will not get the result at a level of a master. For the simple reason that they are lacking the "words" and "phrases" that are needed to formulate what they want to achieve or change, and if they try it, it will be in a dramatically more inefficient way than the master grabbing the 3d brush or material or light editor and adjusting a few faces in the exact location that needs adjusting. You could also say that the direct translation of "wanted quality" into "necessary action" already happens in the brain of the master, it is not externalized or communicated in any form, apart from the result of the (proper) action. And how do you want to train AIs to mimic this in domains where there is no such external written down training data? And why would anyone who values their skill even want to transfer it to an AI?
your describing the role of a manager, not a programmer.
I'm not dare to say that I achieved mastery in creating music, but I have spent a lot of time doing it, and this is my exact thoughts on AI generated music. It's great for AI covers and memes, but it will take infinite time to guide an AI to create a song you want. It's fast for people who lack the skills of a music producer, but for music producer it's slow and imprecise.
AI can mainly make things that are mediocre.
Currently, AI image generation is purposefully being stifled so that the TRL (Technology Readiness Level) is lowered. Even still, the market is trying to utilize AI image generation. It will spread like a virus.
I've been an artist before, and your brain always wishes it could bring the magnificent emotional detail from your mind to the art piece. this is the climax of art for the artist. finishing a piece is just work a lot of the times. this is why sketching is so popular and fun for the artist
@@williamtburt nono, the birds haven't been revealed as spy cameras yet so that the BRL (bird readiness level) is lowered. Even still, the people are trying to reveal birds as spy cameras. The knowledge will spread like a virus.
I've been a bird spy catcher before, and your brain always gives an emotional high from revealing a bird. This is what its like to climax for a bird spy catcher. Catching a bird is just a lot of work a lot of the times. This is why using security cameras ti catch the birds is so popular and fun for the catchers.
2:20 This is by far the greatest use for the current generation of AI imho. You cannot rely on these models to actually write code for you, nor should we want to. and they're also shit for learning entirely new concepts as they lack the structured approach real educational content can provide. But if you know the right questions to ask about something niche, can feed it data about something very specific, or are looking to break into a topic related to something you're already somewhat proficient in, they can be an insanely powerful starting point.
For me, the number of projects made possible or vastly more simple just by asking something "Are there any tools or libraries that do X and work with Y?" is amazing, So often, no knowing what you don't know is the biggest barrier to doing interesting things, and these models are some of the first tools I've ever found that can solve that problem an order of magnitude better than a search engine.
I wish these AI companies would focus more on what the models are already pretty good at instead of trying to make them replace in any every job that could exist.
Current generation of LLMS is: bad questions get bad answers.
With most people being incapable of realizing how bad their questions are; let alone not being misled by the "answers" that are generated wordsoup from the most complicated adlibs systems ever devised.
I can’t tell you how great it is to find someone else who sees this! I’ve not been able to dig as deep, but I’ve become convinced that abstraction has the unintended effect of _increasing_ complexity. This is super counter intuitive, completely antithetical to what they claim to do, and makes people I try to explain it to look at me like I’m nuts, but I’m convinced it is true.
That invisible barrier of abstraction is also what keeps commercial software protected.
This exploration of the merits of compilers leaves out assemblers as a concept. In low level assembly languages, you get 100% of the control that you got from writing directly in machine code. Every assembly statement maps directly to a known sequence of 1s and 0s. It is even entirely possible to write critical sections of your C or other high level language program in Assembly as long as you carefully maintain an API for the C and Assembly to communicate with. Rollercoaster Tycoon is a great example of commercial software written entirely in Assembly so that the developer did not give up any control at all at any point, yet got at least some of the benefits of a higher level language by using an assembler instead of a compiler. By adding some syntactic sugar on top of an assembly language (macros mostly) you can even get a lot more power at your fingertips in a custom assembly language or an assembly / high level hybrid dialect.
I think the real problem might be the hardware getting more complex, not the software.
They used to say "computers don't make mistakes" in old movies
AI is an abstraction of the computer
That's why it does mistakes
They were right. Computers do not make mistakes. The humans who made the training data for the AI algorithm do.
@@markgacoka9704 Yes and no. First of all, computers sometimes *do* make mistakes. From the FDIV-bug in the Pentium to random events like high energy particles or failing components due to aging/overheating (see the the Intel 13th/14th disaster).
Secondly, the AI algorithms themselves are flawed - they are basically stochastic parrots on Speed and would, even given perfect training data, only replicate (i.e. overfitting/recall) or still hallucinate plausible looking BS. They do *not* incorporate theorem provers or even just syntax checkers. This means that the algorithm itself is - by design - very error prone, too. It has to be, otherwise it wouldn't be able to generate novel solutions and just be a code snippet DB that can be queried using natural language.
Wow, you’re hitting right on something I’ve discovered while working on my software project. Most of my struggle has been determining what dependencies to use, and I’ve found that the decision has a lot more than just how much the bundle grows.
A big part for me is considering learning the library. For example, I decided not to use lowdash, mainly because it would expand the size of documentation to research when trying to solve a problem, which would actually slow me down.
I have to constantly keep my perfectionism in check so I can balance between an acceptable pace of progress without leading the project into a big ball of mud.
The problem created by abstraction is the same problem that is created by symbolism. Translating semantic content between languages still has linguists stumped too. We know all the phonemes: the way a mouth can make a sound, and we can diagram how they all fit together to make meaningful utterances in a given language, but we can't *translate* the meaning.
The bidirectional mapping between layers of abstraction was a really interesting idea. I enjoyed the video, especially the idea of replacing functions with some other fundamental building block.
imo, as you are talking about this is that Julia is actually pretty close to this. I don't think it'll happen with Julia simply because the architecture still relies on LLVM, and, at lowest, C API's to lower level. But the way it handles functions and packages allows a lot of interactivity through the entire library.
You unintentionally have the answer in your post, assuming you accept the universal reversibility constaint as correct.
LLVM IR, except with much more infrastructure in the VM for carrying out the operations in both directions, as both a compiler and decompiler, and some kind of requirement for extended debug symbol inclusion.
I don't accept the constraint as correct though, because it necessarily requires universality of language concepts, which it seems would result in really bad languages.
I love how this video gives me time to think for myself. When you finally got to explaining reversibility, I was pleased to discover that what you were talking about was exactly what I had been thinking about for a portion of the video. Thank you for inspiring me in this way (and making me feel smart).
Not is not that complicated to understand
Code or Compiler => deterministic system
AI => indeterministic system
> but but we should be chill to have shittier programs
> but but we should allow ourselves to continue in the decline trend
After 20 years in the industry, it is quite an awful take since we all refuse to be accountable for the decline of quality in software across the entire industry. We didn't had a REAL breakthrough in more than 20 years everything else added more bloat and mediocre solutions.
Most of the good solutions came from people programming ON THEIR SPARE TIME while big corpos are leveraging that code without contributing back.
Even your example at 13:33:
- Cross platform framework that we have sucks -> Nothing beats good old C/C++
- DSL always sucked
- ORM introduce such a bloated code and queries that you will be forced to re-write it anyway, at best use a code generation tool that way better
- Frontend frameworks are a distraction and bloated -> look what solidjs did with a very simple library OR look how fast we can be with just server-side rendering and jquery
- Most games these days sucks, big corpos are switching to UE because of incompetency in their teams -> using a game framework or game libraries is still better than a game engine
- Website builder are just the most awful thing ever for a real corporation, if you're a solopreneur yea sure it is good and WP works for most cases
It is not a "Low level vs High Level"
It's how can we build better technologies that enhance the human potential and it's creativity without creating unnecessary constraints
It’s all hype-driven nonsense in the software industry now. People aren’t learning programming out of a genuine passion for creating or innovating anymore; they’re chasing the latest job trends. And who’s behind this? The same big corporations that treat developers like disposable cogs in their profit machines. These companies flood the market with buzzwords, shilling Rust, C#, Swift, Java-whatever language lines their pockets best-and the masses lap it up like obedient sheep.
Most of the jobs they offer are glorified "web shitter" roles anyway, building yet another bloated, ad-infested app or a soulless SaaS product. But that’s not even the worst part. The new wave of developers, indoctrinated by corpo propaganda, parrot their talking points: “C is useless. C isn’t memory safe. C is old. C is bad.” What an absolute farce. They don’t see the bigger picture: C is the foundation, the backbone of everything they rely on, from operating systems to embedded devices. But no, big corpos can’t profit from a language that empowers individuals to think critically and build efficiently, so they smear it instead. It’s laughable. It’s tragic. It’s exactly what you’d expect from an industry hijacked by marketing and mediocrity.
When I started to code, I coded in C#. A pretty high-level language, and I managed to get visible results in Unity and WPF. Just after that it was necessary (or beneficial) to dig deeper into it, to understand underlying processes, to optimize functions, to pursue good project and code structures, to go low level and write code that accesses memory directly.
It doesn't help to dissuade people from coding by putting them in front of C and write menial and simple things which require a lot of effort. I remember what joy I had when I wrote my first program, a calculator in C#. Terrible code, but the end-result was quite good. I also remember when I managed to have 3D objects move in Unity. It was great to see your code actually *do* something. Well, today I work on a game that spans the observable universe in meters, and is riddled with systems and math that are of extreme difficulty (already done).
That's why my favorite language since around 1999 when I started programming, is C. It has just enough abstraction, but not too much. If only there were as many jobs for C developers as for Java, etc.
That! I don't understand when people put C as low level as almost assembly. It's so far easier to understand and write code than assembly but still very very good for speed that I don't see any reason for anyone to ask for more abstractions than this.
i think you are underestimating on how much abstraction gives C gives you, it is a LOT more over raw asm
the jump from asm to C is much bigger than C to the most insane boilerplate oop language out there.
@@khhnator Agree with that. That was the point. Assembly to C is big breath of fresh air. Code can be read as a high level algorithm. C to C++ is tiny. Sometimes I start a C++ project but a lot of my code looks like pure C. And creating too many classes and hierachies doesn't make it easier to read in my view. Sometimes it adds more complexity. C was enough for everytyhing.
C is fine for smaller programs, but extra features in C++ like namespaces, function overloading, class functions and class constructors/destructors, operator overloading, the STL is massively helpful in larger projects. I'm not a pro, but all the syntax rules in C++ often allows people to make code that is easily misunderstood. "const" for example is a very context sensitive keyword and often poorly explained to beginners. The C/C++ code styles with excessive C macros are a nightmare for new people who were only taught C++ and basic syntax rules for programming concepts only.
@JohnDoe11VII Yes, objective oriented programming is very helpful for large projects. The issue with C++ is that the language is already huge and keeps growing... C++11 was a useful step forward since C++98, but then the updates accelerated rapidly. There is now a new revision every 3 years. It's hard to keep up, and the language is becoming very complicated, if it wasn't already.
In order to fix some issues and to implement new features, last year I (partially) reverse engineered the firmware of synth. The insights you gain, when you try to decompile such a small piece of code are invaluable. There are so many inefficiencies, stupid conversions, bugs, code copying, security issues. Most of these, I assume, stem from using libraries, that in-turn use libraries, i.e. abstractions built on abstractions. Nobody ever looks at the end-result.
My fundamental problem with AI programming is being able to interpret the prompt as a programmer would. The AI doesn't push back and say "Actually that's not a secure pattern" or ask for more information it just "does it". More of my time is spent unfucking things that should not have been done, then it is doing things. It's also not determinative. Compilers will at least give you the same answer twice. That with the hallucinations makes me think I'll spend more time understanding what was just written by AI, versus writing it myself.
We can't get quantitative pushback in good LLM tools because people already scream to high heaven when companies tune the AI response to moderate it from becoming toxic.
They are being built for corporate midwestern fake helpful unhelpfulness.
Like a glossy tier0 support rep that fakes an empathetic: "oh that must be frustration".
As they have absolutely no real experience, or power, to actually solve the problem; or even at bare minimum recontextualize the problem into a better problem statement for someone _else_ to solve.
The support playbook end result is output that leans towards emotionally manipulating someone into feeling better about their NEW problems.
Yes.
I'm a QA and often biggest part of engineers job is to figure out what exactly client/management wants. Making exactly what is asked is a surefire way to get ungodly poor UX which often fails to resolve actual needs of users.
You need to figure out the intention and than offer back solutions. Current generative neural networks are incredibly bad an getting intention and maintaining context.
Exactly. Lack of determinism in AI programming is what will prevent it from being widely used. Even if we can get to a point that you can write whole huge programs just from a prompt i don`t think any banking system will allow for that. Users probably won`t be happy when their account balance is subject to hallucinations :D
This is very good. I see so many ignorant people (including myself) trying to make "simple" decisions with big consequences without proper analysis and trade-off awareness. Usually following current "industry standards" or previous experience while not taking into consideration project needs and specifics. Nobody talks or thinks about complexity, long term maintenance or control loss when using different libraries, abstractions etc.
This feels like a very academic view of programming. In engineering terms with stakeholders and deadlines, you will use whatever is the best tool available to make the thing work that you need. If your codebase is already written in a language, you will likely extend it in the same language, and if a black box library solves a problem, you will use it. Despite some idealistic views about how we "should" program, most people are just programming to complete a task, and so the black box method will continue to survive.
Very interesting video. One point on your diagram @5:45 : when AI generates code in a high level language, it is because it has data under this form to get trained on. If there was no high level code data, it won't be able to generate it. If AI was trained only on binary code data, it could probably do it pretty well, but we humans would have a lot of difficulty to undertand the output. So in a way, the AI should possibly trained on both, high level and low level code, so that they can translate inbetween both levels, similar to english-german translators. But ultimately it relates to human supervision, and implicitely the reason why we humans prefer high level code to binary opcodes is because it is easier to supervise (aka control?) at scale than binary opcodes at scale. If AI does not need humans in the loop, then it can go full binary code, but then who would write the prompts...?
Please stay on this train of thought. You are dangerously close to a very important realization.
?
??
Interesting video, well done!
Couple of observations:
- Compilers are deterministic, while LLMs of today have a random component (temperature setting).
- We can write sections of ASM code in our C programs (either by linking them or by using a pragma section, depending on the compiler), so we can still have control over the critical parts even with a "one-way" development flow. Same thing for Python and NodeJS if you go through the trouble of writing a native module, which is a bit more work
Software guy with 24 years of experience here. Started coding with C/C++, then C#, JavaScript and Python. Now I code in English too :)
Enjoyed watching this! Made me look at abstraction from a completely different perspective.
As a x64 assembly programmer expert in vectorization, my time to shine has finally come.
I don't agree, the vast majority of problems are how to get your programs logic right. Better tools should ideally give you more time for the analysis and verification, but in practice are just used to churn out more code at a faster rate. "We'll fix the bugs later...Yeah, right...." AI will make this worse. The bigger problem is the jenga tree of unmanagable dependencies. To solve a meaningful problem you need to manage the abstractions yourself, else you're borrowing someone elses abstractions. It's a bit like trying navigating Paris with a city map of London.
Learn how to use quality assurance tools to guide the LLM.
The result is much higher quality code than humans write.
I work on a PhD whose main message is how security issues in hardware break down abstractions and that's why we can't easily compile security countermeasures into programs. So this is close to my heart as well-I spend most of my time pondering such questions.
I think the expectation of a one-to-one mapping through abstraction is a red herring: it fundamentally contradicts the point of abstraction. Connecting the lower levels back to the higher levels already exists in the form of debug information, and we already know how this goes: badly. If you higher-level language captured all the subtlety of the lower-level one it wouldn't have the benefits that make higher-level languages interesting in the first place.
Creating a connection between levels, though, *that* is a different piece of meat that I couldn't agree with more. We already have some of that, too: with ABIs and FFIs. I can write an assembly function in a C program to have performance-critical code in assembly and performance-flexible code in C. Which brings me to a pet peeve of mine: expanding a function call into a function body is not a drop in abstraction level; you do that without changing languages. Functions are by far the most successful abstraction in programming, but they don't constitute a lowering. It's the opposite, in fact: it's the one abstraction so successful that we implement it in every language. Which makes it the perfect bridge for connecting C code and assembly code and code written in other languages with other abstractions.
I'm willing to take a gamble and claim that we already have the tools to fulfill your vision: we know how to make binary interfaces to connect pieces of code from different languages (we just need to standardize them), and we have the function interface as a near-universal abstraction to give semantic meaning to that interface.
"Anymore"? I started programming in 1984 and I don't feel that there were less bugs then.. I believe that you have a romantic view of the past, maybe just to make a point?
Now I do agree that AI does not solve that issue: it just repeats the same errors, since it learns from existing software well known for being buggy..
These were early days - check how in 90s and early 2000s everything "Just worked" without Product Management team constantly messing features back and forth and teams compromised mostly of technical people.
@@piotrd.4850 no, it wasn't. I still remember those times. I already had a PC in the 00s, and every time I bought a game or software, I prayed to all gods that it would boot up on my PC. Because sometimes, it just wouldn’t. And that was it. No patches, no internet, no forums, just returning the CD or floppy disk back to the store.
It's just people tend to forget things and wear nostalgia tinted glasses.
@@alexm9104 oversimplifying is rose tinted, but they have a point regardless. I think what has really happened is that the ratio of well built software and rapidly built cheap stuff has changed. There have always been such types, but the later was generally relegated to games or low cost apps, while stuff that cost real money to by also cost real money to make.
I think what has happened is that the later, cheap fast software, has not only become the norm, but the expectation, so all the companies that used to invest in development are under pressure to adopt the 'fast' model.
I work at a company that makes avionics, so the kind of software that really can not afford mistakes. our new VP apparently had a stint at Google and believes those are the deisng practices we should be using... fast sprints, fast testing, AI assistants, etc. Our quality has plummeted, ...
great video! it's funny, in your last video on programming as theory-building got me thinking about AI - one of the tradeoffs of having LLMs writing our code is that no one has the theory of the program in their head (unless you familiarize yourself with the AI generated code enough you might as well have written it yourself). looking forward to more videos on this theme!
You can migrate assembly in C and C++, because you can write Assembly in those languages. That's like one of the major selling points. There's still hickups but not your "barrier". I worked with such libraries in those languages.
First time here and I really enjoyed the video, subscribed instantly. I'm not too hopeful about the last philosophical portion about learning something in programming and transferring the knowledge elsewhere - we have a very bad track record of using the things we've learned on managing complexity so far.
Google fires real employees to put their AI to review my app, now I am getting rejected with generic responses. :(
Yea this is doing down hill
@@MelroyvandenBerg Wait till that shit start reviewing loan applications, healthcare procedures, and government regulations. Will be shitshow, we will need years to fix the reversability problem.
@@quagengineer1877 look into UnitedHealthcare and their AI related lawsuit. seems like it's already happening
With C/C++, you can use inline assembly code. With .NET's C#, you can use MSIL via low level runtime methods.
... AI is a marketing term that means nothing. There is no definition that everyone agrees on.
And machine learning... is not really what these AI pruducts is...
The large language model is simply a fuzzy API to use tools we already have, like compiler errors and static analysis.
Like, here is a fact: Largw language models cannot do math. They are terrible at it. Because you cannot do word prediction so well that you can do math. It can never work.
But chatGBT can do math! Yes... programs, that are NOT large language models, tries their best to spot math question, and then it uses a normal calculator to do that math.
Meaning chatGBT is just a REALLY environmentally unfriendly API.
And that is what Google and Windows wants. Fuzzy API's.
Because how are you going to compare or replace programs that you do not know what is?
You will be forced to use the Windows or Google AI.
And just like always, they are currently working to change the law so no one else is allowed to make those large models.
This is great for them. Monopolies that cannot be challenged, even via law. No competition!
And once the user cannot leave, you can treat them as bad as you want it.
We are ignoring here that the models are build with stolen code.
The split between library and your own code sounds like the effect of matching the organizational structure: the libraries are written by domain experts who communicate via documentation.
10:08
I think it is easy and comfortable to sit in a chair and criticize the software industry as problematic because it "don't start from low level", but they never really showed any concrete evidences on their conclusions, it is just yapping for the most part.
Abstraction is like putting a mountain between you and a low level concept, you can build a rail, or a road to go over that Mountain but it still has its own problems
Abstractions are probably one of the biggest problems in software. Developers create abstractions of problems they don’t fully understand. It makes them feel good. It’s easier than investing the time to understand the problem space. Spend less time abstracting and more time writing code and talking to people.
0:25 if we accept the definition on the screen ("Software crisis is a term used in the early days of computing science for the difficulty of writing useful and efficient computer programs in the required time."), then it's obvious what causes the "crisis". Since we include the word TIME in the definition, there lies the problem. This is artificial. It's not that we could not write "useful and efficient programs", it is that we add an artificial constraint, which is - more often than not - added by non-programmers, like managers, the board, etc. They are obsessed about the 80/20 principle for example. I can just hope they don't really dare to use it to program nuclear reactors and such.
This is an interesting video but considering AI something that could ever be an abstraction in the first place is strange to me. The stuff we have right now is inherently random, with no way to turn that off? And even if we could, the slightest change in the prompt text could affect the result substantially. In the same way that a senior dev telling a junior dev to do something is not abstraction, AI can’t be abstraction either.
I don’t really have much to comment on the rest of this though. I’m in the position of using typescript primarily and that only transpiles down to js, which is already human readable, and sourcemapping makes debugging easier, even if the js gets uglified. I very rarely if ever need to look at that layer. Maybe things are different in other languages though?
The one note I have about abstractions via a compiled language is that oftentimes the abstraction language just has different goals in mind. Usually simplification. If you could go both ways the abstraction language would need to support literally every bit of syntax in the base language in its own way, which in many cases sounds pretty pointless. It’s like you have an abstracted language with gc compiling to a language without gc where you have to manually free memory. It would be _intended_ to not have the syntax to manually free in the abstracted language, but it would be _required_ if it’s two-way. Any abstraction in your paradigm would therefore just be syntactic changes and well-known code being turned into what’s functionally macros via abstraction syntax, and it would otherwise be a superset. Abstractions would be far less useful, no? Maybe I’m not quite understanding though
"In the same way that a senior dev telling a junior dev to do something is not abstraction, AI can’t be abstraction either. "
BRILLIANT
Sure abstractions can be non-deterministic, it's just usually not stochastic. A non-stochastic abstraction can still be non-deterministic in the way that it must account for all reasonable implementations of the interface of the module that has been abstracted over, such as the final implementation of your program on all its viable hardware targets. The final deployed machine code is not determined by the programmer, but they have at least specified some constraints on the behaviour of the final system, so what's different when the final system can be stochastic but still stays within the boundaries of those constraints?
@@MagicGonads While I agree with what you're saying, I think my core argument is kinda unrelated. I just don't think language for communication could be an abstraction. There's just too many ways to say things, and too many ways to interpret each of those different ways.
In the case of LLMs I think it's even worse. And I don't think all the training data in the world could fix that, either. Even if there's an LLM you can turn randomness/fuzz down to 0% on, I think the sheer variance in possible outputs you could have just by changing your phrasing a bit is enough that you need constraints expressed in ways other than english language just to verify it's doing what you want. In which case you're desperately using a leaky wooden bucket, covering over all the holes you can spot with duct tape, just hoping you've caught them all. You basically need to have a language with rules so there's no possible confusion of intent, and to have the LLM be trained on that instead. Why aren't we just using a normal programming language?
And none of this is even taking into account AI alignment as a concept! It's a black box, who knows if the weights interpret things the way we do. Who knows if there's hidden stuff in those weights? If AI generated code includes a backdoor is it a valid abstraction to you?
I guess it comes down to whether you believe something is still an abstraction if using it causes it to fail for no discernible reason sometimes
@@ChiriVulpes You're close!
I developed a free open source platform that is designed to:
. Convert pseudocode to whatever language you like (ABSTRACTION)
. Use extremely strict quality assurance tools to scan the code for flaws.
. Automatically guide a human developer or LLM to fix all of the flaws.
. Automatically run reports on the entire code base and improve it.
. Automatically write and run unit, mutation, and integration tests.
The result is much higher stability, security, simplicity, and speed.
@ChiriVulpes wrote: "You basically need to have a language with rules so there's no possible confusion of intent, and to have the LLM be trained on that instead."
Incorrect.
The code just needs to be able to pass very strict tests.
Intent is proven with unit, mutation, and integration tests.
@ChiriVulpes wrote: "Why aren't we just using a normal programming language?"
Pseudocode is much easier to read by NON-programmers, Quality assurance tools, and LLMs.
Write pseudocode to CLEARLY COMMUNICATE INTENT to non-programmers, QA tools and LLMs.
Use a platform to convert the pseudocode to real code and fix all the flaws in the code.
You can also use code written in "a normal programming language" as the prompt to the LLM.
It is more challenging to refactor code written by experienced developers to a more simple style.
Developers often use OOP and other techniques that make the code harder to comprehend.
The programming paradigm will shift when developers revert back to writing lots of boilerplate.
Procedural boilerplate code encapsulated in < 3000 token sized functions is much more clear.
Correctly structuring the code while avoiding abstractions allows us to automate quality assurance.
Absolutely Nailed it. Both the description of the problem and a great potential solution.
Pointing to the stars yes. Nailed it? Not quite but salute the thinking and the nice presentation.
There is an important difference between current AI and compilers. Compilers are determenistic, and it is possilbe to explain what exactly each part of a compiler does and why is it necessary. With current AI, we do not know what will it exactly do to the code. There is some interpretability research, but it's just not enough (yet?). Will this problem prevent "good enough" code or not - that's a separate question though
Dude you mix my favorite subjects, abstractions and philosophy awesome video!!!
The "2 hours coding + 6 hour debugging" vs "5 min ChatGPT + 24 hours debugging" is simply not right. If you know how to use AI, it's more like "2 hours failing to role your own + 6 hours researching hot to do it right + 2 hours implementing it + 1 hour making it compile and "work" + 12 hours debugging it" vs "30 minutes AI + 1 hour code adjustments + 1 hour unit tests and it works".
or just use a search engine and get the original stack overflow answers with code the AI screws up by trying to combine several answers instead of just being able to find the correct one for your case that likely has the most votes.
The non-AI case only applies if you learn nothing from project to project. If you're working intentionally and not just copy+pasting other people's code, you'll learn how to execute a project with minimal or no research at all.
@@michaelcummings7246 Laughable. Stackoverflow is full of crap. And the most voted answer is usually "Use boost" followed by "you're doing it wrong!!!"
@@zoepentaleri If you can "execute a project with minimal or no research at all" you're probably a web-developer code monkey, not an actual software engineer.
@@zoepentaleri so using AI isn't doing the same thing? It is just hiding the same thing behind the latest shiny.🙄
This fatal flaw is what essayist Peter Welch was talking about in his essay "programming sucks". It is the same flaw that prompted Bill Bryson to make his famous "They are, in short, a perfect match" observation. It is this flaw which I devote my life to, if not eradicating, at least mitigating.
To put it bluntly: If software I use does not meet its contract for usability, it will not be used for much longer.
I love this topic but almost never find someone that has something truly interesting to say about it. But this video was different. Good thinking :)
Thank you for the clear explanation. I had a thought about loss of control once, I appreciate your detailed analysis.
Something that comes to mind immediately is the idea of training and LLM to generate code on a byte by byte level. Impractical today, but like you said, if if the speed and volume of these models increase, it could theoretically solve this one to one problem.
Thoughtful take. And even assembly itself is more abstract today (virtual registers, micro-programming etc).
To be fair they were probably right about the performance of code produced by early compilers. I mean just the fact that "optimising compiler" used to be a term that was in use, but now people just say "compiler", because the base assumption is that it can do a lot of optimisation, should tell you something about the optimisation level of code produced by early compilers.
I like your thoughts about working on all levels of abstractions (16:00 etc). However, there already is a simple solution for that, which is also widely adopted: Dependency Inversion.
A simple example: My functionality depends on an interface, that abstracts all complexity. There is a "standard implementation" (think about the library from your earlier examples), that implements this interface and that is used throughout the code. Now this is what I would usually don't touch (the equivalent to using the compiler standard output). But in certain cases, I need to override this module. This is where I would inject low-level code (or even: optimized byte-code) into my module that accesses it via the same abstraction.
We also do this in UI, where standard components that render some buttons have "slots", where we can inject our custom implementations in case we need more control over the "header", "footer", "action buttons" or "body" of a certain element.
Very simple, highly effective. And it allows you to ignore the lower level if you want to, but you can always jump to the level, when you need it.
i don't know about other programming languages, but this is basically python. You can take any function/class and add onto their functionality for a custom implementation (or completely override it)
in terms of UI at least for browsers and other things a user would enjoy. I've often felt that allowing us some level of control would be really helpful.
hum, While a user could redownload their software if they screw something up it'll be neat to have some sort of "Revert to Default Conditions" button too.
Like what a motherboard has in their BIOS.
Also also. Thanks for not making this click bait. Your video was well put togther and well thoguht out.
Count me as a new subscriber.
Your comment about looking for reversibility in abstractions reminded me of two things: reversible computing, and how a dual object is represented differently in lambda calculus and in sequent calculus. I'm not certain that the category theory concept of a dual object is exactly what you're looking for, but it may be in that direction. A dual object in a Hindley-Milner type system (based on the lambda calculus) may exist for a specific abstraction, but they are generally non-trivial. A dual object in a Gentzen type system (based on the sequent calculus) may exist, and the logical inversions are generally structurally symmetrical. To my knowledge, this type of reversibility cannot be generalized, see cellular automata such as Conway's game of life, but would likely help in the expression of these reversible abstractions.
Wow, that was very refreshing, it's the first time I've agreed with essentially everything a creator has said.
I think reversible abstractions are only currently out of reach for performance reasons, with enough compute it should be possible to alter every high level code character (individually) in every possible, or maybe conceivable way, and observe the resulting low level code. Those changes could provide an extremely large amount of minutia to train a potentially revolutionary LLM. Reversible abstractions. There's no reason I can think of why every abstraction level could not cascade together to allow prompt to compiled code via any number of interchangeable language abstractions - with editing possible at any level, in any language.
I think AI will surprise us in ways we can cannot possibly understand without layers of abstraction. Reversible abstraction in this realm might unlock a far deeper comprehension and mutually beneficial communication.
Really good video thanks :)
But that sacrifices the readability of the high level code, as it can't be designed to express the details of the low level adjustment natively
@@MagicGonads Isn't that just another level of abstraction?
If the model is trained on high level, readable code and how alterations of that code affect the low level code, we know that AI can run anything in reverse from language to diffusion. Essentially changing the correct single low level bit would generate the high level code - including code comments.
At the rate of current advancement, 5-10 years seems like a reasonable expectation timeline.
@@74Gee the high level code lacks the syntax to specify the low level semantics, no matter what, and even if you find a sly way to trick the compiler it's gonna be an unstable hack
@@MagicGonads Yes I agree it lacks the syntax, however given the correlation a trillion minuscule source code edits and the resulting binary, and the notion of how AI can correlate the sound of a swiping finger to a fingerprint, or the flickering of a door entry LED to a code, or a prompt plus noise to an image, do you think that the reverse correlation of bit swap to source is always going to be impossible?
I don't.
@74Gee not impossible but unintelligible and unstable
Neither abstractions nor low level will get away with the problem: "Complexity". You said it in the video, humans love to simplify.
But you have to keep in mind how it took for us to trust compilers enough to actually use them on a daily basis!
Nintendo still prefered to code in ASM most of the time even in the N64 era. They turned off compiler optimizations because they didn't trust it and rightfully so, since much of the early code produced had bugs introduced by the compiler and not the logic. This becomes even more frustrating if you want to use less-deterministic software like AI.
Debugging an AI could be a nightmare even for compiler programmers and everyone who took a step into this field knows what kind of nightmare compilers already are :D
The N64 is a weird system. The bottlenecks in the N64 made it so that normal all purpose compiler optimizations could create slower code on the console. The N64 also had expensive carts, so getting code as small as possible for smaller ROM chips was more valuable than slightly faster code, compiler optimizations can dramatically expand the program size. Nintendo also wasn't very experienced (like most) at the time in a 3D space, huge performance issues and wonky collision (tick tock clock the worst map) exists in Mario 64 indicating inexperience, rushed design, and a lack of communication between creators.
It is so great to expose what one has in mind on a public "AI" website. For the website operators.
This video was awsome ! I never let comment useally but you deserve this one. I hope your solution will help the research.
Is not a fundamental "flaw" of programming. It's a defining characteristic. Even though I don't like the guy, the famous quote of Grady Booch is true: ""The entire history of software engineering is one of rising levels of abstraction". AI is adding yet another level
Another effect of increasingly capable tools and abstractions is with the increasing speed and lowering cost of iteration (which is obviously good), we become ever more tempted to think we can get away with not fully understanding the problem we're trying to solve, because we can "just iterate our way to it," (while possibly planting the seeds of catastrophic failure.) A positive effect of iteration friction is that it makes it increasingly profitable to take the time to make sure we understand the problem at hand first. Similarly to how high interest rates tend to cull less-disciplined real estate deals and business formation, and expensive recording studio time used to cull unoriginal music recordings.
There's a perfect fix for this.
Give testers the time they've been requesting for decades.
Really enjoyed these topics and your coverage of them, excellent video! I like your idea of “two-way” abstractions, allowing software engineers to cognitively navigate from top to bottom of a multilevel abstraction architecture. I’ve had a similar idea, but I was unable to come up with abstractions that can easily organized like that. The problem lies in that between some levels of abstraction, the translation between the two is not algorithmic, but heuristic. For example, if I have a system comprised of interacting state machines, and I wish to instead implement them with a petri nets to increase concurrency and distribution, the conversion/refactor will be heuristic, since there will be different solutions to the most effective way to implement the same behavior as the state machines. Maybe that’s not the best example, it’s just relevant to the levels of abstraction in the architecture i’ve been envisioning.
Your example of the user interface editor that directly updates the styles applied in the browser does show your concept working, but the abstraction barriers between the color of a UI element and the actual variable that sets it is probably one of the most direct abstraction boundaries to cross.
I’ve always been someone who cares about the how and why of things. I’m in school studying data science and I’ve noticed that the farther I get into my degree the less people care. There’s almost a resistance to understanding among students and professors. Don’t ask questions that challenge someone else’s understanding. Just copy paste and act like you know what’s going on. I’ve walked in on conversations about programming where people are very passionate about this language or that but really it seems they are expressing their love for something to make it seem like they really understand that thing. “What’s your favorite programming language?” Is a silly question for someone who’s only been in school for a couple years. The level of appreciation you have for something should come through deep understanding and time spent with that thing. I would love to more rigorously study things but it seems no one has the time and I’m too lazy to do so individually. I almost want to switch to a pure math degree where understanding is the main focus.
When I first used Copilot I was actually excited ---
Because AI can essentially REPLACE a lot of janky abstractions that developers would shim and work around to enjoy the time savings of not writing it themselves.
Ideally this should increase the vigor and energy of building sideways into new ways of code execution and developer thinking, vs. just building another layer overtop of the pre-established pile of software abstractions.
Imagine a future where we just learn assembly and work with an AI to build the best assembly for our target systems, that's entirely possible. We could even create a bunch of fancy visualizations for 'grok'ing the code, but ultimately just be creating and massaging the simplest kind of code we can understand and the computer can interpret.
This also opens us up to stop worrying (as much) about typing comfort, so more verbose but explicit languages like Zig could be less painful (once the training data includes all the recent updates to the language and standard library)
I'd say we keep an open mind, and definitely worry less about building 'middle layers' that don't do anything except restrict programmers into stupid paradigms.
Awesome video, That's exactly what I'm thinking about for few loong years. I can see so much abstraction. And trying to find ways to reduce it but I can see so much resistance mainly in inevitable questioning status quo.
Fundamental problem is that most of the people find thinking painful and prefer to offset thinking and architecture decision to "experts" and frameworks. Which drags them deeper into abstraction well.
And there is just not enough of those who would be willing to change stuff fundamentally and go through little pain and experimentation to become stronger on the other end.
You got me there with the last philosophical bit. Very interesting thought !
I really like these higher-level (no pun intended) ruminations about the industry. Well argued, solid video!
C#
It has an intermediate language, you can use something like ILSpy to decompile it into source code. Doesn't always work, usually is impossible to recompile (very good example of the software crisis) but you can read the source code. And then edit the intermediate language, which is almost assembly and it will tell you right away how a source code for that would look like. Very useful for modding if you don't care about publishing.
I hope AI finds the right way to map classes to tables.
It’s great at pointing you to the docs or get a ballpark when you can’t name a certain thing you want to do.
As someone who works as a software developer I think this problem lies mostly on the sales side of the product. A lot of the times we have to make quick fixes that doesn't solve the underlying issue it just covers it only because there is "no budget" for the correct change, since it usually takes much more time than just covering it.
2:50 - I used to be that meme; but actually debugging comes a lot easier now. A pro tip is to know which LLM to use for what certain tasks. Kind-of like a carpenter will have multiple hammers - One for framing, one for general nailing, one for pounding stakes.