The Devin "AI Software Engineer" Scam
Вставка
- Опубліковано 16 кві 2024
- Is Devin AI a complete scam? Or is it a revolutionary AI powered autonomous software engineer? Or somewhere in between?
Original Devin AI Video:
• Devin's Upwork Side Hu...
@InternetOfBugs Video:
• Debunking Devin: "Firs...
@ThePrimeTimeagen Video:
• Is Devin A Scam?
Reddit Post:
/ kujyidr
Prepping for your frontend interviews? Use code "conner" for a discount on my product FrontendExpert:
www.frontendexpert.io/conner
🎬 TikTok: / connerardman
💼 LinkedIn: / connerardman
💻 Video/Coding Gear (affiliate): www.amazon.com/shop/connerardman
Business/brands 👉 youtube@connerardman.com - Наука та технологія
Shoutout to Internet of Bugs!
all i see is a kid listening to a grown-up how to understand that some videos aren't what we expect.
Scott Wu's last company, Lunchclub was also an "AI matchmaker". It raised 24 million in its Series A. 2 years later he left and it never scaled. I heard rumours there was no AI in it at all. People were manually matchmaking in the background and the model used to assist them in this was not predominately AI.
The genius coder that can't even ship code
😂
What can you do, whatsoever it's efficacy it is a marvel of engg. You should not mock it.
@@AI-Based-Short it's not a marvel of engineering if it's fake 💀
Devin is going to eat your lunch
@@J3R3MI6 I'll wait until the claims are proven before supporting them. I've yet to test an AI that produces quality source code or codeless applications.
Junior dev creating bugs and then fixing them seems accurate.
Sometimes, i wonder how mindless some people can be. Ai developer will be useful for one type of people, Developers. The reason is simple, to tell ai to make an api call, you need to know what api is and how it works, for you to order ai to build a database, you must know what database is and how it works, the same goes almost everywhere.
I'm just a junior dev, and forgetting ai coder, with the state ai in image generation, I can tell ai to generate a beautiful art, it will generate what it considers beautiful, but I'm not an artist, i can't indicate the combinations, the lighting, the structure that make a great art, which means, an artist will get way better art from ai compared to me.
The same goes for devs, ai coder will only be useful for people who know coding. If you believe AI can take your job, then i can state that you don't even have a skill to begin with.
Perfect comment.
Man,can you help in where can I learn coding?
Idk how to code a single line but I want to learn.
You are absolutely right. But people dont understand that and are hysterical
Even if AI doesn't fully replace jobs, it's productivity multiplier effects can allow a company to operate with less engineers.
@@mikehuang1369 same thing happened when the first OOP language surfaced. Programs became 100X faster to be produced. But, for some reason, now we have 100X more developers than pre-oop programming. Productivity multiplication sometimes means easier access to tech -> more ideas -> more tech advancement -> more jobs. Only time will tell.
More than a month,still just a closed demo.Oh well
12:52 "they [AI] cannot understand the entire context of a repository..."
Neither can many developers, unfortunately.
The CEO of nVidia appears to have made claims about Devin's capabilities to drive up stock prices and increase the sales of a specific series of high end video cards.
Software engineers won't be replaced totally any time in the future. Even if a model has a 99.9% accuracy rate, 0.01% of the time it will mess up. So 1 out of 1000 instances, it will mess up, and an instance could be a page of code....a couple of lines of code....it does not matter. What matters is that you cannot leave a machine that can break and/or have a 1 in 1000 error rate handle a multimillion dollar infrastructure on it's own. If it messes that infrastructure it will cost companies millions and millions of dollars.
Auto gpt has been around for over a year FYI. Not from nowhere.
Their website is built with webflow..... a website builder. didn't even code it themselves
VCs getting milked left, right and center 😂😂
lol when you said you go to work at 10:45AM and leave at 11AM after the standup and get some lunch
Accurate
As always, you made great points throughout. Skepticism is always a good starting point, particularly when you will be making decisions that have a large impact on people's lives. Even when news articles make grand claims, and the cultural zeitgeist says one thing: critical thought, paying attention to history, and not jumping the gun are important.
Your comment at 9:00 is spot on. If LLMs asked clarifying questions rather than hallucinating, that alone would be a step change.
I've built multiple bots that do just this. It works incredibly well. GPT is much better at asking good questions than it is at answering questions. Can't understand why more people aren't doing this.
@@neildutoit5177 got any references on how to do this?
We all want Devin or any AI to fail so bad so we can all keep our jobs and feel appreciated on Monday mornings……
(I wish I bought AI shares).
That day will come whether we like it or not.
That is not what the video is about. Aren't you a programmer
@@TrusePkay we also need the robots to understand sarcasm.
Interesting. I still dont think the time to replacement is that far off though.
You talk too much.
Why?
@derektheg946 because we are right on track with predictions. There is still room for growth on pretty much any front you want to take it. And we haven't hit any barrier that might slow it down.
@@Icedanon software engineers have a lot more work than just code tho, like being to work with other humans, their weird requests and making them happen somehow 💀. Take up meets, come up with specific ideas for specific real life problems.
No AI will be capable to handle humans lol 😂, we are too stupid and unpredictable for them.
@@daphenomenalz4100That's what the graphic designers said 💀
smells like Theranos' Edison machine.
Is Devin Chinese for Theranos?
if the popular AI channels are the only one talking about it, they're overhyping it and leading you to their aggressive sales funnels. Once in awhile you can gain some insight from these guys, but I don't have to time for profit-driven reviews and marketing fluff. Context relevant to the subject is key for me.
I am glad people caught the next Theranos or the next FTX in the making in its tracks before it got made.
it's not a scam, it's just a bad agentic framework
Just like all the other agentic frameworks
Thanks for opening our minds… sometimes people are so easy to be fooled and scammed
Not only "sometimes".
Yea Carl's video was enlightening to say the least, and I'm glad he was able to pull back the veil a bit and explain what Devin actually did.
With ChatGPT, if you tell it to ask you clarifying questions during the coding process, it will. Although I have never done it, I assume you could train a custom GPT to do it by default too.
The problem is ChatGPT isn't actually intelligent. You can give it basic logic problems and it will fail them. I don't mean looking up a famous logic problem from some website it's already trained on, but rather making up an entirely new problem that requires any kind of logical deduction. It will fail miserably. The only reason ChatGPT looks decent as a programmer is when you ask it interview questions which it already knows the answer to. The moment you start asking it for anything more specific than problems it's directly heard, it really starts to choke. It also has a big problem with being unable to say "I can't do that." and will prefer to give you a wrong solution instead of telling you what you want is impossible, or telling you it doesn't know how to achieve it.
Thanks for explaining this. It should alleviate some fears in the industry
bro said it at 1:30 same with artist and new generation kids who now will only type or think about art or such
I just want to be able to sketch out ideas with a ui and some functionality take it to an engineer
Why they dont accept my request? I made it since more thank 2 months.
Not a fan of the hype train - If I cannot try it it does not exist
Hopefully they make one that works. Can't wait to prompt our own apps!
We would have AGI already if our existing language models could ask meaningful questions without stuffing more and more context in the prompt.
Please define AGI
@@hydrohasspoken6227 An application that solves problems with the same general versatility that humans have.
The current situation of tech industry defrent universal.
So Devin pulled a Rabbit?
8:55 ??? just tell it to ask additional questions lol
but but but.... the team are the best competitive programmers in the world... and they have won medals from competitions that are for the smartest people in the world and they have top tier degrees from the best universities that have ever existed... and have been coding and solving algebra problems since they were 5...
oh wait.... I think all of the above is not really THAT relevant as to whether or not their product actually works and does what they say it does right ? oh....oh....hmmm...
You are 100% right.
so a even worse github copilot?
Yeah LLMs have never been told what is good code and bad code.
Many devs don't know either...
@@tarcus6074 Yeah I know you are trying to make a joke, but seriously, that's not true. Not knowing and not doing are very different things. LLMs don't even know what code is. They have no conception of correctness. They have no conception of design. They have no conception of anything. It is a statistical model of a language that only predicts the most likely next word in a sequence based on what it's been fed in a s training data. That's not anything close to what a bad dev is doing.
@@tommy12331 Lol even better point.
Just not true though. The base model has certainly been trained on a lot of text that explains the differences between good and bad code, but more importantly, the base model is just the base model, to become gpt4 the base model goes through supervised fine tuning and then reinforcement learning from human feedback stages, at which point it is absolutely told what responses are good code and what response are bad code.
When you say "It is a statistical model of a language that only predicts the most likely next word in a sequence based on what it's been fed in a s training data." you have to first note that that training data isn't just raw text from the internet, it's also the supervised learning training sets and the reinforcement learning from human feedback data, and second, that isn't all that the prediction is based on it is also based on the prompt text, and third, in order to do these sorts of predictions, the llms need to create internal world models, which is what gives them a conception of what they are doing. They DO have conceptions of things. Researchers have literally been able to isolate which parts of a network encode different concepts. It isn't just like a big regression algorithm calculating probabilities, it's a model of the world which is being used to calculate the probabilities.
Devin is a scam no doubt but I do think that the current LLM techonology, if developed by a team who isn't just trying to scam vcs and actually tries to do this properly, could do what Devin is promising in the short term.
@@neildutoit5177 Pretty much all of this is wrong. Maybe how they train it is right, but we'll never know what happens behind closed doors. What we do know is that we don't understand even small neural nets, so we definitely don't understand 1 trillion parameters beasts. We also understand that there is literally no mechanism for reasoning (by design and admitted by anyone working on this stuff, it's not a knock against it, it's what it is designed for). It's a giant, super expensive, probability lookup. There is no model of the world. There simply can't be. Based on what you said, you don't really understand that the prompt text is the input text and then the next word is predicted based on that. If you put "Cats are" it will output "fast" maybe because that is what it has seen occur next the most. LLM stands for large language model. It is a language model. That is it. It models languages statistically. If it were something else that did more, you better believe they would name it something better. What kind of terribly marketing would they have if they had a superintelligence and they named it a language model. That's just dumb. A lot of people are fooled by it because it looks like it is thinking when it can't. It's been shown probably millions of times now that it doesn't understand anything. It will spew complete nonsense when it doesn't pick up on the right clues in the prompt.
If Devin get fire , he could get a Robotaxi to work 😅😅😅My Grandma thinking...😅
8:50
Perplexity have follow up question
The way these AIs micro-break-down tasks is something I might find useful as a sever ADHD sufferer. Perhaps I should ask ChatGPT to do this for me... I always thought they'd be more useful as managers/producers than as engineers.
i agree, i do feel like they are better at taking manager roles than actual engineer roles.
8:35 phind Ai also does that
I like how he acts like "faking demos" is relatively new............
My brother in christ, windows was a fake demo for 3 years.
You missed the point entirely, who cares if WIndows faked a demo, its still a disgusting thing to do buddy.
@@NimVimmy point was that this has been going on a lot longer then we realize, rather stating the industry has always been scummy,
But thankyou for showing your ignorance by stating I missed the point “buddy”
@@Obamanamamama Where ignorance is bliss, 'tis folly to be wise.
@@tommy12331 haha
devin is back
To where it belongs, the trash.
Um its opensource now lmao
It isn't.?
Now i am so happy 😊
Have you guys seen the text to image results from Dall E’s a year ago?…
How about image journey today?
This is the most primitive state of Ai…. Just project the non stop exponential progress that’s bound to take place every single day until we all die and beyond, everything you think it can’t be done by Ai, will be done faster and more accurate via Ai.
Just the notion that this primitive version of Devin already exists, is all you need for thousands of people getting paid tons of money to work every single day to make this happen.
It will happen.
Only it's not exponential progress. AI progress is actually slowing down. The amount of extra benefit they're getting from adding more parameters is less and less. It's reaching diminishing returns and will likely eventually plateau.
@@taragnor you’re clueless tbh…
@@GeniusSays You're buying way too much into the hype fed to you by tech CEOs who want to boost their share prices. AI intelligence is overrated simply because it's great at taking tests. So give it a bunch of questions that have been on prior tests and it will nail them since it has that knowledge, An AI coder like Devin might look great in that it'll easily pass Amazon's interview questions, because it's been preprogrammed with those solutions.
The problem is that it fails basic logic. Make up your own logic puzzles for it, not stuff you've got off the internet, or try changing around a lot of words in the puzzle so it isn't something familiar, and it has no real reasoning capability and will constantly make mistakes. I've given ChatGPT simple logic problems like sorting 7 items, with a set of rules like "The red item is 2 steps left of green" and so forth and it tends to fail repeatedly at that sort of basic logic and deduction. You can tell it truly doesn't understand what it's saying.
And honestly I don't care how many algorithms you try to pack into the AI coder, You're just not going to render software engineers obsolete with an AI that can't use basic logical deduction.
@@taragnor You keep ignoring today's Ai is the worst version we will ever interact with, it will get better infinitely. If you can't understand this concept, i can't help you.
@@GeniusSays Yes, that's exactly the hype they want you to buy. That's essentially the same line every conman gives you, promising some amazing thing in the future. Elon Musk has been telling us that Tesla's self-driving is "just around the corner" and "coming next year" for the past 10 years or so. Spoiler: It's still not here, and what improvements have been made definitely aren't exponential progress.
As far as tech getting better infinitely. that's never happened with any discovery. Cars and planes didn't get infinitely faster, ovens didn't get infinitely better at cooking and so forth. GPT-3 wowed the world with it's potential. What followed afterwards hasn't been a huge improvement. Progress is largely slowing down. I talk to GPT4o and it doesn't seem much different from GPT 3 or GPT3.5. It still has trouble with basic logical reasoning. Now I know they've added a bunch of bells and whistles like image processing and stuff like that, but they haven't done much to improve the basic core intelligence.
Nothing is infinite, eventually everything hits a limit of some kind and progress slows or outright stops. That's how it's been with every discovery. If someone every tries to sell you something that's going to get better infinitely, they're selling snake oil. There's always a progress curve, the only tricky part is figuring out where we are on it.
Google is better than devin :)
Honestly i dont even care if or if it isnt a scam. Im jumping ship anyway even if i like it, computer science & IT is already (or going to be) oversaturated where im at.
I've been outsourcing my work to Devin for a while now. It's not a scam
Devin CEO in the comments.
proof ? :)
Sure, how do you even have access? 'for while now'
Can i hire devin
How'd you get access?
I have first hand experience of a couple of companies who believe the hype and ARE going to be replacing their engineers with AI tools. It will end badly but in the meantime there are devs who need to fine jobs.