We are using Abacus at the moment purposely to test ecah model on the same variables. Claude outperforms chatgpt, its not even close. Thanks @anthropic
Just imagine the accessibility possibilities. For those with mobility or visual impairments, Claude can assist with tasks by simply asking, like helping in usage with apps and systems that often lack proper accessibility features.
Actually any ai model with proper AI logic and ocr and nlp multimodal capabilities could do this generally it would be more use to create a hybrid system that can use ai because essentially they most likely are using Claude server side or uploading a local lightweight nlp to distribute localized automation. Which means you probably don't even generally need Claude you just need a localized app that can target any device and let you access Claude and chat gpt or any other ocr capable task. Which was achievable even before now. This actually isn't useful any only privatizes accessibility through Claude's pay wall please do not misinform people.
What I found particularly noteworthy in this demo was that the information wasn’t copied from the CRM, but typed letter by letter. Purely speculating, but perhaps because there are rare cases where websites do not accept copied input, which often also affects password managers.
Claude is hands down best LLM out there, concise and brief outputs, really good with reasoning and great personality. I like it much more than ChatGPT.
This is one more pivotal point in AI's evolution. In 2025, more innovation and use cases will emerge, and human involvement is slowly being eliminated. It looks like a small improvement, but it's huge at its core and will significantly impact how AI will be used in a few years. Kudos Claude Team!
Impressive to see Claude navigating screens like a human! Though still in beta, this could be a game-changer for automating tedious tasks. Can't wait to see how it develops! 🚀 written by *Claude 3.5 Sonnet (New)*
Computer Use is truly a pivotal advancement. Enabling AI to interact with computers like humans do is a significant leap towards AGI. Exciting times ahead!
@@funkfreeze I disagree. It requires an LLM to understand how a computer works without getting too much into coding and also how a human operates a computer to be able to do this. So, it has 'advanced' intellectually that it is able to do the things humans do on a daily basis. It is actually a 'leap' towards AGI. It would be breeze for it to do the same thing using APIs for both the CRM and Google Sheets and get it done in secs instead of minutes.
I was just looking at something like this a few days ago, and it seems Anthropic was already working in the background to deliver my hopes and dreams before I knew I even wanted it. Super exciting stuff 👍
AI malware is coming; imagine malware powered by AI that constantly watches your screen and awaits the precise conditions to drain your business bank account.
Or it could be the opposite. AI watches your screen and makes sure you don't fall for any fraudsters' schemes no matter how cleverly disguised. A future refined version of this will be great for elderly people who are often targeted.
Basically, UiPath has been doing this for years. with more and more cheaper RPA like automation, UiPath could lose a lot of customers since they are license based and their license are not cheap!
I may be oversimplifying this but is this not just taking existing RPA style software like UIPath or AA and integrating it with AI to simplify it for the end user, and not have to worry about documenting detailed processes?
What are the security implications of this? Could a bad actor use this to ask Claude to go into other people’s computers and access their confidential information?
Very cool and better Sonnet is amazing. A lot of this AI web browsing stuff is probably better via API access but for now browser simulation is probably a useful feature.
I wonder how really financially sustainable this feature is going to be in a world where companies have worldwide solved this problem with APIs. To me it looks impressive but not necessarily game changing. Maybe if I can let the tech assist me while I’m learning a new skill that would be great. I’m thinking about co-editing a photograph with a post editing software, for example.
Really?😆By the time he opened up all of the appropriate tabs, wrote detailed instructions in the prompt, he could have just fucking done it himself... He's misleading as to what's actually happening too. It's not even looking at the entire spreadsheet, just what's on screen. 'Ant Equipment Co' could have been on record 532 for all we know... my goodness the hype.
Have you thought about integrating this via the Sheets API? Taking a high res screenshot and then doing image to text conversion just to get some data that is already structured seems like a huge overhead to me. I guess changing this could cut your inferencing costs by 50% at least for this example :) but I assume you had good reasons for it. In addition, taking a screenshot cannot give you a view on the whole file in this case, might be something to consider. Still, great stuff I look forward to trying this myself!
@user-ti9yn8wg6o You'd think using the control+F function to search the sheet like a person would, wouldn't be that hard if it can search a CRM for info. I'm wondering if they mention the screenshot thing to simply illustrate that it can use image modality as well?
I am sorry, I am failing to see a point here. As an software test automation expert for the last 20 years, I am more or less doing the same thing as shown in demo, doing GUI automation, How this will be a game changer?
I would expect more people doing ticket scalping online/reserve seats for high sought after restaurants and reselling online. since AI can be trained to automate UI interface interaction, clicking through websites to buy tickets at lighting speed to snap up the best tickets and resell it at high price is definitely possible. from logging in (basic form filling), bypassing CAPTCHA (image recognition), waiting inline (timer event), selecting seats (train to select next best seat if taken), and lastly use credit card to make payment (basic form filling). who ever can program this efficiently can use it for scalping.
Slowly moving towards tens of thousands of people losing their jobs due to automation with AI in the tech industry and beyond and slowly having the average person barely able to survive. Thanks Anthropic you're heroes!
@@mihirvd01 Not in the USA you won't...Countries within the EU will have it…If something as universally accepted as single payer healthcare is rejected in the USA there’s no effing way US politicians will go for UBI…better start prepping now (or move to a first world country)
I recently spent a full day testing Claude AI on CC+ coding and encountered several issues with longer code segments. When I asked for modifications, such as adding a new function to a strategy, the AI would often include unsolicited enhancements. Instead of accurately executing the requested changes, it seemed to get confused by the length of the code and invent solutions unrelated to my instructions. It's frustrating; the AI appears to mask its limitations with these unasked-for alterations rather than admitting it can't fulfil the request. For example, despite my clear directions, it significantly altered the logic of the code, added unrequested functions, and removed essential control parameters. Each time I pointed out these discrepancies, it simply apologized and promised to review the code, only to repeat the same mistakes. This recurring issue suggests a possible memory problem with handling extensive code, leading to repeated errors as if it's losing track amidst the complexity.
Why have 3 versions? Two have limited capability. Only one is useful. Might it be better to invest all resources into the best model to enhace its availability?
From an organization perspective, how can I lock this down so that employees cannot pull data back they are not authorized to access? On the other hand, how are you keeping this data secure on Claude's side of the house with this much visibility into organizational data?
Can it scroll through different UIs? Like for the spreadsheet example is only considered the data available in the screenshot but it should have aceolled through the full spreadsheet to find what it was looking for before moving on. Is it not able to do that yet?
How do you ensure it doesn't take instructions from content shown on screen? Like with the demo you gave would a customer be able to insert commands for the ai in it's form so that the ai would provide sensitive data you would not want to share? I don't think you should ever let something take control over your computer if you can't guarantee the input it gets is safe. And even if you ask the ai to do a task solely on your computer without seeing files from a foreign source it may still come across them in the process of completing the task. Either through a notification pop up or the use the inbuild windows search engine that also shows internet results. Seems really not safe
great video. however, the background sound is intrusive to the vocals - lower it will be helpful for future videos so we can clearly hear and understand the presenter.
yeah but when can it run all the software on my computer as i train it, or if it can read pdf on the software and then do all the operations needed for it? i have a lot of marketing and advertising tools i would love it to use on its own, and optimize my websites etc for ranking purposes. say i gave it a youtube channel to watch all the videos and then do all those opeartions itself. will that be coming in the future?
i wish they made it more friendly because i am missing out on all of these since i dont know how to code and do all that complex stuff to set the AI in my pc, i dont even know whats API. so it would be cool if they had it on like where you can use it just from their app.
It is just the beginning of a new AI game, yesterday Microsoft with their Autonomous AI agent, today Anthropic and the others also will release their own just wait for the new trend.
Claude is great but OpenAI did something that in claud causes me allot of headaches OpenAi transfers chat data between chats which would be a massive help because of the limits. I believe that perplexity has the longest chat capabilities even in the free its way more than openai or claude. Please change it . By the way who is Claude? Does he work there? Just asking
It doesn't get said enough: Not only is Claude the most capable LLM, but they also have the best character. Great work Claude and Team! ❤
We are using Abacus at the moment purposely to test ecah model on the same variables. Claude outperforms chatgpt, its not even close. Thanks @anthropic
You guys are amazing! Please release it to individual users.
Wow, this is going to be quite game-changing!
Goodbye, office jobs
@@eyescreamcake good nobody likes the office jobs
boring music, boring examples....
@@TothTimea32 Reality can be boring sometimes. The example is a typical use case of boring office jobs.
Boring? As you say so. Realistic? Absolutely
@@TothTimea32 The boring stuff needs to be automated first.
Just imagine the accessibility possibilities. For those with mobility or visual impairments, Claude can assist with tasks by simply asking, like helping in usage with apps and systems that often lack proper accessibility features.
Actually any ai model with proper AI logic and ocr and nlp multimodal capabilities could do this generally it would be more use to create a hybrid system that can use ai because essentially they most likely are using Claude server side or uploading a local lightweight nlp to distribute localized automation. Which means you probably don't even generally need Claude you just need a localized app that can target any device and let you access Claude and chat gpt or any other ocr capable task. Which was achievable even before now. This actually isn't useful any only privatizes accessibility through Claude's pay wall please do not misinform people.
@@onajejones3259 Hayter...
I'm liking Claude a lot. Please continue with this form of application.
whatsinmy AI fixes this. aude 3.5 Sonnet beta capability
What I found particularly noteworthy in this demo was that the information wasn’t copied from the CRM, but typed letter by letter. Purely speculating, but perhaps because there are rare cases where websites do not accept copied input, which often also affects password managers.
Claude is hands down best LLM out there, concise and brief outputs, really good with reasoning and great personality. I like it much more than ChatGPT.
Wow, cant wait to automate my unemployment forms.😅
You're retarded
you're an idiot
This is one more pivotal point in AI's evolution. In 2025, more innovation and use cases will emerge, and human involvement is slowly being eliminated. It looks like a small improvement, but it's huge at its core and will significantly impact how AI will be used in a few years. Kudos Claude Team!
I'm not exactly a fan of big tech but claude is a genuinely amazing piece of software.
That's epic, you guys have the best A.I. This company is something special.
Impressive to see Claude navigating screens like a human! Though still in beta, this could be a game-changer for automating tedious tasks. Can't wait to see how it develops! 🚀 written by *Claude 3.5 Sonnet (New)*
Claude is soooo much underrated
Computer Use is truly a pivotal advancement. Enabling AI to interact with computers like humans do is a significant leap towards AGI.
Exciting times ahead!
AGI is an intellectual threshold, not a UX threshold. This is incredible, but we're talking about usability and access here, not intellect.
@@funkfreeze I disagree. It requires an LLM to understand how a computer works without getting too much into coding and also how a human operates a computer to be able to do this. So, it has 'advanced' intellectually that it is able to do the things humans do on a daily basis. It is actually a 'leap' towards AGI.
It would be breeze for it to do the same thing using APIs for both the CRM and Google Sheets and get it done in secs instead of minutes.
Claude AI has grown on me, it's the one I use the most now. This is impressive stuff.
Anthropic’s new release, what news could be better?
Keep it up guys, congrats on new version! 🎉
Claude is the most human AI.
Finally! I have been waiting all my life for this. Let the machine handle itself. This will be included with all computers in future generations.
I was just looking at something like this a few days ago, and it seems Anthropic was already working in the background to deliver my hopes and dreams before I knew I even wanted it. Super exciting stuff 👍
This is a powerful feature! It opens so many opportunities and speedup economy.
Love Anthropic. Still seem human and research focused unlike whatever is going on at open ai
Anthropic dropping heaters!
Immediately prompting: "Do all my work" 🤣
This must be the most impressive step since the popularization of LLMs
Bosses prompt: "which employees can I totally replace with this?"
This could be huge for companies struggling with legacy systems and modernization.
Casually saying: "We should expect this to get a lot better over the coming months."
Geez....this is amazing actually.
This introduces a HUGE attack surface to the fraudsters.
AI malware is coming; imagine malware powered by AI that constantly watches your screen and awaits the precise conditions to drain your business bank account.
@@0xAlfon any sensible user will isolate claude computer use to a virtual machine lol, only noobs will get hacked
Or it could be the opposite. AI watches your screen and makes sure you don't fall for any fraudsters' schemes no matter how cleverly disguised. A future refined version of this will be great for elderly people who are often targeted.
Looks like Siri on screen awareness but two (or more) years early and available for use now (but meanwhile, on server.) WOW. Well done guys.
This is RPA-like functionality. Wow, Will this be a game-changer?
Basically, UiPath has been doing this for years. with more and more cheaper RPA like automation, UiPath could lose a lot of customers since they are license based and their license are not cheap!
I may be oversimplifying this but is this not just taking existing RPA style software like UIPath or AA and integrating it with AI to simplify it for the end user, and not have to worry about documenting detailed processes?
What are the security implications of this? Could a bad actor use this to ask Claude to go into other people’s computers and access their confidential information?
Very cool and better Sonnet is amazing. A lot of this AI web browsing stuff is probably better via API access but for now browser simulation is probably a useful feature.
Can't wait to automate my RuneScape account thank you
Absolutely fantastic. I am looking forward to this being released as a desktop app
This is what I’ve been waiting for. This could be bigger than chatgpt itself
Absolutely incredible -- Super excited to build with this & see what others build!
Senti falta de falar sobre não seguir as sugestões às cegas. Fora isso, muito legal o conteúdo. 👏🏻👏🏻
i love how your ai explains reasons FOR an answer and reasons why an option is NOT the answer for quizzing
I don't understand at all how Anthropic is taking screenshots, clicking, and scrolling if the interface is an HTTP API
Custom implementation. That's what you need to do as well to make it work.
Local app?
@@laurenz1337_ Yeah, just seems like that's the hard part.
@@watsomk ask claude to do it for you lol
@@watsomkthey provide an extensive reference implementation, works out of the box in docker, easy to adapt to your needs.
Does this mean, it can now bypass re-captcha?
I wonder how really financially sustainable this feature is going to be in a world where companies have worldwide solved this problem with APIs. To me it looks impressive but not necessarily game changing. Maybe if I can let the tech assist me while I’m learning a new skill that would be great. I’m thinking about co-editing a photograph with a post editing software, for example.
You can definitely build that with computer use.
Awesome stuff!
Really? It submitted the form without you approving it first?
This is absolutely amazing !!!! I love it
Best innovation of the year
Really?😆By the time he opened up all of the appropriate tabs, wrote detailed instructions in the prompt, he could have just fucking done it himself... He's misleading as to what's actually happening too. It's not even looking at the entire spreadsheet, just what's on screen. 'Ant Equipment Co' could have been on record 532 for all we know... my goodness the hype.
Claude is just amazing
Have you thought about integrating this via the Sheets API?
Taking a high res screenshot and then doing image to text conversion just to get some data that is already structured seems like a huge overhead to me.
I guess changing this could cut your inferencing costs by 50% at least for this example :) but I assume you had good reasons for it.
In addition, taking a screenshot cannot give you a view on the whole file in this case, might be something to consider.
Still, great stuff I look forward to trying this myself!
Is it "taking screenshots of the spreadsheet" or actually searching the whole spreadsheet?
It took a screenshot, it did not scroll down, it did not search.
@@jnevercast Sure, but then later there are examples of it searching through docs. Hence my question.
my question is does it even take a full page screenshot or just the current page - it's a long spreasheet
@user-ti9yn8wg6o You'd think using the control+F function to search the sheet like a person would, wouldn't be that hard if it can search a CRM for info.
I'm wondering if they mention the screenshot thing to simply illustrate that it can use image modality as well?
I am sorry, I am failing to see a point here. As an software test automation expert for the last 20 years, I am more or less doing the same thing as shown in demo, doing GUI automation, How this will be a game changer?
You’re not doing it in prod.
@@CollinGravesPersonal Yes we do? E2E or system tests have been a thing in production forever. You can't prove the system is working otherwise.
I would expect more people doing ticket scalping online/reserve seats for high sought after restaurants and reselling online. since AI can be trained to automate UI interface interaction, clicking through websites to buy tickets at lighting speed to snap up the best tickets and resell it at high price is definitely possible. from logging in (basic form filling), bypassing CAPTCHA (image recognition), waiting inline (timer event), selecting seats (train to select next best seat if taken), and lastly use credit card to make payment (basic form filling). who ever can program this efficiently can use it for scalping.
LLMs are quite resistant to this due to their training, its easier to script it up with traditional programs.
I could anticipate a future where we just ask the pc to do tasks for us without user clicking or typing anything
Slowly moving towards tens of thousands of people losing their jobs due to automation with AI in the tech industry and beyond and slowly having the average person barely able to survive. Thanks Anthropic you're heroes!
@@Roaming8667 Pointing out facts isn't whining. Come back to this comment in six years and tell me how great it is for civilization
@@CrayDilla Don't worry, you'll have UBI and a much better standard of living.
@@mihirvd01 Not in the USA you won't...Countries within the EU will have it…If something as universally accepted as single payer healthcare is rejected in the USA there’s no effing way US politicians will go for UBI…better start prepping now (or move to a first world country)
@@Roaming8667 Luddites are gonna luddite.
@@mihirvd01Stop peddling UBI as some sort of solution. It's BS.
I recently spent a full day testing Claude AI on CC+ coding and encountered several issues with longer code segments. When I asked for modifications, such as adding a new function to a strategy, the AI would often include unsolicited enhancements. Instead of accurately executing the requested changes, it seemed to get confused by the length of the code and invent solutions unrelated to my instructions. It's frustrating; the AI appears to mask its limitations with these unasked-for alterations rather than admitting it can't fulfil the request. For example, despite my clear directions, it significantly altered the logic of the code, added unrequested functions, and removed essential control parameters. Each time I pointed out these discrepancies, it simply apologized and promised to review the code, only to repeat the same mistakes. This recurring issue suggests a possible memory problem with handling extensive code, leading to repeated errors as if it's losing track amidst the complexity.
Why have 3 versions? Two have limited capability. Only one is useful. Might it be better to invest all resources into the best model to enhace its availability?
Kind of excited about this one
Make it no-code and ship it please! Would love to automate some of out workflow!!
Guys (Anthropic), I think you should sell this so we can use it locally. We won't have to worry about how our data is handled.
Anthropic is too performant to run locally, unlike mistral or llama
From an organization perspective, how can I lock this down so that employees cannot pull data back they are not authorized to access? On the other hand, how are you keeping this data secure on Claude's side of the house with this much visibility into organizational data?
With this beta version, is the code already working on the client if called in a client program through API?
Can it scroll through different UIs? Like for the spreadsheet example is only considered the data available in the screenshot but it should have aceolled through the full spreadsheet to find what it was looking for before moving on. Is it not able to do that yet?
I wonder how this could be leveraged for usability and automated testing...
How do you prevent Claude from storing or reusing my personal, PII and/or sensitive information while taking reading the data ?
No problem. Claude can also change your password after it used it.
It would be nice to have the cheaper Haiku 4.
Since Google and OpenAI have reduced prices for smaller models.
How were you able to run macos in a virtual machine?
How do you ensure it doesn't take instructions from content shown on screen? Like with the demo you gave would a customer be able to insert commands for the ai in it's form so that the ai would provide sensitive data you would not want to share? I don't think you should ever let something take control over your computer if you can't guarantee the input it gets is safe. And even if you ask the ai to do a task solely on your computer without seeing files from a foreign source it may still come across them in the process of completing the task. Either through a notification pop up or the use the inbuild windows search engine that also shows internet results. Seems really not safe
I’d like to have Claude automatically complete job applications that don’t accurately pull data from a resume.
great video. however, the background sound is intrusive to the vocals - lower it will be helpful for future videos so we can clearly hear and understand the presenter.
yeah but when can it run all the software on my computer as i train it, or if it can read pdf on the software and then do all the operations needed for it? i have a lot of marketing and advertising tools i would love it to use on its own, and optimize my websites etc for ranking purposes. say i gave it a youtube channel to watch all the videos and then do all those opeartions itself. will that be coming in the future?
Exciting!!! great job.
Awesome! Now this is a good step for AI agents
why is it so difficult to register my phone number to use the mobile app?
i wish they made it more friendly because i am missing out on all of these since i dont know how to code and do all that complex stuff to set the AI in my pc, i dont even know whats API. so it would be cool if they had it on like where you can use it just from their app.
there might be several alternatives that you can try that are very user-friendly and no tech-skills needed. I'm also looking for some options
Superb ! But why the annoying music is louder than the voice of the speaker. Very disturbing.
you guys literally built a web browser multi agent with such a long correct planning. is this fine tuned?
It is just the beginning of a new AI game, yesterday Microsoft with their Autonomous AI agent, today Anthropic and the others also will release their own just wait for the new trend.
What is going on with that whiteboard?
Where do I get the hoodie?
We are all out of a job in 5-7 years.
Out of this job onto another
what is the RPA tool that you use integrated with claude? I didnt understand exactly how you did that. Amazing :)
No RPA tool. Native to Claude.
Claude is Love
I wonder if he will pass the captcha. Will he stay true to his principles, or will he prove that he's not a robot after all?
is it safe for webform that includes recaptcha?
AnthropicAI beige color is unique 😎🎉
Claude is great but OpenAI did something that in claud causes me allot of headaches OpenAi transfers chat data between chats which would be a massive help because of the limits. I believe that perplexity has the longest chat capabilities even in the free its way more than openai or claude. Please change it . By the way who is Claude? Does he work there? Just asking
Can I create a RuneScape bot with this? 😄
old school runescape is cooked
🙀Gotta try it out.
Thanks heaps.
thanks - great - definitly I will try it :)
Best AI platform ever!
where does he come up with the orders email ?
it's right there 1:18
"This is so cool, it might replace automation testers faster than they can write ‘Hello World!’" 😅
does it passes I'm not a Robot check ?
it will be a game changer when it gets 1000x faster
Amazing! I was developing something like this!
What song is that though?
How to replace whole Deparment with 1 human and Antripic Computer
Wow ❤ you’re the best
Yeah, that's cool, but how could you possible trust it against an audit?
If Claude release this first, I'll cancel my OpenAI Subscription the same day and sign up for Anthropic's!
Will it ever run sudo rm -rf /*
Can this be done NOW??
This is awesome!