What Theo did in this video Step 1: Show a common exploit not just limited to S3 Step 2: Say Scary, Nightmare, Terrifying a bunch of times Step 3: Plug your favorite service
I tried to reproduce the exploit from the article with my service. That's what I learned: 1. This bug has nothing to do with S3 and only works when you insert HTML content from users into the DOM. 2. The article focuses on , but the author does not mention that it is impossible to download malicious html / xml / svg code through , in any way at all. Browsers have taken care of this and any attempts to execute JavaScript in files downloaded via will fail. 3. UploadThing and any other service will not save you if you insert HTML from user files into the DOM. This means that the vast majority of services are safe, even if they have incorrect validation in S3.
Is the problem here really S3? Not trusting user input is cyber security 101. If a dev fails on that principle at this fundamental a level, switching out the object storage solution is just treating the symptom rather than curing the cause. They need education on cyber security fundamentals.
Was thinking the same... Not Cloud savvy because, well, this video tells half the story, but a quick search told me you can run some lambda post upload to check on things. That could be used, but still gives you a small window of opportunity. Probably too small to be used, but still there. IMHO tough, this is a failure on AWS's part. They SHOULD only receive "as binary", and serve only "as binary" unless some post upload function changed the file to a specific mime type, ie, default is binary, changing it is on the owner.
We can be in favor of teaching people how to avoid shooting their own foot off, while at the same time being against the distribution of footguns. Saying "Bad dev needs to know better!" is a way of rationalizing losses, not preventing them. People have finite lifespans and are replaced over time. So even if everyone is constantly learning, the equilibrium state is always going to contain tons of avoidable errors. The design of intuitive tools and dummy-proof solutions are an actual way of improving that result.
Hmm. I don’t quite like this approach of, “Hey S3 is “hard” and you are probably going to get it wrong, just use this thing instead”. Using S3 actually isn’t hard and I don’t want people to walk away with the impression that it is. It’s worth learning how to use it, and you are smart enough to get it right.
coding isn't hard but still AAA game companies create thousands of pretty obvious bugs in their games and they remain inside until release. the only hard thing in coding is finding the correct information about HOW it is savely done without exploids. 99% of the information that you can easily find is unsafe and can easily be exploided. that is also why copilod/chatgpt is so horrendesly bad at coding secure systems even if it is just about a basic user text input. 90% of the case it won't even verify that the string isn't being escaped from. for chatgpt/copilot to give that answer the MAJORITY of the code it trained on had to be like that. so yeah good luck with finding the right information as a new dev that doesn't have a "mentor" and/or computer science degree.
@@dyto2287 it's possible to be a solid backend dev in JS... The problem is people that call themselves a "full stack" dev just because they did a codealong to spin up a few routes on Express.
Is there any data that supports the idea that this type of pattern is a common way to use S3? I don't think I've ever even had a use case to let users upload files directly without any validation like this. Anyway I'm not quite seeing how any of this is an S3 problem. Wouldn't this be a problem common to any file server API? Even if you were running your own file server on a raspberry pi
This video is totally misleading… These are not S3 issues, they are bad code / lazy coder practices. The issues shown would have the same effect regardless of where you store the files. The message shouldn’t be don’t use service …, it should be never trust user generated input. If you let the user specify the file name to be saved, and don’t sanitise it fully, it’s easy to see how they can break any server.
The point is the complexity in setting up S3 correctly makes it a pain to setup and really easy to mess up. Your not gonna get an error if you misconfigure, your just gonna get hit. Also don't forget DDOSing private or public buckets to increase spending.
I never heard Theo say don’t use S3 in this video. That wasn’t his message. His message was “it’s easy to mess up with S3, make sure you know what you’re doing”.
@@jkdmyrs true, but the title says s3 is a security nightmare. My point is that this is no more valid than saying SQL is a security nightmare, if you let users input go through unchecked. The same security nightmare exists on HDDs if you follow the same bad practices.
@@ben-brady S3 is not complex to setup. Just make it private, control paths on a server side and sanitise input. Both examples in the video of complexity were S3 agnostic. And while agree that there is an issue with cost of requests on an S3 bucket, that’s got nothing to do with this video.
@@kilwothe video showed several security issues that are easy to be vulnerable to if you don’t set up S3 right. To me, that’s worthy of saying it’s a security nightmare.
The title of this video misleadingly suggests that S3 is the problem, when in reality, it’s poor frontend development practices. Key generation should not be handled by the client; it should be managed by the backend API
Interesting, but no mention of SignatureV4 on those issues? We should always have a SignatureV4 to prevent the user to change anything from the parameters set by the server on the presigned URL (like the key for example) and upload files into a temp "folder" or temp bucket.
02:19 one authentication is enough if your server and user can reuse connection (like wss, ssh): user creates connection to your server user sends password (or whatever other secret) and "permission request" server sends "Yes u can upload" user sends file data
Not really an S3 issue. Probably some devs copy pasting code from tutorials or UA-cam tutorials. S3 docs are not beginner friendly but not that bad if you read references instead of guides. Never used Post Signed URL because of the lack of validations. But did not know it can be used in such an exploitive way. Letting the client generate a key is also a bad design and not an S3 issue. Can also happen if the user is using storing files on the disk. Anyway enabling object versioning is a good idea to prevent any loss in case of any accidental upload. A very good and interesting video on this topic.
Looks like it, an ad to something that makes it easier to do this right. But someone who is actually trying to do things right and isn't rushed into it doesn't need these products.
we have been uploading files onto the internet since 90s. You can't tell me that it is such a big problem to upload a file. It sounds more of a skill issue than anything else.
MIME types are not checked by S3 as far as I'm concerned (allowed extensions can be set, but you can lie to them in the MIME type), so how do you check that if not on your own server?
Fun fact, if you were doing this, doing security research; you would straight up get sued in Germany. No "thank you", no reward, straight up a lawsuit at which you can be put in jail for 3 years or get fined up to 50k EUR. I hate my country for taking security so seriously by shutting those down who do it for a living in good intentions. Locally, we call it the "hacker paragraph," but legally speaking, it's § 202a StGB
Absolutely love how Eva casually strolls around and hack everything she can get her hands on :) Majestic girl! And Theo is being her popularizer via YT, while she can focus on her stuff - 100IQ move :)
So basically: * Choose the filename (object key) on the server side * Force the mime type to be something that you expect * In case you allow your users to upload HTML (or more accurately, you allow the text/html mime type), make sure that your cookies are set up properly And that's it? Doesn't sound that hard, really
Shouldn't we use Signed policies instead of presigned urls for uploading files to S3 bucket? Signed policies seems to be more secure alternative because backend can specify what kind of file user can upload, size of the file and where it should be uploaded
Server validation is the key. That is why frontenders should not do backend work and why I don't like ssr trying to also do too much backend logic. I am sure your services have some vulnerabities as well, if not the same if the user misconfigures its backend (since you also rely on S3). Lastly, unsure how secure are your services, but large corps usually require certifications, which S3 do have, but you do not necessarly (talking about file access - since buckets are technically owned by you, and that is not very secure for most organizations)
User input is a security nightmare. < fixed the title. Don’t trust user input for anything, anyone who says you can is lying. Also, account ids are not to be considered secret.
9:55 The cookie HTTP tag has nothing to do with it being "HTTPS" - the HTTP flag makes the cookie inaccessible to JavaScript. There is a separate flag for secure cookies. Also, responding with an access token on the /me endpoint is crazy and unnecessary: If you're already using HTTP cookies for auth (refresh), just stay with that without ever exposing it to JS.
Bruh, why not receive the stream and then pass it to S3, like, all data regarding the upload will be on header, and most people can use an API Gateway SaaS to handle the authorization part... I find it kinda nuts to expose a service directly to an inherently untrustworthy client... People need to understand that all data that comes from the client is bad until vetoed/validated!
I know it's not my right, but I'm proud of Eva. I hope she does great with her bright future. She gives me hope that those behind my gen can keep things going.
What if they simply limit and define the type of fileuploads. I mean not everyone needs a google drive like storage ?most may simply need a Jpeg, png upload
THANK YOU SO MUCH FOR REPORTING ON THIS. Thankfully I am smart enough to sanitize stuff properly but boy did I get scared that I messed up somewhere and this could work.
there is no such thing as "secure". no matter how many people there are on your team... no matter how many HOURS you dump into "security"... within a few hours of release someone out there will find an exploit. period.
This... this has nothing to do with S3 and everything to do with programmers that don't know basic security. I'm not really fond of AWS but can't you make better videos to promote uploadthing (a service i do actually like) instead of this nothing burger? Do better!
What Theo did in this video
Step 1: Show a common exploit not just limited to S3
Step 2: Say Scary, Nightmare, Terrifying a bunch of times
Step 3: Plug your favorite service
I tried to reproduce the exploit from the article with my service. That's what I learned:
1. This bug has nothing to do with S3 and only works when you insert HTML content from users into the DOM.
2. The article focuses on , but the author does not mention that it is impossible to download malicious html / xml / svg code through , in any way at all. Browsers have taken care of this and any attempts to execute JavaScript in files downloaded via will fail.
3. UploadThing and any other service will not save you if you insert HTML from user files into the DOM.
This means that the vast majority of services are safe, even if they have incorrect validation in S3.
Is the problem here really S3? Not trusting user input is cyber security 101.
If a dev fails on that principle at this fundamental a level, switching out the object storage solution is just treating the symptom rather than curing the cause. They need education on cyber security fundamentals.
Was thinking the same... Not Cloud savvy because, well, this video tells half the story, but a quick search told me you can run some lambda post upload to check on things. That could be used, but still gives you a small window of opportunity. Probably too small to be used, but still there.
IMHO tough, this is a failure on AWS's part. They SHOULD only receive "as binary", and serve only "as binary" unless some post upload function changed the file to a specific mime type, ie, default is binary, changing it is on the owner.
Most of the JScript kiddies think they know shit when in actual world, they don't know jack shit.
We can be in favor of teaching people how to avoid shooting their own foot off, while at the same time being against the distribution of footguns.
Saying "Bad dev needs to know better!" is a way of rationalizing losses, not preventing them.
People have finite lifespans and are replaced over time. So even if everyone is constantly learning, the equilibrium state is always going to contain tons of avoidable errors. The design of intuitive tools and dummy-proof solutions are an actual way of improving that result.
AWS account IDs are not a secret...
be careful theo might block you
Exactly, who gives a shot about the account ID
They are PII though, so it's good to still conceal in most situations.
Anything personal is secret
Trying to flog his upload thing project for sure.
Hmm. I don’t quite like this approach of, “Hey S3 is “hard” and you are probably going to get it wrong, just use this thing instead”.
Using S3 actually isn’t hard and I don’t want people to walk away with the impression that it is. It’s worth learning how to use it, and you are smart enough to get it right.
These are JS devs that code backend. What did you expect? 😂
coding isn't hard but still AAA game companies create thousands of pretty obvious bugs in their games and they remain inside until release.
the only hard thing in coding is finding the correct information about HOW it is savely done without exploids. 99% of the information that you can easily find is unsafe and can easily be exploided. that is also why copilod/chatgpt is so horrendesly bad at coding secure systems even if it is just about a basic user text input. 90% of the case it won't even verify that the string isn't being escaped from.
for chatgpt/copilot to give that answer the MAJORITY of the code it trained on had to be like that. so yeah good luck with finding the right information as a new dev that doesn't have a "mentor" and/or computer science degree.
@@dyto2287 it's possible to be a solid backend dev in JS... The problem is people that call themselves a "full stack" dev just because they did a codealong to spin up a few routes on Express.
@@dyto2287 we expected a smug loser to come in bragging about doing something equally mundane.
As a dev who's had to use aws since I only just a wee js dev.. s3 was always the easiest thing I had to work with.
Nice Ad Theo.
Is there any data that supports the idea that this type of pattern is a common way to use S3? I don't think I've ever even had a use case to let users upload files directly without any validation like this. Anyway I'm not quite seeing how any of this is an S3 problem. Wouldn't this be a problem common to any file server API? Even if you were running your own file server on a raspberry pi
Yeah this seems like an S3-backed third party issue, not S3 directly.
"There's no world where you can override someones file" - I bet someone is going to take that as challenge accepted heheh.
This video is totally misleading… These are not S3 issues, they are bad code / lazy coder practices. The issues shown would have the same effect regardless of where you store the files. The message shouldn’t be don’t use service …, it should be never trust user generated input. If you let the user specify the file name to be saved, and don’t sanitise it fully, it’s easy to see how they can break any server.
The point is the complexity in setting up S3 correctly makes it a pain to setup and really easy to mess up. Your not gonna get an error if you misconfigure, your just gonna get hit. Also don't forget DDOSing private or public buckets to increase spending.
I never heard Theo say don’t use S3 in this video. That wasn’t his message. His message was “it’s easy to mess up with S3, make sure you know what you’re doing”.
@@jkdmyrs true, but the title says s3 is a security nightmare. My point is that this is no more valid than saying SQL is a security nightmare, if you let users input go through unchecked. The same security nightmare exists on HDDs if you follow the same bad practices.
@@ben-brady S3 is not complex to setup. Just make it private, control paths on a server side and sanitise input. Both examples in the video of complexity were S3 agnostic. And while agree that there is an issue with cost of requests on an S3 bucket, that’s got nothing to do with this video.
@@kilwothe video showed several security issues that are easy to be vulnerable to if you don’t set up S3 right. To me, that’s worthy of saying it’s a security nightmare.
The title of this video misleadingly suggests that S3 is the problem, when in reality, it’s poor frontend development practices. Key generation should not be handled by the client; it should be managed by the backend API
Interesting, but no mention of SignatureV4 on those issues? We should always have a SignatureV4 to prevent the user to change anything from the parameters set by the server on the presigned URL (like the key for example) and upload files into a temp "folder" or temp bucket.
presigned urls are, already presigned. what am I missing?
The server sets the upload path, and modifying it will cause the hash used to sign the original request to fail
02:19 one authentication is enough if your server and user can reuse connection (like wss, ssh):
user creates connection to your server
user sends password (or whatever other secret) and "permission request"
server sends "Yes u can upload"
user sends file data
Not really an S3 issue. Probably some devs copy pasting code from tutorials or UA-cam tutorials.
S3 docs are not beginner friendly but not that bad if you read references instead of guides.
Never used Post Signed URL because of the lack of validations. But did not know it can be used in such an exploitive way.
Letting the client generate a key is also a bad design and not an S3 issue. Can also happen if the user is using storing files on the disk. Anyway enabling object versioning is a good idea to prevent any loss in case of any accidental upload.
A very good and interesting video on this topic.
my hate meter for theo has been growing exponentially lately !
is this just a ad
Looks like it, an ad to something that makes it easier to do this right.
But someone who is actually trying to do things right and isn't rushed into it doesn't need these products.
we have been uploading files onto the internet since 90s. You can't tell me that it is such a big problem to upload a file. It sounds more of a skill issue than anything else.
Curios what you think about streaming the upload to S3 instead of waiting for the full buffer in the service ingress?
MIME types are not checked by S3 as far as I'm concerned (allowed extensions can be set, but you can lie to them in the MIME type), so how do you check that if not on your own server?
When generating the presigned post url you can specify mime type conditions
Fun fact, if you were doing this, doing security research; you would straight up get sued in Germany.
No "thank you", no reward, straight up a lawsuit at which you can be put in jail for 3 years or get fined up to 50k EUR.
I hate my country for taking security so seriously by shutting those down who do it for a living in good intentions.
Locally, we call it the "hacker paragraph," but legally speaking, it's § 202a StGB
Absolutely love how Eva casually strolls around and hack everything she can get her hands on :) Majestic girl!
And Theo is being her popularizer via YT, while she can focus on her stuff - 100IQ move :)
M$ makes sure that you will never go to S3 sleep ever again :/
So basically:
* Choose the filename (object key) on the server side
* Force the mime type to be something that you expect
* In case you allow your users to upload HTML (or more accurately, you allow the text/html mime type), make sure that your cookies are set up properly
And that's it? Doesn't sound that hard, really
Shouldn't we use Signed policies instead of presigned urls for uploading files to S3 bucket? Signed policies seems to be more secure alternative because backend can specify what kind of file user can upload, size of the file and where it should be uploaded
presigned urls can do all those.
Ah yes Theo the expert in AWS Sigv4 and Sigv4a. He totally even knows what those are. Stop using clickbait titles
Whats difficult to understand about Sigv4? And he built a product on top of AWS so why exactly wouldn’t he know about it?
Server validation is the key. That is why frontenders should not do backend work and why I don't like ssr trying to also do too much backend logic.
I am sure your services have some vulnerabities as well, if not the same if the user misconfigures its backend (since you also rely on S3).
Lastly, unsure how secure are your services, but large corps usually require certifications, which S3 do have, but you do not necessarly (talking about file access - since buckets are technically owned by you, and that is not very secure for most organizations)
at 1:28 what is the diagramming tool he used in this video ?
excalidraw
@Theo transitioning into cyber security? :D
He barely know anything about software engineering. I doubt it.
He owns an s3 upload service called Uploadthing. This is an ad
User input is a security nightmare. < fixed the title.
Don’t trust user input for anything, anyone who says you can is lying. Also, account ids are not to be considered secret.
9:55 The cookie HTTP tag has nothing to do with it being "HTTPS" - the HTTP flag makes the cookie inaccessible to JavaScript. There is a separate flag for secure cookies. Also, responding with an access token on the /me endpoint is crazy and unnecessary: If you're already using HTTP cookies for auth (refresh), just stay with that without ever exposing it to JS.
Bruh, why not receive the stream and then pass it to S3, like, all data regarding the upload will be on header, and most people can use an API Gateway SaaS to handle the authorization part... I find it kinda nuts to expose a service directly to an inherently untrustworthy client... People need to understand that all data that comes from the client is bad until vetoed/validated!
because that would be phenomenally expensive
Isn't this what the CSRF token is supposed to be for?
I know it's not my right, but I'm proud of Eva. I hope she does great with her bright future. She gives me hope that those behind my gen can keep things going.
What if they simply limit and define the type of fileuploads. I mean not everyone needs a google drive like storage ?most may simply need a Jpeg, png upload
I see no point in the cloud (not my self hosted but one of the corporations), if I still have to worry about security.
please make uploadthings customisable to be able to use 3rd party or selfhosted S3
this convinced me to stop using any startup software products
lol, I turn all the security off and open it to the web. As everything I want it for is public access files.
4:30 HAAAA, I can see so many exploits from that
all this could be fix with proper docs
Did I just waste my leisure time to watch a clickbait? worth it 😂
Can you not shill your service… k thnks bye
THANK YOU SO MUCH FOR REPORTING ON THIS.
Thankfully I am smart enough to sanitize stuff properly but boy did I get scared that I messed up somewhere and this could work.
Yikes. I'm no AWS fanboy, but every point covered here that relates to S3 is a nothing burger. This video is an ad for Vercel-ified S3.
okay okay, you have changed my mind, I will use your service, I hope the python is well adopted :))
Uploadthing Moment
Use this as I vote button for more Cysec/Infosec!!!!!!
So that there is more click bait ?
there is no such thing as "secure". no matter how many people there are on your team... no matter how many HOURS you dump into "security"... within a few hours of release someone out there will find an exploit. period.
I was just about to start a S3 project!?
This... this has nothing to do with S3 and everything to do with programmers that don't know basic security.
I'm not really fond of AWS but can't you make better videos to promote uploadthing (a service i do actually like) instead of this nothing burger? Do better!
Very Informative. Thanks Theo ✌🏻
Eva ❤
kinda weird to call Eva "they" while showing her twitter bio that says she/her, I'm sure there's no malice there just saying
they is gender neutral there is nothing weird about using it to refer to anyone.
@@bm1259 It hides their preferred gender, this is harassment, just like choosing the biological sex.
I'm sorry but there is absolutely nothing wrong with calling some they. The pronoun they is neutral.
Who uses AWS S3 in 2024? 😂😂😂
70% of the internet?
what do you use if not s3?
Almost everyone?
Who could be so disconnected from reality in tech that they'd think that S3 wasn't the dominant cloud file storage service in 2024?