Private EMPTY S3 Bucket COST ME $1300
Вставка
- Опубліковано 5 тра 2024
- Recorded live on twitch, GET IN
Article
/ how-an-empty-s3-bucket...
By: Maciej Pocwierz
My Stream
/ theprimeagen
Best Way To Support Me
Become a backend engineer. Its my favorite site
boot.dev/?promo=PRIMEYT
This is also the best way to support me is to support yourself becoming a better backend engineer.
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Kinesis Advantage 360: bit.ly/Prime-Kinesis
Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))
turso.tech/deeznuts - Наука та технологія
Ironically AWS lambda is $1 per 1 million requests.
So if you pay $1 for lambda, you can cost someone $5
I like the cut of your jib.
one lambda could do more than one put request
I think your account would be banned after just a few thousand requests
Aaaand who wins? Jeff Bezos
The request is to invoke the lambda I guess. The compute of lambda is billed separately
AWS should patch this. Now we know why CDK uses such long hashes for stack bucket creation. This is just plain ridiculous and they should apply a DDoS filter & stop charging for private bucket spam.
yet the fact that there isn't a fix/workaround isn't going to stop security from emailing me, about every bucket in my enterprise, with their newly-found 'exploit'
I'm going to have to force every application owner to use hashed name values (edit: for EVERY service since we know this'll be a new class of 'exploit'), as if they didn't have a hard enough time understanding the infrastructure already.
How bad of a bucket name was it though…
AWS should PATCH? More billings...
This is legitimate traffic and aws support usually has common sense when you explain that it was legitimate traffic but not really legitimate traffic
Why would they switch off their free money machine?! How dare you think that Bezos can not make it to space on your money!
He needs to beat Elon does he not? I mean, if he doesn't put a penis in space, who will?!?!
So... basically what I got out of this is that AWS has no s3 ddos protection and will just push the cost to the owner. Unauthed, untargetted requests to something set to private should not result in charges.
they kinda **do** have DDoS protection. They will scale their backend HW to resolve all requests so there will be no denial of service... *devil grin*
(not the kind of DDoS protection I would like to pay for)
I don't think proper DDOS protection would block this traffic. Requests described in the article were kinda legitimate requests to backup data, although from a suspicious number of users.
The whole purpose of DDoS protection is that the service continues to work even if the service is under high load. Now one option would be to block illegitimate requests. But obviously that's way to complicated when the alternative is to just charge the victim for the additional requests.
They do have DDoS protection, they just charge you for it.
"This was done as an exception"
Kids, that's why you may not want to use aws.
Dont say that bro, I just spend some 1000 dollars on aws certification as part of job requirement lmao
@@shabarinathk8954 as long as you’re not the one footing the bill, you don’t need to worry 😉
@@shabarinathk8954I wouldn't worry about that as an employee since it's not your money. The entire justification for employment is that you earn a fraction of the profits in exchange for them taking the risk, so don't let them take that away from you.
This story blew up. In response, Amazon said they are going to work on stopping this issue. (They are probably going to stop billing for 400 errors, but I do not know their actual planned solution.)
And lose revenue? Well, maybe there is hope after all
Prime doing the lords work by sharing with the masses
Do you have a link for the AWS response? I am curious
@@andreiaoca9611 There's link in the original article up top.
The fact they're charging for 400 errors in the first place and it was an intentional decision and not just a lack of consideration just boggles my mind. Especially the double charge for the redirect.
Consider this - despite literally every developer who has worked with AWS/Azure/GCP asking for it, to this day none of them have added a cost limiter which would shut down your resources the moment it exceeds a user defined amount.
How much profit this cabal have gained from this, I'd like to know.
At least a few dollars/year from me... for one of the cheapest tiers of VM.
you and every other executive looking to claw back some of their cloud spend.
This'll get fixed or AWS will be doling out credits...IF Azure isn't susceptible/doesn't charge for rejected requests.
Azure has AZ Monitor, that can (if set up) shut down your resources when a rule (too high cost) occures.
You can absolutely do that in AWS and I think Azure too.
About 10 years ago, when a Azure representative was selling us their stuff, I asked a very simple question: "So, what happens when someone DDOS my service?"
They answered: "Well, how would you do it now? We don't implement any less than what you have now and our services are designed to scale up, not to shut down".
To which I said "If this was on my servers, the DDOS would just exhaust my resources and my hardware cost is fixed, on the cloud the DDOS just keeps going and I keep paying more and more." to which they just shrugged and looked up the company's food chain to sell their bs.
Cloud does not care about your bottom line, they care about their bottom line!
The moral of this story is that the cloud sucks in ways that you cant even imagine.
I don't think cloud suck.
Typical Cloud companies do.
They don't put Cloud, they put Smog.
It also sucks in ways you can imagine.
Cloud sucks because of ways you can't imagine.
Yeah, to suck the money out of your wallet.
cloud is amazing, this is just some good old capitalism, cloud is bad only for those who can afford their infra
This is why I deleted my AWS account. I was using the free tier only, and I heard one too many horror stories of large unexpected bills. I was concerned I would either make a mistake and leave something running, or I would get hacked, despite a 32 character password and 2FA. But this is a new class of gotcha that was beyond his control. lol
Just deleted mine
Well, how about create empty visa card, and register to aws. And only top up when the billing is reasonable. If it's not, then don't ever top up
@@anbiabohlam5468 Even an empty visa has your id, they will charge you in court of law if you don't pay the bill. That is actually the worst you can do, the best is just to speak to them and try to make a deal or explain what happened. A lot of times they just let it go with just a warning like it was told in the video.
don't forget to nuke it with aws-nuke (this tool will also bye-bye your ability to log in). they will incur fees on resources left running on a deleted account
tbh i feel aws is not hobbyist-friendly, with its notorious horror stories and its least expensive vps plan i could find was ec2 t.nano instances (4.23$/month) + public ipv4 fee 3.6$/month + tax = 8$/monthly, if you have exhausted fully your free-tier.
@@Carlos-rc1ix would they actually do that? would they win?
The company I work at literally had to review and change our bucket names when this story dropped. There was stuff like "production-us-east-1"... I wonder how much we've paid over the years for random bot traffic on crap like this. AWS billing is the devil.
i had a aws trial back in the day testing one of their VPS. next thing i know i wasn’t using free tier stuff and i had a 10k bill afterwards running for 2 months i didn’t realize. you think I paid them? no, though they resolved it by removing my bill. since then i haven’t touched it cause AWS was soo confusing
I’m still regularly amazed that we pretend S3 is “totally fine”. The number of things like this are insane. From billing on “put” to IAM to presigned URLs being utter chaos and often insecure, we can do better.
Which service do you use to store media files for your 'upload things' product?
We could, but no one actually does.
@@vaibhav5783Really you can do any and write your own S3 within week.
99.9% never ever need this scalability even theoretically. And use very very basic features. It's not better than before "cloud era".
@@vaibhav5783 cloud flare has free bandwidth
Azure then?
brb creating a bucket called "SAP-DATA" and making it publicly writable
SIMP-DATA
are you sure you created it yet ?
How in the world can a "free-tier" bucket accumulate a bill that high in 2 days? How are there not limits in place?
Money
"No limits" is essentially AWS's business model. You never know before how much money you're going to pay for what you're doing and don't have the ability to limit it. Barely anybody truly understands all the details of their ridiculous pricing structure.
@_DATA_EXPUNGED_ holy tripping hazard, Batman. I'll keep that in mind should I ever need cloud space. And check whether Azure et al offer that feature.
When you reach a "free-tier" upper threshold, then you will pay as you go.
Because Free Tier is combined with actually paid tier.
Cloud companies usually put their Free Tier with Paid Tier.
So I think what is happening is you are using Free (but limited), but once you do something, that is not included freely.
You pay for it as if you have the paid Tier version.
This is wild.
Yes, the 4xx are expected responses. Technically, any response can be seen as expected.
That they bill unauthorized requests, leaves me speechless though.
Imagine your Phone provider billing from the moment the opposite phone just rings, (instead of the usual when the opposite side takes the call).
It's just a 💩ty thing to do.
I believe you can technically hack in a way to track usage and determine whether or not to disable certain services by coding it into your app and controlling AWS via API calls, but you are solely responsible for whether or not such software does its job. I was just only surprised that they have zero implementation for anything related to rate limiting at all, as a theoretical problem on their end could cause the API call to not succeed, and then you would be billed anyways.
I only have experience with AWS, so I can't speak for other services, but I was under the impression that this is how all serverless architecture was billed (except with at least some form of rate limiting involved).
Okay. Remind me to never use AWS…
AWS skills will do you well for jobs
Or any cloud service.
Do you need a reminder? It’s a rule of thumb for me.
Never ever use aws
@@TheEVEInspiration yeah because monoliths crafted from open source are so secure. Corporations don't even know how often they get hacked :D
Doing some simple math, if you can make 10,000 requests per second, you can make someone pay $4000+ before the redirect fee in 24 hours. Scale that up to a million requests per second, and that’s over $430,000
That sounds like my plan to get that raise I was denied some time ago. I will become a hero by "optimizing" S3 bucket costs and get that raise.
Maybe the bucket was named: bucket-name
How did you hack my naming convention
they updated to: secure-bucket-name
example-bucket
default-bucket
:(
Fs in the chat for the owner of 'test'
bucket-name-east and bucket-name-west to cover both regions...
Thank you for the warning. I've since removed two empty buckets and moved a third to a scrambled name.
If they want to keep developers on their platform they should probably attempt to not bankrupt them. I won't be using AWS for development.
...so if I understood this correctly: the more PUT requests I send to a company's S3 bucket, the more valuable I can make my Put Options on that same company?
That's a strategy if I ever saw one!
The 2xPUT strat
The wisdom is STONK with this one.
@@craigslist6988 Put squared.
This guy is thinking like a quant. LoL
Have you ever had bananas and rice together ? Your comment sounds like you had
The fact that the cloud make things easier to kickstart, usually make companies forget that this is still a skill/service that recomends a certification to be setup and can have dire consequences if missconfigured
It is even designed to have dire consequences when misconfigured. No limits, baby!
Denial of wallet is my favourite type of attack.
It’s easy to figure out some bucket name.
A. Lookup popular tool that does s3 backups
B. See what companies that tool states as its customers.
C. In your account try to create an s3 bucket using said company name-prod-suggested-bucket-name
D. Get a bucket name taken error
E. You are the king
The Bucketpocalypse is nigh
The bucket was getting spammed even before he took the bucket. So, who was paying for the previous spams? This looks unfair.
I guess before it existed you would get a 404 based on DNS resolution. But once a resource gets allocated to that address then the owner of the bucket gets sacked for the api calls. It’s easy to hate on this but without proper configuration anything can blow up. there is AWS Budgets to keep these in check. The bucket names are unique across S3 so that should trigger some alarms on choosing a more specific name
@@iordanrata Defending this is dumb or you are an amazon employee.
There isn’t much chat about the “default” setting in the product… a default value rather than empty value seems like a horrible idea - is this common?
This seems like the worlds simplest attack vector - literally just let people leak into your storage.
@@danielsharp2402 Thanks for demonstrating the state of discourse on the internet.
> Offered the slightest opinion or idea that isn't completely aligned with yours
> "You are stupid or a shill"
@@iordanratasmall nitpick, a 404 requires working DNS pointing to a working server. What you are looking for a an NXDOMAIN. It could be (i haven't checked) that there is a wildcard record, in which case there would still be requests hitting that bucket. I'm assuming Amazon just ignores any cost caused by this.
I saw this article, and I was astonished. It's bizarre that they bill by non-successful requests.
Benevolent billionaires, because they're all hardworking (28h per day) level 3 geniuses who acquired the second look.
This sounds ridiculous. This completely excludes any client-side flow involving AWS S3. You essentially have to always route everything through another endpoint to make it secure. Otherwise you'll always expose the bucket name one way or another.
That goes for on-prem as well. You'd never connect your samba shares directly to the intertubes.
Sooooo now I am going to use a hash for naming anything on the cloud....
RansomBucket: "Dear company. I have a list of all your bucket names. Send me $$$ or I will send random shit to random buckets."
s3 buckets also use a public namespace for buckets even if they are private. AWS should just append a uuid to create a unique arn for every bucket so namespaces can be reused across accounts. For such a great service this such a glaring hole in the design.
Great service 😂
Its not a hole in the design that would imply its accidental. They do this shit on purpose.
Some lawyers at Amazon should be seriously worried upon hearing about this. I'm not surprised about what support said, since they just do as instructed without care about legal implications. But it's beyond me how this functionality ever made it past Amazon's legal experts in the first place. I can't speak for how things are in the US, but I can think of several jurisdictions (outside the US) where Amazon can (and likely should) get into legal trouble with this. Bottom line: AWS chose to design and operate this, making them at least a facilitating party if this end up being abused. Especially if they claim that this is by design and refuse to fix this potential danger to the operating costs of their clients, with Amazon financially profiting from any such abuse.
They'll just addendum the AWS contract. They'll put it into the "You Cannot Dispute Section," when users sign up or send out a new AWS agreement. Ha.
@@complexity5545 That only works in corporate banana republics, like e.g. the US. In countries that can (remotely) call themselves a democratic state of law, you just can't sign away basic legal liability with a contract (and for good reasons). In fact, what might be most shocking about the whole situation, is how many US citizens consider it normal that companies absolve themselves of legal liability through (consumer) contracts, where really they should be outrages about these companies curtailing basic principles of law (you know, the things supposed to protect people from harm by third parties and not of their own fault).
@@elmo2you Your statement is true. That's one of the fundamental rules of the USA, so that scientists can experiment and kill of contracts. I know men working in basement labs in their homes that make circuit boards as a hobby and small business. We'll tell you our product messed up and suggests to you not to use it via a Agreement Change. If the customer keeps using it, then its on him for the liability. We allow private sectors to break backward compatibility. The government tries to keep backward compatibility. We have a separation of Gov't and the Private Sector unlike most all the other countries on Earth. We don't want the Gov't nagging a company into doing something (unless it is a class action lawsuit). Your observation is correct.
Most engineers, patent holders, entrepreneurs, and business owners want it that way. But we're getting an influx of immigrants (customers) who think the gov't should control the private sector; that's bad philosophy and business in the U.S.A. (and starts leaning more to other gov't and religion models (that we dislike)) We believe it destroys free market. Our economics don't exists to help everyone; its meant to support the smart and the smart are supposed to support the poor and stupid via greed, charity, and some tax. Then the price of the goods and produce is supposed to drop. When people vote in politicians that don't know that the price of goods increases and you take away the motivation for us scientists to innovate and invent (which we usually do out of greed). Sometimes we have scientist that invent out of necessity because of hard times.
But your observation is correct. Amazon screwing up and changing the goal post, will motivated business owners to create new networks.
Your last statement is the exact thing that the U.S.A. does not believe in. We don't believe systems are supposed to protect petty and miscellaneous efforts. We only do Class Action Lawsuits usually with mission critical systems like medicine, energy, food & drugs, and all the other interior things that have a high priority; Or maybe when enough people sign a petition to bring it to a judicial court.
You're correct.
Ah, no, that is straight up BS from Amazon, denied acces should never be charged in any situation possible, not to say sucurity issues related to it, one can just spam unauthorized. Aws itself can run the bots as you said, and who will prove it?
well if that is the case then no one is safe because when you create a bucket you can find out if a chosen name has been taken, which gives you a list on buckets to send requests to, if you have bad intentions.
I think DNS resolve would be easier to automate checking of existence of the bucket
If you have a big aws presence and you're worried about this you can use VPC and apply an endpoint rule to your S3 bucket to prevent any requests from the general internet from hitting it.
hey ok thanks so is the solution a private subnet on a VPC? Would that do the trick?
From my understanding the bucket will still be in the global namespace. So even if you limit access with an endpoint rule you still get the denied put requests against it.
@@jaykay2342 correct... the problem IS that you can still get 404 by hitting the url directly into S3. All files have a url, even if the bucket has no access granted to anyone whatsoever. If someone knows the url , and gets access denied, that's a charge to the account's owner
@@ldandco I cannot BELIEVE that in today's modern internet they actually allow DDoW attacks. This is pure insanity!
This doesn't work because you can't hide access to the S3 bucket through the S3 API. Anyone who knows your private bucket name can still attempt accessing it (without credentials) even if you hide your bucket being a vpc and you will be charged for their attempted access
You can configure eventbridge to fire a Lambda function to delete your bucket when the s3 cost overpass some threshold you specify. It is a extreme measure but good for testing reasons.
Brilliant suggestion.
Or better yet in the lambda generate a new S3 name (randomised suffix), copy over the files, update any env variables and then delete the old one 😅
😂😂😂 comedy!! 🎭
@@lashlarue7924 I am not kidding lol, it is like a extreme solution but it 💯 protects it from happening (obviously if it is a testing for a one person only environment), thats what eventbridge and sns is all about, monitoring, but instead of notifing you, it just triggers an action like firing a lambda and killing the bucket. And of course I am not saying aws is not wrong and it sure should be corrected, it is just something you could easily do
It's worth noting that in the event that something goes wrong on, either from your own error or a problem on their end, and the API call to disable some service fails, you are still on the hook and are liable for all costs incurred. The fact that there is no built-in rate limiting of any form despite all the advanced tracking and logging capabilities and live usage statistics they offer to you, just makes me feel like they are preying on a fuck up to happen so they can juice you.
When you aws s3 cp, you use credentials. These credentials belong to someone with a credit card. You can also configure your bucket so that requester pays. This sounds like an opportunity to make money.
"An Amazon S3 bucket name is globally unique, and the namespace is shared by all AWS accounts." This is insanely bad.
Google Cloud Buckets work the same. Even if it's not shared, it can still be accessed even if you get an unauthorized reply. Apparently it's a billing ding regardless of if the access is authorized or not for AWS. Unsure on Google.
ever heard of DNS? lol
@@jordanmcqueen4714yeah, because DNS is namespaced. You can have example.your-domain, and I can have example.my-domain. Global bucket names are an insane legacy decision because S3 is the oldest and creakiest service. It predates even EC2 IIRC
@@jordanmcqueen4714 Websites are supposed to be public, at least enough to tell you to go away.
@@jordanmcqueen4714 I don't see how DNS addresses their concern?
the moral of the story is do not use AWS
Yes, they do. On the young days of the Cloud.
Almost all providers do this.
That's why on the Cloud "If it will or can cost you something, question it"
Luckily, because of complaints, this is dwindling down.
MapboxJS has the same "denial of wallet" capability. You pay per load, and a load is the library initializing. So your costs are unbounded.
wow, just, wow brute force attacks for the win. denial of wallet. love it.
AWS needs to fix this. This is a huge security/safety issue. Why even use AWS's s3 service if they aren't going to manage basic layer for you?
You could always use an alternative company
Oh they all do this? *rubs nipples* oh what a shaaame
aws enabling anonymous financial attacks is crazy
Wish I could like this video twice. Informative and entertaining.
I got caught out a few times by bucket names being globally unique; this is what makes it possible, as there can only be one `my-test-bucket` in the world, and the owner of that bucket must be getting charged for every test project people make when learning about AWS... I would be pretty confident there are a few hundred buckets like these on every cloud provider that are getting absolutely hammered with dummy, misconfigured or default requests.
My strategy these days is to use a web domain I've paid for as part of the name and I make it long. It is highly unlikely to cause a conflict with anyone else's accident. But on the flipside a malicious attacker who knows my domain could try to brute force the name and issue millions of requests.
at 2:13 as you were reading I thinking the same thing, and then BOOM!!! it hit me, it could be that aws is running an underground operation codename "Generate revenue"
12:44 "Two Buckets , One Name" Lmao🤣🤣
Sounds like a thing Amazon would avtually do.
Still, it's something AWS should fix for robust implementation & thus should be liable for the mishap.(i.e. cancel the invoice)
disgruntled employee just needs to anonymously post your S3 bucket address to bankrupt your company
Or use that to gain advantage for a raise by "magically" optimizing raising S3 costs
Unless the customer made some sort of configuration error that wasn't disclosed in the article, this is 109% AWS's responsibility regardless of whether they own up to it. Only AWS has the ability to control access, which makes it there reasonability to control access and eat the cost of bad requests. They need to keep in mind that ghey sre being paid to provide infrastructure. If the customer ran their own infrastructure they would be responsible for putting in safe gusrds, AWS has the same responsibility.
Also, configuration errors aren't automatically the customers' sole responsibility, keep in mind that AWS defines what options are available, what the defaults are and they have better visibility into unintended side effects of configuration choices, so they can also flag potentially problematic configuration to their customers with explanations for why it may not be what they really want.
Spoken like a non-american.
Pretty sure this is illegal somewhere. You can't charge someone for nothing.
I'd bet that the terms and conditions written by their legal team that you agreed to when making your account probably covers their ass, but it is definitely not the behavior that any sane and reasonable developer would expect
Enterprise quality baby ($100k mistakes are fine)
Ah yes, the Airbnb fee-model. Why stop there though, they could charge per TCP connection, SSL termination, once for each hop in their front-end, per syscall invocation, byes (or maybe bits) of RAM I/O, CPU cache and ALU utilization.
I have some private buckets lying around from a project i did to learn AWS a while back, guess i gotta delete those 😬
The bucket names arent in source control so it's unlikely anyone would find them, but not worth the risk
NEither did the open source tool , but given the probability a match was found. Your bucket name should be cryptic and you can decrease your chances of match.
This reminds me of those SMS subscriptions that kids would get scammed into when they went to download some trippy eurobeat ringtone or something and their parents would find out when they get hit with a massive phone bill out of nowhere.
This is wild. Even from the business perspective what’s stopping malicious competitors from using this exploit to ramp up customer charges to try to get them off of S3?
nothing, and that's why it'll get changed.
What's crazy though is that S3 is literally Amazon's oldest service, and nobody's noticed this until now?
@@nisonaticyep. Heard about this a year ago and I don't even webdev. All hosting platforms refuse to implement any cost capping so all hosting services in one way or another are susceptible to the DDoW.
The Free(tm)* marketplace.
* Free is a trademark of wealthy billiomaires yaught club. Marketplace may not be free and actually rigged via collusion and/or an effective monopoly.
also relates to denied web requests, bandwidth. The thing is that managed web hosts have policies to protect their turn key solutions, aws and other cloud providers should manages capped service limits themselves, this should be configurable. But its not a good business for them. If cloud providers offered prepaid rate limited or capped services they would attract more users and people would do more becuase they would have greater confidence. I think many people are genuinely scared they would get these surprise bills. The only mechanism is to set up cost monitoring alerts and serverless functions to shut down instances and such but this is such a tacky work around. Perhaps if the community rallied thy would provided capped services opt in or something like that?
Cloud is a strange game, where only winning move is not to play 🎮
An issue for AWS is some software patterns will check if something exists first and then handle it, write it there or just report its status as missing. Some systems will see lower bills if AWS makes this change.
"Denial Of Wallet" sounds so epic tbh
I wonder who pays when you issue PUT requests to non-existent or formerly-assigned buckets.
so far I have worked on Azure, you can set a limit on amount of cost a resource can use up until they are stopped, also most resources like blobs and containers have random string suffix as default name somewhere in the path so such scenario are unlikely to occur unless you rename them. but this is a huge security issue, imagine somebody just spam creating S3 buckets and wait for data to collect. Its like a passive sonar and big entities (state sponsored) have enough deep pockets to do so and collect intel.
It feels like an analog of fail2ban might be in order so after x unauthorized requests from the same source it gets auto-blacklisted and generates no more charges
Hiding your bucket name is unfortunately not sufficient to protect against this attack, there are many ways to find bucket names. There is actually no mitigation, other than trying to hide your bucket amongst other buckets.
AWS has acknowledged this is an issue (Twitter) and they will be addressing it.
CLASS ACTION BABY!
Billing of Service attacks
You can simply go to your AWS account and try to create a bucket in some region. If you get message that the bucket exists ... well you know the rest. I think I hit "name collision" working on a project and, if I remember correctly, ignored the message and try to open up the bucket because I thought I created it before. There was some image that made me reaize "oh this is not my bucket it's just open random thing"
"Why don't you put your data in my bucket. Denied but still charged"
I think a woman did that too me once.
In this case, you may not even need a bucket. Sounds like just about any API request would likely trigger aws billing, so long as it isn't on the mains DDoS protect protocols.
And this why I'm a proponent of private-cloud
"Denied but still charge", LMAO 🤣13:10
This isn't limited to s3 buckets. Go spin up a bunch of servers and look at the incoming traffic. I remember spinning up a server and the thing was just instantly bombarded with write requests. It's like the IP address was recycled and whatever was trying to write to that server was still trying to write to that server. And all that traffic was my problem to pay for.
2:00 exactly my thought 😁
In case of a 4xx there is no data other than the login packet, so no one pays for it. Amazon is just stealing.
good to know. From now on I will let a password generator come up with my bucket names.
This a **really** bad situation, but per definition not security by obscurity! Security by obscurity is when the inner workings of a system need to be secret for the whole system to be secure.
In this case only the bucket name needs to be secret, which means it's more like forced clear text passwords, which is bad, but not security by obscurity.
The end 😀"Denied but still charged "
Could be solved easily if they allowed to put limits on how much you are willing to spend before the service is disabled. Unauthorized request is just a single case of something draining your money. I honestly don't understand how one can agree to pay without having any control on how much it would be.
It's crazy that DDoWs are a thing, and how easy it is to (theoretically) bankrupt someone.
I can come up with one really really easy explanation as to why a botnet might be scanning your bucket, they're looking for places to stage malware.
If you have a service discovery architecture, how about renaming buckets on a schedule and publishing the new name. Yes, you would need an overlap period as your services picked up the new names but that is well understood pattern.
It is still security by obscurity, but it will aid with the problem of damage caused by a disgruntled employee.
Not sure about AWS, but if it's anything like Azure, there's no such thing as "renaming" a bucket. You'd have to create a new one and copy the data. That's gonna cost ya.
Doesn't S3 require for web-hosting that the bucket name be the same as the domain? So if you know some website is hosting some static content out of S3 you could run up their API bill?
Haha, I was thinking the same thing as I have quite a few set up as s3 hosting.
that's why one should consider to hide S3 behind CloudFront (for public requests), constant requests gets cache in CDN and reduces READs on S3.
yep depending on the setup.... the "classic" way of hosting from a bucket (not necessarily via cloudfront) then yeah
Does they say they are charging you for the put or the put request? If they are charging for the put and they aren't actually putting the data in then it is fraud. If they are charging per request that isn't authorized I would consider that a massive security failure on their part. No unauthorized party should be able to impact your system and cost you money.
Always get them to mail you a bill. Any bill that is fraudulent that goes through the US mail can be charged with mail fraud.
Finally a short video
When I was messing with some cloud computing stuff in school, I was asking my professor why AWS was so cheap, and he said that it only seems cheap at first until you get hit with many random unexpected costs and then gets real expensive out of nowhere either due to misconfiguration or malicious attacks. Later on when I was looking for a server provider for some hobby projects, I ultimately chose a standard fixed-rate VPS because AWS' solution to DDOS attacks was essentially to keep all services up and running no matter what and then charge you for it, whereas my VPS has unlimited data transfer at gigabit speeds with hardware that exceeds my needs. I think the extra money I coughed up is worth the peace of mind knowing that I won't accidentally shoot myself in the foot with a huge bill. That isn't to say that I won't ever use them, as I found their APIs to be quite easy and useful if you don't know how much computing power you need or if they can vary greatly from moment to moment.
So many companies have public S3 bucket links on their websites. Often times specifically to avoid DDOS / Denial-of-wallet on shit like vercel. That shit is so broken
Simple solution: F**K AWS S3 storage. Use any of the better cheaper services
I'm old, still new to the cloud. Stuff like this makes me seriously consider whether to even bother. There should be certificates and firewalls for this!!
It's already been fixed on May 12 this year... but yes, definitely it was crazy something like this happened until now...
This bro is a good dude for not just allowing writes to the bucket. He could have stolen a ton of data from these companies who left the s3 bucket placeholder lol
he'd have been charged a TON of money. The data has to be valuable in order for this to be profitable.
Simple, use a proxy for developers and treat people who have access to true bucket name as if they had nuclear launch codes...
The buckets are still created in a global space. I.e. someone knowing your bucket ID will still be able to spam calls to its URL. Obviously cryptic bucket names hidden behind a proxy have a lower chance of this happening but not eliminate it.
So, the stance Amazon is taking is that the only form of security they offer here is obscurity.
So all it takes is name based collision?
True enough AWS cannot pay out the customer for every misconfiguration. But a selling point of using these greedy enterprises was that they acted as a buffer that provided the most basic of protections. If you need a macro understanding of how the protocols work anyway, you may as well detach from the service and do it in house.
in Brazil, one of our main bank's credit card machine has a "bug" that, after 3 months, it raises the use fee from something like 1.5% to 6%, even though their contract with users says it would go from 1.5 to 3% (numbers are approximate cause i dont remember them fully).
And, unless the user actualy audits what the fees are, they spend the rest of their life paying double to the bank, and the bank WILL NOT inform you of this "bug" and will only rectify it if you point it out to them.
It makes them money, so why would they fix it?
I suppose its the same case here
Will look whether I have any active buckets and never order anything from Amazon again because this is so gross 😵💫
I had that experience with the Rings of Power
Actually, if Amazon allowed enough people to misconfigure S3 buckets by name, they'd actually face enough backlash to automatically generate a random uuid prefix on the bucket name. That sounds like the way to go to me.
It seems that if you use signed urls you almost have no way of avoiding this charge vector.
Bucket names are unique and global. Meaning, figuring out bucket names is a matter of trial and error using the cli or the most enthusiastic ones, the console. Meaning anyone that knows "something" "something" about your company can brute force bucket names until they get one they can't create, and voila.
Yet, one more reason to avoid AWS. As a small business owner, this would be devastating to my company.
This sounds like the making of a large class-action lawsuit against AWS.
It's funny the more you abstract in the cloud for customers, the more scary it can get for them.
This is why AWS is so unfriendly to noobs, how can an aspiring dev afford to make these kinds of mistakes?
Well it's the basics: To exploit the weak you first need to find them.