AWS S3 Is Having Some Serious Issues...
Вставка
- Опубліковано 1 чер 2024
- The slow death of S3 has been happening for awhile now. As AWS's first service, it was kind of inevitable. So much so that I started an alternative.
Speaking of that - uploadthing.com ;)
SOURCES
/ how-an-empty-s3-bucket...
/ 1787844682216792163
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏 - Наука та технологія
UploadThing is a cool alternative, but it lacks guides and documentation. It overfocuses on streamlining and catering to FE uploads as opposed to being what it is - an object storage, which is first and foremost a backend concept. Basically doesn’t live up to it’s promise of making it easier for us
Do you think if it did have those it might live up to its promise or are you not sure until it exists?
I think you may have missed the point a little bit. I found that integrating file uploads and linking frontends to storage to not be straightforward; I believe adressing this issue is UploadThing's goal.
@@leetdavid I love uploadthing for exact that reason, but i also think that the fact that nobody really knows what uploadsthings goal is (except for Theo and his team i guess) - because we are speculating here - kind of proofs MrManafons point a little bit. At least in a sense that it lacks guides and documenation to explain what it does, how it does it and therefore give us a chance to better understand what to use it for.
You do realize it’s just using S3 in the backend ?
I have not been able to understand how S3 is hard to use... seems pretty straightforward. But oh wells!
S3 stores half the internet.
AWS S3 is not hard to use. Hard to secure? No, IAM is easy too. This charging for access denied is shameful though.
Got fixed, afaik.
About 20 years ago a bug was found in Photoshop where the opening of a file that lied about its total size could be used to execute code embedded in the file. This was the reason Intel introduced memory partitioning (no allowing writes to code partitions) because if Adobe could screw up like this then anyone could.
Apparently we've learned nothing 😩
Free tier do not get protection from being put in a situation where they have to pay something. The reason Amazon doesn't fix those issues is because they account for a part of their revenue
The reason for it is that they don't have to offer services at a loss, like the smaller cloud service providers have to. AWS is the biggest in the market, so they can bill you for your traffic like any other hosting provider.
there should be an option like doesnt force u to use a credit card and just lock u in free tier
8:37 Theo: "S3 is on such fragile ground that they're scared now. They could be scared of R2 or Upload thing"
Blud really thinks that AWS is scared of Upload thing 😆😆
Of course he does, Theo is the founder of PingLabs, the company that makes UploadThing. He even uses his name to promote one their pricing tiers with a "Theo's Fav" tag on the checkout page.
His opinions of UploadThing and any of its competitors should be taken with a gigantic grain of salt.
You have to be delusional about your own products in order for them to succeed at scale in the tech space
Uploadthing still uses S3, so they win anyway. And he just wanted to make some marketing for Uploadthing. I don't think they care about Uploadthing, though they could be scared that more and more companies are trying to improve file storing.
He talks about pricing
Better use Linode object-storage, which is s3 compatible but without a lot of the bs. With rate limiting included.
You think S3 is dying? As someone who does multi-year multi-million dollar migrations, no it's not. What a weird take.
Something like S3 obviously doesn‘t just implode and dispear over night. But when better alternatives become available it absolutely can slowly decline over a long period of time and ultimately become obsolete. Dying implies its not dead yet. And you can‘t deny it has serious issues. Working with it directly is an absolute pain. There is a reason there is an entire Industry building easier to work with abstractions on top of it.
@@nymez6968 And just like our ol pal Theo "pay me for dark mode", I think you grossly underestimate the sheer size of s3. For frontend devs, sure maybe upload thing works for them. Great. AWS isn't concerned with that, AWS is concerned with continuing to be bigger than the next 3 or 4 cloud providers combined. S3 is the bedrock of the entire platform. You don't have EC2 without S3, you don't have lambda without S3, etc.
Again, absolutely ridiculous insane uneducated take.
@@Botanical4038 You think size saves you from going away or being replaced, if you fail to stay competitive? It has happened over and over before that something that absolutely dominated a certain market ultimately went irrelevant. Unix used to be the dominant OS and Nokia used to rule the mobile Phone market. Then Linux and the Smartphone happened. IBM once basically owned the PC market. Where are they now? There is no such thing as „too big to fail“.
@@nymez6968 you know what random person on the internet? You're right. I'm sure S3 will cave in, it's only a matter of time. I hope that makes you feel better.
So I'm going to go back to living in reality where money matters, you stay... Wherever you are. Sound good? Have a nice day! And buy $GME
Edit: nonono, don't go editing and deleting messages after I've responded lol
@@Botanical4038 1. I didn’t delete anything, i only added some things that came to my mind later. 2. Where did i say I consider it a bad idea to work with or make a living off S3 right now? 3. Why so angry? I only made an argument why i think it it is POSSIBLE S3 COULD die in the future.
I'm an R2 Cloudflare guy nowadays
Almost every blob storage is built on top of s3 so I doubt it will just go away
Dark Reader please
not just sentiment shift, with that "this is expected behaviour" and "we don't plan to do anything to fix this" is enough for there to be a major legal issue for amazon should someone deliberately find someone's bucket name via whatever method and just ping it with requests. Basically you are being forced to pay for a servicew that you are not using and did not request, this could be considered massive fraud on AWS's part and make them liable and possibly even criminally guilty under the CFAA.
heck aws could ping it themselves and get huge earnings
Isn't uploadthinq just an S3 wrapper, which means you forward requests to the underlying S3 repositories ? I guess you could have some checks in place to map uploadthinq ids to an S3 bucket, but the fact is if someone sends too many requests to your endpoints, your whole website will likely fail and you will have to pay for the charges
Google Cloud Storage is great.
Great node SDK. Really enjoying it so far.
Pricing is also more transparent than S3.
I have a HP Proliant DL360 in my garage, running Debian Buster, and the LAMP web stack with an outdated version of PHP with built in RCE execution. Fight me.
Paying $80/mo for power and countless headaches if the hardware goes down is almost worth not having to deal with this cloud provider BS. I’ve dropped all my big cloud providers and run almost everything off one VPS…
@@chockman3833 My power (for that server) is about $10/month. Hardware going down is a problem only for people who've never used servers, I've got a rack full of servers from Dell, HP, and Cisco and I have to replace a hard drive once a year or so. Pretty much everything I run is HDD based, except for my MC and Factorio servers which have two SSDs in RAID0 for the world folders, they fail pretty much every 3-7 months (depending on server traffic). Good for fast random IOPS which is why I use them.
VPS's are great, I have one myself, but frankly if you have enterprise hardware you pretty much won't have any major hardware headaches. The only things that ever cause me problems are cheap consumer items like the SSDs I mentioned failing. I also buy all my enterprise stuff used off eBay.
@@chockman3833if you can run everything from a single VPS you didn’t need a cloud provider in the first place.
I think they're addressing the security issue because they absolutely MUST. It's such a serious security flaw that is now widely known about that they will destroy their entire business if they don't somehow address it.
This security flaw is INSANE. The S3 namespace is global, so anyone who knows your URI can simply spam it to hit you with a Dedicated Denial of Wallet. That is absolutely CRAZY. To my understanding there is no way you can prevent this by simply putting up a VPC or security group, because although that would restrict access to data, it wouldn't stop the request itself from being made for something other than 200.
AWS is huge but there are alternatives, so this is something that they either fix and deal with ASAP to prevent the devs (very few of us being dimwitted) from leaving for some other solution. I am seriously slowing down several projects I have to wait for this to get fixed, I simply can't build a business around something that can be hit with DDoW, my customers will not accept that (because they're not dumb either).
If you colocate a server with metered bandwidth, it's the same deal. If someone sends 1pb of data your way, you're going to be billed for the ingress. This is not a security flaw with S3, it is simply how most infrastructure services work, they are billed on usage.
Backblaze B2 or Cloudflare R2. Both great for most use cases and direct drop in replacements.
Yep, I use B2 for long-term storage with zero egress through Cloudflare, and R2 for large files in front of their CDN - cheap as chips
I’m catching up to learning the cloud and honestly this down that happened was a great thing got to see first hand. Thanks for the breakdown because that situation was hysterically crazy
My previous employer used Bunny CDN for object storage and CDN. They are insanely fast and somehow dirt cheap, for both storage (close to 100 TB of data) and delivery (over 1 PB/month). I think it's a nice alternative to S3 and R2. I also used Backblaze for long term storage, very good pricing and decent performance when I needed it.
Praise be unto Ceph, our S3-compatible saviour
If UploadThing is built on AWS how do you easily calculate the space used in a bucket and bucket path / sub folder?
If you have tested switching to other cloud providers what would be your 2nd and 3rd choice after AWS?
so S3 doesn’t log by default all activity that it bills . wow
Welcome to cloud, my favorite this year so far is getting IPv4 charges from AWS after never having incurred them for over a decade of use.
Because IPv4 addresses are getting scarce
I got an email explaining that change a number of months ago. TBH it's only fair at this point given ips are so limited.
This s3 4xx thing is a huge problem for me.
My buckets are short and common out of necessity, and obstructing them will have no result.
It's insane that S3 doesn't have a 3xx/4xx back off built in.
I think it’s a good thing, we have been stuck with v4 so needlessly in many places because nobody feels the need to take action.
Cloud newbie. Pardon my naiveness on the subject. I wonder why azure blob storage was not mentioned as an alternative
It is an alternative but I don't think it is S3 compatible meaning your existing code won't work by just switching the endpoint to Azure. A lot of object storage services make a point of using S3-type requests so that apps can just swap over the endpoint that the app points to and the app will still work.
Im probably misunderstanding so if someone can please clarify that would be great; if S3 bucket names have to be unique across all regions within a partition, how is it that this developer was able to create a bucket with the same name as another bucket already existing by the company that generates their backups?
I work in a company that’s spending houndreds of thousands of dollars in a recent partnership with AWS and I’m definitely gonna talk about this with the cybersecurity team cause we will be really vulnerable to this exploit causing millions of dollars to the company and probably could force aws to change something about it
Supabase offers similar pricing, plus you get a database, auth management and more. Best deal for projects of all sizes
upload thing with react native ?
It is hard to block it, since the bucket name is in the url you send to clients.
Whoa!! Congrats xyzeva on another shout out by Theo.
Hi Theo, love your video, very informative.
Could you please always include links to the sources!!!
Thanks a lot.
Why not use Backblaze B2 over AWS S3? B2 is like a quarter the cost.
well, localstack, minio for poc?
Maybe should give Cloudflare R2 a try
This is insane O_O. But I think we all know why it hasn't been addressed yet. It was very convenient for AWS to keep it this way.
This is the way how monopolists work, unless they are endangered, they don't give a f*k.
However, I think Google is the worst, usually if something happens they won't even let you contact them.
Btw. it wasn't a clickbait!
Lets agree most of us come to Theo's channel because he covers "latest" things going on and not for his takes.
For a long time, he consistently gives L takes 😂
upload thing is an s3 wrapper
I don't consider that title to be click bait to be fair.
This issue isnt limited to S3. You get charged for unauthorised requests across almost all AWS services, because of the authorization request
how is uploadthing a competitor to s3 when it uses s3?
Well i got my answer of why not to use presigned URL's to save ourself. Waiting for the explainer video.
Google storage and azure is an option... s3 is not like the only option.
Upload things pricing is actually good, man if i had to store 100gb someday for a project that means im already rich. Ill gladly spend 10 bucks for it
People told me for 10 years "just put everything in S3", nah bro
digital ocean do good object storage called spaces
I don't think S3 is that hard to use. The real problem here is that you think you're on free tier but then magically you're paying thousands of dollars without any notice.
"issues"? I think that is working as designed: the most profitable way.
I usually like your content, but the notion that S3 is dying or that Upload Thing is even a distant concern for AWS is frankly laughable. Your take here seems to hinge on a very narrowly defined set of use cases that don't align with the broader spectrum of areas in which it is used. Talking about pricing and GBs is pretty funny too, 100GB is nothing in terms of S3 scale.
Why not move to Azure Blob Storage?
AWS S3 is such a favourite choice among typical managers who don't understand a thing about technology.
I have worked with many companies where management believes AWS is better because everyone is using it and it's popular and no amount technical reasoning would convince them otherwise.
It's one of the most political decisions made in the tech world by non technical management guys!
damn i was interested in learning how to use S3 over the summer
Same here 😂
Do it. It won't take you a summer and the number of companies using AWS and S3 is not going down over this.
You should learn it.
You still should
you should really learn it considering it’s an industry standard and many other file hosting services are S3 compatible so if you learn S3 once you’ll have learned a bunch of different storage providers
You can repost a video with this title every day, and it will never stop being true
This is why I would NEVER recommend any cloud service for a hobby project. There are cheap VPSs starting at $10/month
S3 has been around since 2006 and they have made tons of mistakes….so many that upload thing has learned and won’t make them. However, s3 has paves the way with out its mistakes all new services won’t be appreciated
LOLOL 100GB $10 a month?! You can get a 2TB HDD for 30 bucks these days.
S3 means Sophisticated Storage Service
Ye
i just subscribed to you!
A dude tweeting on it doesn't mean they are scared. Its simply them trying to be open and acknowledge it. Given the other recent situations, they are likely well aware of and they want to be seen as better. Try not to use so much hyperbole.
Just kicked out s3.
Cloudflare R2
LOL uploadthing took > 15sec to load not a good sign. But doesn't support serverless open source, so not relevant anyway.
OCI Object Storage is a much more secure and cheaper alternative to S3
Hmmmmmm
@@umedzhonizbasarov9049 ?
Btw last few weeks i unsubscribed and not watching your content, mainly of because its no authentic as before not cool. You always doing clickbait though you dont even need it for the videos i dont like clickbait
Yeah I’ve also lost some interest with the recent stuff. It’s a shame
Second
Last comment
Lmao "we charge for unauthorized requests too, that's expected behavior" is killing me
First Comment 🥇
Honestly..... Skill issue.
"what did i learn from all this?"
lesson 1: dont use aws
lulz, aws is trash.
Im your biggest hater
Aws: has been known charging thousands for no reason
Random guy: uses aws and gets charged thousands for no reason
Random guy: surprised_pickachu.jpg