Life After SQL (EdgeDB Is Fascinating)
Вставка
- Опубліковано 27 тра 2024
- Disclaimer: I am an advisor for EdgeDB, I do not ship it in production (I still build my services on PlanetScale)
EdgeDB is getting pretty dang exciting. The idea of a "more relational" db is something I hadn't thought about enough before. Excited to see what they cook!
SOURCES
/ 1782485844814647402
www.edgedb.com/blog/we-can-do...
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏 - Наука та технологія
BTW I'M JUDGING THE EDGEDB HACKATHON. WINNER GETS $5k. hackathon.edgedb.com/
i'm a long time backend sql guy, so i was skeptical about anything said to replace sql, especially after the past decade of non-sql positive hype (typically presenting the benefits of something without the downsides). as you know, engineering decisions are almost always about trade-offs, rather than something is completely awesome/sucks. however, your excellent video here piqued my interest. i'm going to do diligence on EdgeDB for an upcoming project !
To anyone who is in the know… How do these basic queries scale and what is the experience of tuning queries? Having loving syntax is great but I’m genuinely curious to see if this is just a “rapid prototype” database that you’d drop if you get any traction or advanced requirements
the name is horrendous.
clicked on this video wondering why would someone need an entire DB runnign on the edge....
Lol, do you know that Edge also refers to edges in Graphs ?
I didn't :') @@erenjeager1756
Like a lot of things in our industry, these terms become horrendously overloaded. But I get what they meant: edge as in edges and vertices, i.e. graph theory.
It's not their fault that Guillermo Rauch spams the word edge too much on Twitter.
not 100% but some would be nice, replacing redis for me. in memory postgresql is cool
THEY HAVE DOCS FOR RUNNING ON DOCKER OR BARE METAL LETS GO SELF HOST GANG
Been using self hosted dockered libsql with their client and had speeds surpassing drizzle but now I'm going to have to try this out so I don't have to write sql
@@Frostbytedigital what’s the point of dockerizing libsql? i know it allows for network requests but isn’t the whole point of sqlite that its embeddable?
do i get all the GUI dashabord query builder and other stuff provided when i go with their cloud solution ? thou otherwise what's the point?
@@joe_ervin10 sqlite is embeddable libsql makes it shareable; you need a server to fetch the initial db to clone for an embedded user for example
@@GreatTaiwanI also want to know if we we get a GUI dashboard while running a dockerised solution.
It's Like GraphQL without the GraphQL burden
And with GraphQL 😂
the dgraph query language is limited like graphql, this looks more powerful.
It's like graphQL, but with edging
Gonna have to disagree about including its built on postgres on landing page. As a user experimenting with edgedb, that's a piece of info I'm glad to have. I know the core db engine is rock solid because postgres has been around so long. I can guess what features they might easily add by looking at postgres extensions and features. I know I might be able to make some existing tools interop with it.
They actually allow READ-ONLY access to the underlying Postgres Data, so that you can use your favorite Postgres tooling for some complex visuaization or other tasks.
SQL 1989?!? I was using it when I was in college in the early 80s and using Informix and Oracle SQL in '87 when I had my first job. Pretty sure SQL had been around a while before that.
Yeah it's from 1974. I can't think of what he's confusing with that was introduced in 1989
Theo in the impet to pump-up his friend's DB made a poo-poo.
Theo makes a lot of mistakes in his videos. I keep that in mind when watching these.
According to Wikipedia, it first appeared in 1974, so 50 years ago!
edgedb is AMAZING, solves all the problems old SQL systems have, even have integrated migrations system. the language is very comfortable and modern. the documentation is amazing. and it is a front to postgresql which itself is obviously amazing. I have no idea why it isn't all the rage.
Because it was still in beta last time I checked a few months ago
Truly the best Edge(ing) experience (tm)
Groan. Also (tm)
No neo4j mentioned?
would have been nice to see the movie example in Cypher
ikr? this is basically just an aura db with a graphql layer on top (but obviously, is handled a bit nicer)
Perhaps because Neo4j is not a relational DB? It's a graph DB. 😊
I don't really get why people don't like SQL. Personally, I find it fine to work with. What I do get is that there are different kinds of people out there than me, so I'm happy this is a product people can try out and use if they like it. I'll probably be stuck with SQL for a long time cause of my job, but I'm also happy to say that I don't really mind that at all.
One thing that does get frustrating is, as in the example Theo points out at 13:41, different implementations of SQL have different ways of writing it, so you can't just write SQL, you *have* to write MySQL or PostgreSQL or Oracle SQL or SQL Server SQL, etc.
I've used edgedb since I saw it on one of fireships top 10 dbs videos way back and have absolutely loved how they've keep making it better and better. 10/10
andy said sql ain't going anywhere no matter whatever new gimmick comes around.
I mean of course it’s not, but that doesn’t mean this isn’t cool
It'll go away some number of decades after something else has high enough acceptance that people aren't choosing it anymore. Whether that's two decades after next year, or 10 decades after 200 years from now, it'll probably happen eventually.
@@DavidLindes“There was 10 competing standards.”
“We remade the 5th as the 15th praise us.”
Of course. It's way too big and integrated into everything. But why not build the new stuff in the new thing, assuming it's all upsides?
But I guess time and trial will tell if it really doesn't have significant downsides.
SQL will continue to exist for a long time, just like Cobol.
As much as I love SurrealDB, I must say, this shit is HOT
I used EdgeDB when it was quite new, and it was already amazing back then, albeit with some caveats in the query language. Glad to see it grow and becoming better!
Self hosting it is super simple, which I very much appreciate. Main gripe with it is the lack of a language server for .esdl files. It has syntax highlighting, but no autocomplete or formatting, which is very annoying.
Working on that.
@@YurySelivanov I’ve been waiting eagerly since the Rust parser blog post. Good to hear it’s still being worked on. Looking forward to it.
Same. And there is one specific bug with their eqdeql js lib. But that's about it
"they are not a sponsor" [...] "if they suceed, i get a bit of a payday" ah yes
He said upfront that he has been advising them, so yes, presumably if they do well as a company, then he'll continue advising them - getting a payday.
Rather than the video itself being the source of the funds, his advice is the source
This isn't the dunk you think it is
@@Spiker985Studios Doesn't change that he first intended to make it seem like the video isn't biased by saying it isnt sponsored, even though it is obviously is biased considering that.
The first line of the description is a disclosure. A verbal disclosure is at 45 seconds, just after the "hook"
How much more disclosed does it need to be?
@@GoldenretriverYT He never said that the video isn't biased, it is. He just said that it wasn't sponsored since it isn't.
So, if it's based in Postgres... It's just a wrapper over sql?
yes
The whole market is regressing back to the SQL api. This is a bold move cotton.
Is not a bad idea having new alternatives, the catch is that SQL has decades of development and it's availability is unparalleled.
Also another channel, Asianometry, is covering the history of SQL in a series of videos, and how its ideas are the response to the hierarchical databases of the time.
the fact that they have sdks for almost every language is so cool. I have been wanting to write my backend in go for a while now but there’s just not enough auth libraries that support it. I’ll definitely have to check this out
I would rewrite "based on PostgreSQL" to be "powered by PostgreSQL"
"Based on", to me, implies that the source is "based on" that thing
"written on top a PostgreSQL foundation"
I am building a developer tool that uses EdgeDB as a core feature. Glad it’s getting attention.
12:22 null is a fundamental set in set theory. SQL should provide support for null since relational databases are based on sets and set theory. Why is everyone up*set*? 😊
Exactly
Null is not fundamental to set theory. Unless you mean the empty set Ø. But that is just the initial object in the category of sets. Most of the time null is problematic since having it means you are no longer in a boolean world but need to acommodate three logical values: true, false and null.
There is no null in set theory. Null is a design 'mistake' in computer science.
@@gencade2504 Read the linked in description article. It talks about the creators of SQL themselves saying that NULL was a mistake and breaks the relational model. It was a mistake that was too late to fix though so it stuck.
what about surrealDB?
Surreal is too new, tries to cover a LOT. and to implement features, they must build it almost from scratch.
I've tried it for a while and IMHO it's awesome! especially integrated DB migration, and query language looks good too.
SQL 1989... you may want to check with your video editor on that one...
I'm a big fan of EdgeDB(just have a glance and their Book in the docs, it's amazing 🤩), i truly hope they would get much more traction over the coming months
The only thing that i'm really not happy about is the polymorphism implementation. Unfortunately folks went the abstract\extends\inheritance way instead of union types, i think this was a mistake. Yes, i know that i can model union types, they got so many things right, but not that "feature", imho.
Very satisfied with edgedb after using it for 17 months.
It's a force multiplier. Never been this fast in my 20 years in the biz, and never written so few bugs with any other db.
Name is fine, its graph edges but now that word is pretty much hijacked by the edge so yeah.
10:20 how do you snapshot your schema changes and manage migrations??
So it is just an abstraction over plain SQL? And what is stopping them from doing a redis?
Pretty much seems like it. It is a language neutral set (or more likely, a subset) of SQL, that is a bit more convenient to write. I would hate to be the one to hit the capability wall... With ORM you can at least fall back to raw dogging SQL.
Someone doing a redis sometime in a distant future should not really stop you from using it. Nothing is stopping noone from pulling a redis.
am i wrong in thinking query vectorization, columnar storage, distributed query engine are the future for sql. what's the problem with sql that we always see ppl asking what's next for sql... lol a more improved sql
Prisma frustated me soo much that I love EdgeDB since day 1 I started using it
I appreciate the disclaimers, thanks Theo
can one migrate from sql to edgedb and the other way around too
The big use cases that I always have to keep an eye out for are:
* DIstribution - can I have this in pods across multiple datacenters?
* Stored Procedures - can I wrap my queries in sprocs to satisfy security requirements?
* OpenAPI (Swagger) - Can I generate my types against a JSON schema for interoperability with my ReactJS UI?
* Driver Resource Footprint - How much of a hog is this? ODB uses 4gigs for our biggest queries in spikes, otherwise it just sips at memory
How does it build on top of PostgreSQL? Is it a hard fork and changed the actual source code? or a layer on top. What do they get from PostgreSQL that they couldn't get anywhere else?
What I like about EdgeDB is that it’s very accessible and easy to learn.
The beginning has that feeling of watching a movie with somebody, who have seen it and they constantly pause to explain some little stuff
orm is a footgun - if you're going to use postgresql then just make functions (you can return json) - also that SQL example on edgedb website is (purposely?) needlessly obscure with subqueries and could more easily/legibly/compactly be written using common table expression
Alas, I doubt you'll get too much support for the idea that ORM is a footgun. I've been apologizing for my colleagues and I accidentally unleashing that nonsense on the world for a decade and a half. Nobody listens because they don't know what else to do.
And here I'm writing almost all my SQL in stored procs and functions in MySQL Workbench. I feel old.
The real problem with NULL in SQL is that there's only 1 kind of NULL, unlike JS that has 3! And that solves all problems with NULL. ALL PROBLEMS!
It honestly looks pretty cool, and if I had a project that I needed to use a DB for that didn't already have a ton of documents/values put in I'd definitely try it out. I've never really used a relational DB since I got started with Mongo and my couple forays into MySQL didn't go far. Even my attempts at using Prisma weren't too great because it doesn't seem to be as plug and play with Mongo as it is with relational databases, although it could also just be the implementation that's the issue
Looks pretty sweet, assuming the Edge in the name is for edges on a graph?
Correct.
I am sold already
Can it be supabase alternative?
Now I'm lost, do I try to use EdgeDB for my new side project or do I stick to drizzle/mariadb?
Looks cool! I love that it will probably be easier to master than SQL, given all of SQL's edge cases (pun intended), and it's awesome docs. I doubt it'll eclipse SQL cause SQL is SQL, but I can imagine it being the sorta popular alternative people thought Mongo would be.
So, this is just JPQL (that is, Java/Jakarta EE JPA's own SQL-like language) at the source-level (database, not the app), basically. Nice.
I would have liked to know more about the implementation details of EdgeDB itself - is it just a wrapper around a Postgres DB with some additional functionality? Is it just a different API that abstracts standard DB behavior? I'll look at the docs of course, but I would have enjoyed for this video to explain it for me, since you have had such intimate knowledge of the project!
Edit: found it - though unfortunately the docs were no help at all. Look up the talk "The architecture of EdgeDB" given by Fantix King, very helpful! TL;DR: EdgeDB runs their own middleman server in between the Client and the Postgres DB which provides the added functionality
Yes.
Edit: saw your edit
Pardon if this is a bad question because I don't really understand how the architecture of databases work, but does edgedb's query language build SQL queries under the hood, or is there some sort low level functions / Machine code that edgedb calls directly?
@jon1867 No it's not a bad question at all! Never be afraid to ask about things. For what it's worth, both this video (sorry Theo) and the EdgeDB docs itself don't explain the mechanisms of the technology at all unfortunately.
You've basically got it down. As far as I understood the video "The architecture of EdgeDB", EdgeDB's query language compiles into EdgeDB-proprietary binary that gets sent to their custom architected middleman server, that then interprets that binary as SQL, then massages it to the native Postgres binary protocol, then after all of that, it communicates to just a standard Postgres instance.
The main time saves that EdgeDB achieves is from a caching layer within that middleman server to store general queries + prepared statements, their own binary implementations + using native Postgres binaries which save data over the wire at every touch point in the stack, writing part of their EdgeDB server with Cython and Rust bindings for the most intensive processes, as well as utilizing a more efficient communication protocol to their own server + to the Postgres instance via uvloop
The biggest gap in my current knowledge of the technology is that I don't know _how_ the Postgres database is architected after compile / migration time - there are methods of simulating a Graph database e.g. neo4j within Postgres, but I'm just guessing this based on how frequently they use the term 'graph' in the docs. I'll probably spend some time tomorrow getting something up and running to see actually how the database schema looks after the DB has been instantiated
@@TylerJBrown192 One of the questions that raises is, could edgeQL be used on an existing postgres DB with relational tables? Or would it only work on a database built with edgeDB specifically?
If so I think adoption for a tool like this could actually be pretty massively viable. And with performance benefits like that, even if people don't wholly drink edgeQL koolaide they could sprinkle it into important parts of their application.
@@jon1867 Damn, good question, I actually had a very similar thought! But it came from me worrying that if I ever brought in EdgeDB to anything I was working on, I just signed myself / my team up for an insane amount of technology / vendor lock-in haha
There's been tons of "it's coming" comments I can see from the team since even 2019, but I wasn't able to find anything concrete about how to generate an EdgeDB schema from an existing Postgres database. All of their Migration / EdgeDB CLI functionality seems to only build upon an existing EdgeDB schema, not to be able to generate a schema like many standard SQL ORM-esque libraries can
No... keep the name, I can finally say that I'm 'edging my database' (gen z brainrot is catching up to me fr 🗣)
So first the vercel stopped edging, just for my database to do so instead
So its just a wrapper for SQL... meaning you are still using SQL.
Also (slightly related) I am sick of people saying NoSQL is better because of performance/scalability. You can scale SQL lol. You pick NoSQL if it fits your data model better.
I’m really excited about EdgeDB. My biggest hold up right now to using it is that it is new and I’m worried about running into edge cases.
Also, I wish I knew a little bit more about the deployment story. Can it be backed by a postgre instance in RDS?
Finally we can edge in peace
I guess, my concern, when considering switching to a different database is that I'd have to learn how to write queries safely. Like, with SQL you have the potential for SQL injections and there are techniques to mitigate it like prepared statements and such. Is there documentation on how to write EdgeDB queries safely?
Unless you concatenate EdgeQL queries with data there's 0 chance of any injections.
How would it handle very large sets? For example, 1B objects that are related to smaller groups of objects. E.g, (Measurement (billions), Measured Variable (hundreds of thousands), variable tag (thousands))
so it's not a database and it does not run on the edge - Edge DB is the perfect name! 😂
this is more a "backend in a box" and less a DB, isn't it? you can't even access the PostgreSQL underlying. (which would worry me. 🤨)
Awesome content as always, although there's something I don't really understand (and the EdgeDB site itself doesn't really clarify) - the "N+1 problem" is presented here as being a *syntax* problem - as in, it's a pain in the arse to keep having to do sub-queries to get the data you need across multiple tables.
But isn't n+1 actually a *performance* concern rather than syntax? e.g. the classic example, if I have a blog posts table, and an authors table, and I want a query that returns some data from each, a single query will need to run a sub query against the authors table once for every blog post that comes back. I don't understand how edgeDB solves that part?
What do you think about Convex?
Convex is really good if you need websocket-like speed from updated data (like a chat app) but it falters in writing complex queries
Is it faster than raw SQL? I’ve tried ORMs and query builders… they’re severely limited and slower than raw SQL.
It is faster than ORMs but can't be faster than SQL since it compiles to SQL
tackling sql is a must, but usually leads to failure. edgedb seems so promising in this path.
This is going to have huge success on junior devs and some companies that will buy the pitch or ripe some bens. sql is not good enough ....
The SQL code in the comparison wouldn't pass code review 😕 What happened to steelmanning?
Unfortunately edgedb docs just give me "Application error: a client-side exception has occurred (see the browser console for more information)." on mobile... Can't see what I want to see.
The lengths this industry will go to avoid learning SQL...
Between NULL and timezones, also feeling making sand think was a mistake
'Edge' is the only reason I clicked on this video... It's not too late to change the name
EdgeDB seems to respect and implement aspects of The Third Manifesto, which talks about doing relational more properly in a DBMS than SQL does.
this is not an advertisement trust me bro 100% legit pinky promise they are all my friends but this is for real you know me i mean you don't know me irl but whatever, suscribe!
I'm guessing Drizzle use functions rather than a big class holding all the possible methods for Tree Shaking
I don't understand it, if I have a postgress instance running can I just use Edgedb?
So this is basically an ORM for Postgres with extra steps?
Great story about SQL and what may come next. Please lead us further down the path!
I spent most of the last couple of days playing with edgedb and it's api etc. Brilliant! haven't seen something this interesting since I first discovered Svelte 👍
Hmm? do you mean Graph Databases? I think Graph DBs like Neo4j are indeed challenging the norm, especially with Cypher language that is being adopted by other Graph DBs, it's just not going mainstream fast enough...
I love the part 100% Open Source. I hope they maintain it.
13:37 i may be wrong, but after more than 20 years coding mostly for accountants I think they might be responsible for Null. Several have told me they must have null to indicate unknown as zero could be a valid known amount while it might also indicate "amount unknown" so null makes sense to them... so an outstanding balance on a customer account for example could be zero coz they just paid, or zero because the invoice hasn't been raised yet. I've tried explaining there are alternatives etc. but they think everything is a spreadsheet and get stressed out if there more than 2 dimensions.
I feel like a single bit (boolean), for an "IsInvoiceRaised" value would be infinitely more helpful
Inclusion of data instead of the absence of data, every time
If that reason is true, so maybe a valid type UNKNOWN should have created back then......
Guys I got it! Nobel prize moment: THE MOAT IS IN THE FONT!😅
2:10 I've also been confused with its name suggesting it has a relation to the edge before
Why can’t we have an orm that lets you write your table definitions in SQL and parse the types from that? Every new orm comes with a new schema definition language 😅
Because ORMs are a fundamentally bad idea. Ted Neward called this out in his 2006 article on "The Vietnam of Computer Science". Neward's arguments on the impedance mismatch are quite good, and my experience bolsters his arguments much, much further. For example, ORMs fundamentally thrash the CPU cache making efficient performance nearly impossible. They also eat memory like candy, which isn't great in a scalable system. You ideally want the path from disk to the client to be as short as possible. Finally, they generate ridiculous amounts of unnecessary code that drastically slows the production of applications that use them.
I've been doing something much slimmer and more effective for over a decade now. I've rewritten the platform multiple times for numerous employers. Results are a 20-100x impact on productivity, measured in comparable lines of code metrics. I've finally gotten sick of doing it again and again and am founding my own organization to release it to the world as OSS. Keep an eye out for this alternative later this year.
Why not Postgres + JSONB columns?
But how does it work? Is it just a transpiler to SQL?
I think citing the Posgres basis is a GOOD thing. Even if its not exposed, you're saying you use one of the best foundations there is for a quality DBMS.
What about performance? I see edgeDB is writing in python why didn’t the use rust or go etc.?
DB on the cloud... meaning we will have to do http queries to it..... Theo, are you not the one saying this is a no go in one of your recent videos ? ......
I just spent so much time learning drizzle after learning prisma after learning graphql.... I still just want to write my own sql lol
Looks great. I'd be in if I didn't lose 1000 hours of my life buying into RethinkDB.
I would definitely try it for some project - but only if it had support for webstorm editor. Otherwise not much interest dealing with plaintext schemas
What is the difference between this EdgerDB and any ORM? Looks like it's an ORM
If its open source based on postgress you c extensions to it can use postgress extensions for stuff like vector math
Interesting insights on the limitations of SQL and the innovative approach of EDG DB. As you explore new tech solutions, consider enhancing your online presence with our SEO service. We specialize in boosting website visibility, attracting more traffic, and driving conversions. Let's chat about how we can optimize your digital strategy for success!
I’m building a personal project, I’ll give it a try since they offer a free tier!
The iceberg graphic alone had me to go the website just to see it for myself
Building on top of postgres is a good idea. Storage engines are hard to write. It can always be swapped out later.
finally a video that youre not just whining but artually sharing information
"Disclaimer: I am an advisor for EdgeDB, I do not ship it in production (I still build my services on PlanetScale)"
Maybe it's edge like cutting edge?
CuttingEdgeDB would be way better to avoid confusion.
This is interesting, would there be EdgeDBlite? Just like there is SQLite to SQL?
Isn't the point of SQLite just to be easier SQL, and doesn't EdgeDB already accomplish that? Or do you mean like having a really small install footprint? If so you are probably looking for Turso.
@@benbowers3613this isn't the same application for SQLite you are forgetting mobile apps and countless other platforms that leverage SQLite
@@benbowers3613that isn't the only utility of SQLite you are forgetting mobile apps and other platforms relying heavily on SQLite for its portability not because of easier sql
I'm a DBD(Database developer) for close to 2 years now. One of my understanding is this is practically an ORM?
One of the things that I dislike ORMs is the fact that when using ORMs is that they do no understand the data you store and how the Engine will interpret it and create the best execution plan there is. This can cause to performance issues. Some of them are easy to solve with indexes other require you to rewrite the query in a way to make the engine go to the execution plan it will be best for performance. No one like slow responses from a DB. Your app can be written Rust and be connected to a badly maintained DB. It does not matter how fast your App is if your DB provides you data slowly or in some cases a bad data.
My biggest question here is how do I do my perf tuning in the PostgreSQL or in the EdgeDB platform?
EdeDB has a fully functional Query Analyzer just like conventional SQL databases. Just put "Analyze" is front of your query like you would for most SQL databases.
It is not an orm. My best guess is that it is pretty much a syntactic sugar on top of SQL. What do you mean by "they do not understand the data you store"?
It's not an ORM. It is a Set-based DB implemented on top of an underlying Postgres database. The distinguishing factor is that EdgeDB Normalizes the data on the backend to better support their set fundamentals and optimize for their performance. It allows Postgres to handle efficiently allocating the page files for quick access while more directly controlling the query planning (something ORMs don't typically do outside of bare query splitting for related models).
The cool part is since it's 100% open source, you can dig in and see exactly how far they have taken this concept and the implementation details of it.
Same questions here, having tables with millions of rows and some queries will make it very slow, not a fan of those surpises as well, not sure how this is fixed.
@@trapfethen it is not at all a db implementation on top of psql, psql IS the db itself. It does not touch anything related to page files, psql does that. It does not more directly control query planning over normal sql, it uses the exact same interface. Whether typical orms are smart enough to not send multiple queries over the network is beside the point entirely
Hey guys, i've been following this project for some years and it's really promising, and now it's really taking shape. I'd say _do_ check it out if it seems like your cup of tea! I've used in a couple toy project and now using for a _big_ one.
I love to edgeDB at work
EdgeDB in t3 stack, when?