Wow! Thank you very much for this video Code to the Moon! Just the two of us at the moment, but my brother and I have huge plans for SurrealDB. We are working on lots of performance improvements, GraphQL integration, and our own key-value storage engine coming in due course. Thank you once again for this video!
_Came for Rust, stayed because the unique abstractions in the design actually managed to make the idea of database structure actually appealing to me..._ 😹
Thanks for posting this! I've surveyed the DB landscape wayyy too much and still, this is my first time seeing this database! VERY impressed and intrigued. Also, great job on the overview! Really concise and thorough introduction.
The content of your channel is real: great video + audio quality + thumbnails + transitions + rhythm and so much more. Really inspiring! Thanks for that super fast and amazing tutorial! 🙏🏻
This looks really interesting. Having worked for many years with traditional SQL, and having migrated a few years back completely to NoSQL databases for speed and scalability reasons, I'm really curious to see how SurrealDB scales. If it combines the scalability of NoSQL databases with the ease and familiarity of SQL then could be a very exciting!
I'm sure this will become very popular quickly. It seems to be very well thought out and a great fit for a ton of use cases, it's written in Rust which is always a plus and it has super consistent and slick looking branding and all that. The logo looks so cool, I'm tempted to use surreal just so I have a reason for putting that sticker on my laptop :-P
I agree Flying Squirrel! To your point - In addition to being feature packed, it has incredible branding which I feel like matters more than most technical people would like to admit
These awesome projects like SurrealDB are a result of developers using a language in which they don't have to care about memory and bang their heads when their application segfaults. Awesome work by SurrealDB devs!!
The devs fpr this project should feel proud. Its not easy to create a whole new database architecture, let alone one that leverages the benefits of it predecessors. This will catch on
Those multi-cluster stuffs that are in "future" on surrealdb's roadmap are even more ambitious. If they can deliver on those and survive Jepsen, I'll tip off every hat to them!
Thanks so much! I'm actually just beginning to get my feet wet in the backend development world, and I have to say it's a very exciting time to be doing so. I've got just enough experience with a couple other relational and graph databases to see the massive advantage of some of these features, but I've yet to really dive into Rust. I'm getting the impression that its time is coming soon, but according to the forum murmurings that's an impression that Rust has been pretty good at creating for quite some time without a lot of real progress. However, looking at the hard facts with this framework, it seems like they might finally have a real game changer on their hands. I hope it works as smoothly as it appears to.
Thanks for watching! Fwiw you don't need to have any Rust knowledge to use SurrealDB! I agree it does seem like a game changer - the current version does have some of the quirks that you'd expect in any beta software, but overall it seems to work really well in its current state!
@@codetothemoon I managed to get an instance served up locally and it worked great, but I'm wondering if it's possible to create a cloud-hosted instance through something like Google Cloud Functions or if that is something I will have to wait for until they release their cloud hosting features?
@@codetothemoon yes, especially for small projects it's like having a firebase in a box, at least for me having something like this, if the security layers and event emission works properly, completely removes the need for a server between the client and the DB. And the fact that you can embed js functions into fields makes it even possible to have some business logic too (with limitations I guess) Definitively not the way I would use for a well structured long-term project, but for everything else I don't know... on paper it kinda fits perfectly
Nice video and nice product! For the curious, this db uses TIKV under the hood which uses RockdDB, an LSM tree based database. Cassandra uses the same technology.
@@SurrealDB Do you think it’s more effective to build your own storage engine right now or have a pluggable architecture? I ask because there’s several features that I think would be great if users had a choice and they seem complex to implement in that way out of the box, so if users can choose the right storage engine for their use-case, it would probably be better. Some of the feature choices I’d like to see are LSM vs. b-trees, enabling/disabling record versioning for time-series data, enabling/disabling zero-copy for websockets (I really want something to dethrone Kafka that will integrate natively with a primary data store (Sorry, Redis. Keep dreaming)), and compression algorithm choices. Some of those aren’t even really in current storage engines, so I’d be really interested to see how they can be implemented. I’ll certainly be keeping an eye on this project. Wish you the best of luck!
@@CharlesChacon We always intend to support lots of pluggable storage engines to give developers the choice. Our own key-value store will enable us to offer additional functionality down the line that we can't currently do with the current architectures out there.
There’s a JVM based multi-storage mode database called OrientDb. They have the same concepts/similar concepts. Distributed fault tolerant. Document store, relational, and graph, schema is optional, integration with kafka, and more. Uses SQL variant for query. The documentation is a bit lacking though.
@@codetothemoon I haven't done any performance testing. The backend storage engine is very robust and a lot of integration with other systems like Kafka, MQTT, etc. It's a pretty "featureful" database.
I'd like to know more about row-based permissions and end-user authentication features. They are both mentioned on their website as features of the DB, but I can't find any documentation about these features.
There are definitely some gaps in the documentation. But the SurrealDB team is really responsive and helpful on their Discord server, maybe try getting help there!
@@codetothemoon that’s a good idea! I’ll definitely check it out. I really hope they transition to a documentation first approach, good documentation is fundamental for stuff like frameworks and databases. Thank you for the tip though!
Absolutely revolutionary. Got a question about the performance of the edge queries - while most were in the 100 microsecond range, the edge queries were in the 500 microsecond range. Does that give anyone pause? I have no idea how this stacks up with other graph database tech so it could be a nitpick.
Great question - I hadn't delved too deeply into performance benchmarks. In the docs they openly state that the performance currently doesn't yet quite stack up to the more established databases, so I'm wondering if we should wait for a more stable release before reading too deeply into them. Definitely something to keep tabs on though, and it's not unreasonable to be cautious about types of queries that seem to be slow in the beta.
I'd really like a nice way to represent rust style enums in my database. I've never seen a database do that before. I would use Surreal for everything if I could do that
I like the idea behind this. I'm just not sure if it'll fit any of my projects. I have things split into three distinct layers SvelteKit (Cloud Edge Deploy) -> FastAPI (Local Python Microservices) -> PostgreSQL (Local DB). Mostly I prefer the DB to be as plain as possible which is just storing the data and the Local Microservice layer handles Auth, Data validation, some caching, etc, etc. This means what I'd really be looking for if I was switching to a new GraphDB is a Rust or Python equivalent of FastAPI to connect to SurrealDB. It'll be interesting to watch this project grow and see what other utilities pop up from this.
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
It's probably not a good fit for every use case, especially when one db paradigm can handle all of your access patterns and you don't need the built-in auth and REST APIs that Surreal has. But yeah I agree it'll be cool to see how the project develops!
This is very interesting to me. The features are indeed ambitious but quite compelling. Would require a lot of testing before using it in a production product I suspect but I want to try it in my normal research and experiments code. As usual, well explained and informative.
I think SurrealDB is really interesting too, but yeah I wouldn't use it in production quite yet. I'm definitely looking forward to watching how the project develops!
Hi I! We’ve already been building production applications on it for a few years now. However we are looking to release the stable Version 1.0.0 really soon!
My understanding is that it aims to support all of the complex types of queries you'd expect to be able to perform on a traditional relational database. JOINS specifically are eliminated as you are expected to use record links, graph edges and FETCH instead, but you can achieve basically the same thing. I believe subqueries are currently supported (but maybe ask on the SurrealDB discord server to be sure). Not sure to what level CTE-like patterns are currently supported or what is planned there, again maybe check with Tiobe and Jaime.
I'm starting to see more Obsidian in rust dev videos... I'm inclined to think you guys have super meaty brains... 🧠 side note let me know if someone is making Obsidian in Rust... so I can meme about it.
@@leeroyjenkins0 Obsidian MD is not open source, so I don't know what it's written in and note-taking being called a fad is funny can't tell if you're trolling or legit.
Funny story about why I used Obsidian for this - there isn't any readily available SurrealDB SQL syntax highlighting, so I had to hack Obsidian to use the same custom JavaScript syntax highlighting that the SurrealDB website uses 👍
This is really cool and all, but when evaluating new things to work with, "version control" is the first thing that comes to mind... if this can be easily integrated into Git-like repos, it'd be a gamechanger for me. Also... localization... and transaction support across tables.
Agree, version control is huge, and I'm not sure if Surreal plans to support it. At one company we had implemented our own homegrown version control inside Postgres, and the number of bugs and general pain it caused over many years was hard to fully quantify. It'd be nice to have something like that built in to the database.
I love it, except for one thing: is there a simplified schema definition syntax? I support defining strong schemas, but the code for doing it looked really verbose, too wordy vs normal SQL
yeah afaik this is currently the only way to do it, at least the only way that exists in the documentation. The team seems really receptive to feedback, maybe hop on their discord and shoot them a message!
Looks like a great product. Thanks. How to we do a full text search on a field? Is it efficient (unlike oracle search where % wildcard or any text partial search will kill the efficiency while search?)
The docs mention support for full-text indexing, but I'm not sure exactly the steps required to enable that for a field (maybe just create a specific type of index on it?). The docs do appear to have some holes in them at the moment, but the team is extremely responsive and helpful on their discord server, maybe drop in and ask them!
The current scalability story seems to be pretty good. They've stated in their docs that performance isn't up to par with the more established databases yet (I believe they specifically use Postgres as an example). I'm not sure how much security auditing they've done, but I'd imagine they'd wait to do that until a more stable release (as opposed to beta).
Summary: Surreal DB is a new database that aims to take the best aspects of each major database Paradigm and combine them all into one database. It is schema-less by default, has a sql-like query language, and supports record links to establish relationships between records in different tables. It also has a graphql-like query language for fetching data from multiple tables at once, and supports websockets for real-time updates.
Good idea, I've added it to the video ideas list! There are definitely some gaps currently in the documentation. But the SurrealDB team is really responsive on their Discord server, you might try getting help there!
I have a soft spot for databases and SurrealDB sounds very interesting. I must say that I have often problems with joins, so not needing it sound great. I am not proficient with rustlang yet, so I wonder if I cound use some other language with SurrealDB?
Jaime I think he is referring to automatic schema migrations like what FlywayDB does in the traditional SQL world. Sean - correct me if I'm wrong about this 😎
You mention Events being used for real time updates to users via websockets, is that use case one with an example in the SurrealDB docs? Is it something we can build out in Rust? Is it better to directly interface with SurrealDB's API?
Hi Sean. As mentioned on my other comment, we are working on the Rust client to connect Rust with a SurrealDB server. However you can already use the database library itself embedded in Rust. Not sure if that answered your question or not, but let me know!
thank you! actually currently working on a new video that will feature SurrealDB (but won't focus specifically on it). I'd love to do another full video including all of the new stuff since this one, we'll see if it bubbles to the top of the video idea list 😎
@@nobytes2 "potentially" because it's not automatically faster, it depends on what you write, but for the same code in c and in rust you will either have rust and C roughly equal (same llvm backend) or rust faster because the compiler has more information to prove more things about the program and use more agressive optimisation. If you want data you can look for it yourself you have Google too.
It’s pretty cool for a multi-model approach. Right now I’ve switched to graphs exclusively because moving away from SQL and tables has been easier for me. I haven’t seen the need to keep them.
@@jaimemorganhitchcock6195 I understand the desire to combine different DB models. A better metaphor for how I see this is sticking with a hybrid vehicle instead of going full electric because it’s assumed that having both gas and electric together is more convenient. Convenient sure but is it a better design? I don’t believe so. Currently the database I mainly use is Dgraph. I’ll see how Surreal compares but I haven’t found anything that’s close that simplifies my architecture as much as Dgraph.
Dgraph is definitely a cool database @Khari99. With regards to multi-model, we wanted to ensure that SurrealDB was easy-to-use coming from all different database backgrounds. In addition, the design is such that the performance isn’t affected regardless of what data model you use in SurrealDB - obviously there are no JOINS, so SurrealDB is definitely closer to MongoDB/Dgraph as a concept. We have some really great things planned on our roadmap, so hopefully we’ll sway your mind in the not-too-distant future 😊!
@@jaimemorganhitchcock6195 While I like Dgraph a lot, I don't think it's perfect. Outside of the GraphQL API, I feel like there are a lot of ideas that you can pick up on to make SurrealDB better. 1. Automatically generated CRUD queries from a schema definition. I've only seen this implemented with GraphQL so I don't know if this is possible for your architecture. But this significantly reduces how much code needs to be written and maintained in the server. 2. Implement lambda functionality. Dgraph has a philosophy of moving the DB to the middle layer and allowing queries to trigger lambdas that either run custom computations or can reach out to a server externally for some request if needed. I really like this approach as it helps further reduce architectural complexity. But right now I'm limited to JavaScript lambdas. Rust lambdas would be amazing. 3. Auth & Access Control patterns. I looked through the documentation for Surreal and haven't seen anything around this topic. I understand the philosophy of leaving this up to server logic but if these features are baked into the database, it's one less thing for me to worry about. Even though I moved away from Fauna, they have this integrated into their DB, which is why I tried them out in the first place. 4. Logical Inference Rules. I've only seen these implemented in TypeDB. one of the things that Dgraph is missing that almost makes me want to develop a DB in Rust myself is the ability to create a knowledge graph intuitively. If you could look into how TypeDB accomplishes this and think of a similar implementation, this would be awesome. 5. Indexes. I also didn't see this in the documentation. Dgraph has indexes that enable full-text search (along with many other index types) out of the box. I would rather not have to implement elasitcsearch separately.
Hi Rea. There are close similarities with traditional SQL, but the way that record IDS and foreign keys are stored are different. We plan to work on a realtime sync-connector/importer so that you can import data easily into SurrealDB from Postgres/MySQL/MongoDB/RethinkDB in due course.
Sounds like it's not available now but I imagine there are droves of folks with the same need, I'd be surprised if this isn't one of the higher priorities of the team.
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
Personally I think I'm waiting tor the interface to be implemented. Currently using PocketBase and it's great to use - but I'm always keeping an eye out for alternative to broaden my horizons. SurrealDB looks great - but the lack of a UI right now is keeping me away still. Hopefully soon!
I'm curious about the what kind of backup solution they have. Over the years many databases and come and gone, all promising to be easy turn key solution to all problems. But when you start to look at them more in detail, as you do when you use them for real project, you always find that the old databases have these "complexities" for reasons not apparent initially.
I think this healthy skepticism is a great way to approach new databases like this, and one that I had going in as well. During the time I immersed myself in it, I tried to uncover any gaps between its promises and what it delivers, and I couldn't find any. That's not to say there aren't any, just not any that I could find. The biggest issue I ran into was missing / incomplete documentation, which I'm confident will be addressed. YMMV
Can you please also tell us for what purpose it can be used. For example, RedShift can be used for data warehouses, MySQL can be used as you traditional DB, as well as your reporting DB, Dynamo DB can provide you fast partition based access, so it can be used where the application would like to get / set single partition data. In which category would you like to put Surreal DB in?
The category that wants more PR bc they have no real users or sane use cases. The one where something as important as a database is supposedly written and maintained by 2 brothers. The category you'd be insane to use for anything be it production or hobby or even just looking more into it than this. Choose a well established, tested, stable db like a sane person. Not one showcasing random coder's praise of them and every 3 star project using them on their site. Databases are not the place to try experimental software, look for tested and reviewed software instead. Take CM with a grain of salt, small channel and recommending a lot of new projects with no user base or sane use case bc they were paid to do so. In fairness to CM, they have kept these sponsored videos separate from their more general fast tutorials and made it clear they are sponsored from the getgo. In fairness to me and my criticism. CM did outright recommend it to their entire audience without explicitly pointing out its obvious week points like lack of a community or any large company production use and likely lack of testing due to such. They only covered competitors like PostgreSQL's weaknesses. Very obviously bias and thus inherently dishonest (as in not telling the truth they know) review which is very disrespectful to their audience
i looked for this comment, after all thus hype i decided to replace sqlite db in my backend...and omg... this surreal db has so many bugs and issues i recon it will not be ready to any serious prod deployment in next 1-2 years, seriously...
@@jaimemorganhitchcock6195 I think this refers more so to schema migrations/versioning. Aside from the under-the-hood capabilities, I think this would be one of the top feature requests
Thanks - unfortunately I don't have the queries posted anywhere, but you can check out README.md on the SurrealDB GitHub repository for some good examples!
Hey Osej! Currently you can run SurrealDB on top of TiKV (tikv.org) which would enable you to scale your data to 100+TB if you need. We're going to be improving our documentation on our site soon so that it makes this easier to get started with, but for the moment head to the bottom of this page: surrealdb.com/docs/start/starting-surrealdb
This DB looks really interesting. I would love to try it for a future Rust project. Does it cost anything to use? Is it available at any existing cloud hosting provider?
Nice! It's completely free to download and host however you like. But SurrealDB is aiming to also provide a managed cloud version of it that comes at a cost, which I imagine will be a pretty compelling value prospect.
Is it just the special query language and the websocket real time updates that makes it different from normal SQL databases? Everything it can do in this video can also be done in normal SQL databases. Also, is it ACID?
I think the main differences from traditional SQL databases are * Under the hood it is a key-value store, which allows for basically infinite horizontal scalability because it lends itself well to sharding. Traditional SQL databases need something like Vitess to do this * It supports graph operations * Strict schemas are supported but not required * Supports REST endpoints and end user auth out of the box I'm probably missing something else but those are the first ones that come to mind. More broadly though, choosing a database paradigm is currently one of the most impactful decisions in any software project, and making the wrong choice can lead to immense amounts of pain. The main value of SurrealDB is that it is aiming to be aiming to be a jack of all trades, such that it can support any unforseen use cases that might arise after a database has already been chosen. For example - if initially a project just requires a KV store but a few years later a new feature requires a relational database. SurrealDB gotchu. And I believe it is ACID, but check the docs and/or ask on the Discord server to keep me honest.
Can this access the file system from functions? Thinking in terms of writing files to other storage. Would love a tutorial on the graph queries if you have them down. They can twist your mind when not familiar with the syntax.
Wow! Thank you very much for this video Code to the Moon! Just the two of us at the moment, but my brother and I have huge plans for SurrealDB. We are working on lots of performance improvements, GraphQL integration, and our own key-value storage engine coming in due course. Thank you once again for this video!
Hey just wanted to say thank you for this amazing DB keep up the good work.
@@alandto3996 Thank you very much for the kind words. We certainly will!
Thank you very much! ❤️
Looks like a brilliant project. Would love to see key-values and websocket auth pubsub soon! 💛
@@fire17102 WebSocket auth already here Tami. Live queries and pub-sub coming very soon!
_Came for Rust, stayed because the unique abstractions in the design actually managed to make the idea of database structure actually appealing to me..._ 😹
Is it unique? Looks like OrientDB, which has been around for a decade.
Thanks Wolveric, glad it piqued your interest!
OrientDB is written in Java - I'm sure it's fast, but there's kind of an upper bound to how fast it can be...
Me too
This looks extremely interesting. I suspect this will grow very quickly.
Thank you very much @theherk! We have a lot planned on the roadmap!
Agree!
A directional edge connection is the first time I've seen an arrow operator feel appropriate
❤
In F# you use this operator
It's similar to the syntax of Cypher, the query language of Neo4j
Arrows for marking function output types and for dereferencing a pointer to a compound object both seem appropriate to me. Just my 0.002¢
I agree! Really the perfect operator for working with graphs
DEFINE, CREATE, RELATE, SELECT ... all six letter commands. I *love* it.
Wow, I hadn't noticed that!
keen eye
Ok, this is insane. I was dreaming of a DB that would do graph connections in trust to move away from neo4j , but this is amazing. An absolute killer.
I agree! It definitely has massive potential.
Thanks for posting this! I've surveyed the DB landscape wayyy too much and still, this is my first time seeing this database! VERY impressed and intrigued. Also, great job on the overview! Really concise and thorough introduction.
Thanks for the kind words Mike, glad you found it valuable!!
The content of your channel is real: great video + audio quality + thumbnails + transitions + rhythm and so much more.
Really inspiring!
Thanks for that super fast and amazing tutorial! 🙏🏻
Holy crap. If this pans out to what it’s teaching for this could single handedly solve a lot of the database headaches.
I agree! Choosing a database paradigm is really challenging, the prospect of not having to make that choice upfront will be so liberating...
This looks really interesting. Having worked for many years with traditional SQL, and having migrated a few years back completely to NoSQL databases for speed and scalability reasons, I'm really curious to see how SurrealDB scales. If it combines the scalability of NoSQL databases with the ease and familiarity of SQL then could be a very exciting!
Agree, it seems like the team is aiming to do just that!
Time to start playing with the new boy in the block. Thanks for sharing.
nice, thanks for watching!
I'm sure this will become very popular quickly. It seems to be very well thought out and a great fit for a ton of use cases, it's written in Rust which is always a plus and it has super consistent and slick looking branding and all that. The logo looks so cool, I'm tempted to use surreal just so I have a reason for putting that sticker on my laptop :-P
I agree Flying Squirrel! To your point - In addition to being feature packed, it has incredible branding which I feel like matters more than most technical people would like to admit
at 9:15 you could of simplified the statement to be *SELECT armor:dragon
Thanks Dabbo will give this a try!
These awesome projects like SurrealDB are a result of developers using a language in which they don't have to care about memory and bang their heads when their application segfaults. Awesome work by SurrealDB devs!!
What a cool demo! It really showcases SurrealDB's simplicity and usability. Thanks!
Thanks and thanks for watching!
I am brand new to DAW and soft soft - these tutorials are excellent an very helpful to get soone like up and running. Appreciate
Glad you found it valuable, thanks for watching!
The devs fpr this project should feel proud. Its not easy to create a whole new database architecture, let alone one that leverages the benefits of it predecessors. This will catch on
I agree that what they've done is really impressive! Seems like they have a ton of momentum already, hopefully it continues
You guys are great and SurrealDB genuinely feels interesting. Can't wait to play with it.
Thank you very much for the kind words Onizuka ❤️
It's worth giving a try for sure!
Those multi-cluster stuffs that are in "future" on surrealdb's roadmap are even more ambitious. If they can deliver on those and survive Jepsen, I'll tip off every hat to them!
100% agree, can't wait to see how things pan out. I actually hadn't heard of Jepsen, I'm glad you put this on my radar!
Thanks so much! I'm actually just beginning to get my feet wet in the backend development world, and I have to say it's a very exciting time to be doing so. I've got just enough experience with a couple other relational and graph databases to see the massive advantage of some of these features, but I've yet to really dive into Rust. I'm getting the impression that its time is coming soon, but according to the forum murmurings that's an impression that Rust has been pretty good at creating for quite some time without a lot of real progress. However, looking at the hard facts with this framework, it seems like they might finally have a real game changer on their hands. I hope it works as smoothly as it appears to.
Thanks for watching! Fwiw you don't need to have any Rust knowledge to use SurrealDB! I agree it does seem like a game changer - the current version does have some of the quirks that you'd expect in any beta software, but overall it seems to work really well in its current state!
@@codetothemoon I managed to get an instance served up locally and it worked great, but I'm wondering if it's possible to create a cloud-hosted instance through something like Google Cloud Functions or if that is something I will have to wait for until they release their cloud hosting features?
This is the most interesting thing I have seen in a while in the tech world
Same, it seems pretty disruptive if the vision fully materializes!
@@codetothemoon yes, especially for small projects it's like having a firebase in a box, at least for me having something like this, if the security layers and event emission works properly, completely removes the need for a server between the client and the DB. And the fact that you can embed js functions into fields makes it even possible to have some business logic too (with limitations I guess)
Definitively not the way I would use for a well structured long-term project, but for everything else I don't know... on paper it kinda fits perfectly
I couldn’t believe you got that event to work just like that 👏
Thanks, I practiced a bit before recording the video 🙃
Nice video and nice product!
For the curious, this db uses TIKV under the hood which uses RockdDB, an LSM tree based database. Cassandra uses the same technology.
Good to know, thanks
Thank you Ismaj. We are working on our own key-value store which will bring some additional benefits to SurrealDB in due course.
@@SurrealDB Do you think it’s more effective to build your own storage engine right now or have a pluggable architecture?
I ask because there’s several features that I think would be great if users had a choice and they seem complex to implement in that way out of the box, so if users can choose the right storage engine for their use-case, it would probably be better.
Some of the feature choices I’d like to see are LSM vs. b-trees, enabling/disabling record versioning for time-series data, enabling/disabling zero-copy for websockets (I really want something to dethrone Kafka that will integrate natively with a primary data store (Sorry, Redis. Keep dreaming)), and compression algorithm choices. Some of those aren’t even really in current storage engines, so I’d be really interested to see how they can be implemented.
I’ll certainly be keeping an eye on this project. Wish you the best of luck!
@@CharlesChacon We always intend to support lots of pluggable storage engines to give developers the choice. Our own key-value store will enable us to offer additional functionality down the line that we can't currently do with the current architectures out there.
Thanks, and thanks for the info - I wasn't aware of TiKV's relationship with RockDB!
Really strong stuff you serving today!
Authors really taked their code to the moon 😄
lol thanks! 🚀
There’s a JVM based multi-storage mode database called OrientDb. They have the same concepts/similar concepts. Distributed fault tolerant. Document store, relational, and graph, schema is optional, integration with kafka, and more. Uses SQL variant for query. The documentation is a bit lacking though.
Nice, this actually wasn't on my radar. How does it stack up performance-wise?
@@codetothemoon I haven't done any performance testing. The backend storage engine is very robust and a lot of integration with other systems like Kafka, MQTT, etc. It's a pretty "featureful" database.
This is an amazing video on introduction to it's concept, beautiful.
thank you, glad you liked it!
This looks very promising!
Thank you very much @Elf!
I agree!
Looks like I found a new go-to databse for prototype projects (and non-prototype ones once it matures a bit)
same here!
tbh I can't wait what sorts of projects the community will come up with now that we have rust
agree, there are so many interesting Rust-powered projects popping up, even CLI tools to replace ones that have been around for decades...
It's pretty SURREAL that a database engine like this exists... Too bad I work as a front end dev and never have to deal with databases
Perhaps this tech could have some side-project potential for you?
It's ok, given that there seems to be a new JS framework every day, there's bound to be a SurrealJS at some point... 🙃
You can start creating your side projects with Nodejs just for fun :D and connecting with SurrealDB
I'd like to know more about row-based permissions and end-user authentication features. They are both mentioned on their website as features of the DB, but I can't find any documentation about these features.
There are definitely some gaps in the documentation. But the SurrealDB team is really responsive and helpful on their Discord server, maybe try getting help there!
@@codetothemoon that’s a good idea! I’ll definitely check it out. I really hope they transition to a documentation first approach, good documentation is fundamental for stuff like frameworks and databases. Thank you for the tip though!
If they manage to do the websocket part, it will immediately become much more useful.
Agree, it will be a game changer!
Fantastic video, loved it. I will be playing with it this afternoon.
Thanks Thomas, let us know how it goes!
First vid I watched *exploring* SurrealDB.
Looks like something I gotta know, being a big PostgreSQL fan.
Thanks :)
Thanks for watching Tobias!
looking forward to learning from the rest of your videos. Subscribed!
Thanks Justine, very happy to have you onboard!
I love it. Gotta test this
It's worth giving a try!
The event thing was crazy, amazing
agree!
Absolutely revolutionary. Got a question about the performance of the edge queries - while most were in the 100 microsecond range, the edge queries were in the 500 microsecond range. Does that give anyone pause? I have no idea how this stacks up with other graph database tech so it could be a nitpick.
Great question - I hadn't delved too deeply into performance benchmarks. In the docs they openly state that the performance currently doesn't yet quite stack up to the more established databases, so I'm wondering if we should wait for a more stable release before reading too deeply into them. Definitely something to keep tabs on though, and it's not unreasonable to be cautious about types of queries that seem to be slow in the beta.
I'd really like a nice way to represent rust style enums in my database. I've never seen a database do that before. I would use Surreal for everything if I could do that
That would be cool!
I like the idea behind this. I'm just not sure if it'll fit any of my projects. I have things split into three distinct layers SvelteKit (Cloud Edge Deploy) -> FastAPI (Local Python Microservices) -> PostgreSQL (Local DB). Mostly I prefer the DB to be as plain as possible which is just storing the data and the Local Microservice layer handles Auth, Data validation, some caching, etc, etc. This means what I'd really be looking for if I was switching to a new GraphDB is a Rust or Python equivalent of FastAPI to connect to SurrealDB. It'll be interesting to watch this project grow and see what other utilities pop up from this.
Hey Matt! I understand your tech stack position. We have some awesome things planned for SurrealDB so hopefully we can sway your mind in due course 😀
@@jaimemorganhitchcock6195 If you get full-text search in there, I’ll bite!! 😄👍🏽
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
It's probably not a good fit for every use case, especially when one db paradigm can handle all of your access patterns and you don't need the built-in auth and REST APIs that Surreal has. But yeah I agree it'll be cool to see how the project develops!
Wow that looks bonkers good!
Thank you very much Ratstail91 ❤️🙌
I think "bonkers good" is a fitting description!
This is very interesting to me. The features are indeed ambitious but quite compelling. Would require a lot of testing before using it in a production product I suspect but I want to try it in my normal research and experiments code. As usual, well explained and informative.
I think SurrealDB is really interesting too, but yeah I wouldn't use it in production quite yet. I'm definitely looking forward to watching how the project develops!
This is such an informative video. Thanks for creating. I am going to try myself.
Thanks Singh! Fantastic that you are going to start creating! Please let me know if there's anything I can do to help.
@@codetothemoon Thanks for replying. Will definitely do 😊.
This feels surreal. The web UI seems not ready yet but the project is interesting.
Surreal indeed! Yeah my understanding is that the UI will be available relatively soon.
Reminds me a bit of datomic but in rust...
How long until it’s production ready?
Hi I! We’ve already been building production applications on it for a few years now. However we are looking to release the stable Version 1.0.0 really soon!
hadn't heard of datomic, thanks for putting it on my radar!
Looks amazing!
As a data analyst/engineer the one thing I really want to know is: Does this support complex joins, subqueries & cte?
My understanding is that it aims to support all of the complex types of queries you'd expect to be able to perform on a traditional relational database. JOINS specifically are eliminated as you are expected to use record links, graph edges and FETCH instead, but you can achieve basically the same thing. I believe subqueries are currently supported (but maybe ask on the SurrealDB discord server to be sure). Not sure to what level CTE-like patterns are currently supported or what is planned there, again maybe check with Tiobe and Jaime.
It’s totally worth learning new skills cause then you get excited and confident to create
It's true! 🔥
After web framework galore we now have database galore
For reals!! This is getting ridiculous!! 😂
Hah yeah, I'm skeptical of all new projects - JS, Database, or otherwise - but this one really seemed to stand out from the crowd.
As soon as LIVE queries are implemented, I am switching for sure :)
they do sound pretty handy!
I'll definitely be trying this out on my personal app later!
Nice, let us know how it goes!
Super cool. I just need to figure out how to use this at work.
Hi Nathan. Thank you very much. Join the Discord or GitHub if you have any questions - surrealdb.com/community
Convincing others to buy in is the hardest part! Even for game changing technologies.
That's super cool; thanks for bringing this!
Thank you very much David
Thank you David!
Thanks for watching!
Could we say that this is SURREAL
❤❤
Indeed!
This is cool and interesting. I might try it out, but I’ll be keeping work with single databases for their specified purpose.
Nice, yeah definitely something to explore but probably not ready for production yet. Also, I see you are a fellow Babylon 5 fan!
I'm starting to see more Obsidian in rust dev videos... I'm inclined to think you guys have super meaty brains... 🧠
side note let me know if someone is making Obsidian in Rust... so I can meme about it.
@@leeroyjenkins0 Obsidian MD is not open source, so I don't know what it's written in and note-taking being called a fad is funny can't tell if you're trolling or legit.
Funny story about why I used Obsidian for this - there isn't any readily available SurrealDB SQL syntax highlighting, so I had to hack Obsidian to use the same custom JavaScript syntax highlighting that the SurrealDB website uses 👍
This is super cool, would love to check it out
Thank you Zaha! Join the Discord or GitHub if you have any questions - surrealdb.com/community
Yeah I think it's worth a look!
This is absolutely the level up!
I agree, the SurrealDB team is moving mountains!
This is really cool and all, but when evaluating new things to work with, "version control" is the first thing that comes to mind... if this can be easily integrated into Git-like repos, it'd be a gamechanger for me. Also... localization... and transaction support across tables.
Agree, version control is huge, and I'm not sure if Surreal plans to support it. At one company we had implemented our own homegrown version control inside Postgres, and the number of bugs and general pain it caused over many years was hard to fully quantify. It'd be nice to have something like that built in to the database.
I love it, except for one thing: is there a simplified schema definition syntax? I support defining strong schemas, but the code for doing it looked really verbose, too wordy vs normal SQL
yeah afaik this is currently the only way to do it, at least the only way that exists in the documentation. The team seems really receptive to feedback, maybe hop on their discord and shoot them a message!
Looks like a great product. Thanks. How to we do a full text search on a field? Is it efficient (unlike oracle search where % wildcard or any text partial search will kill the efficiency while search?)
The docs mention support for full-text indexing, but I'm not sure exactly the steps required to enable that for a field (maybe just create a specific type of index on it?). The docs do appear to have some holes in them at the moment, but the team is extremely responsive and helpful on their discord server, maybe drop in and ask them!
@@codetothemoon Thank you!!
I really like what I'm seeing, I just hope it won't end up like rethink db
Thank you very much indeed @gorudonu ! We have lots planned for SurrealDB!
Agree it seems promising!
This looks pretty interesting. I’ll have to play with it some
it's worth checking out!
SurrealDB seems very interesting. If I ever am granted the luxury of time I'll gladly experiment a bit.
Nice, it's worth checking out!
How scalable is this? What read/write performance does it have comparing to postgres? Did they pay for a security audit?
The current scalability story seems to be pretty good. They've stated in their docs that performance isn't up to par with the more established databases yet (I believe they specifically use Postgres as an example). I'm not sure how much security auditing they've done, but I'd imagine they'd wait to do that until a more stable release (as opposed to beta).
Summary:
Surreal DB is a new database that aims to take the best aspects of each major database Paradigm and combine them all into one database. It is schema-less by default, has a sql-like query language, and supports record links to establish relationships between records in different tables. It also has a graphql-like query language for fetching data from multiple tables at once, and supports websockets for real-time updates.
Sounds about right to me!
I'd love to see a video showing how to use it with Rust. The documentation doesn't seem to exist yet.
Good idea, I've added it to the video ideas list! There are definitely some gaps currently in the documentation. But the SurrealDB team is really responsive on their Discord server, you might try getting help there!
I have a soft spot for databases and SurrealDB sounds very interesting. I must say that I have often problems with joins, so not needing it sound great. I am not proficient with rustlang yet, so I wonder if I cound use some other language with SurrealDB?
Kimmo - absolutely, I believe they have client libraries in several other mainstream languages. No Rust knowledge necessary.
When doing an UPDATE what happens? Are migrations with an up/down still advised? Any gotchyas or considerations about what are 'safe' migrations?
Any defining a schemafull table locks us in!? Oh yikes.
@@seannewell397 Hi Sean. Please could you clarify what you mean by 'migrations with an up/down'.
Jaime I think he is referring to automatic schema migrations like what FlywayDB does in the traditional SQL world. Sean - correct me if I'm wrong about this 😎
Thank you for this video,
You rock!
Thanks for watching Scott!
concept is cool. Is that micro second in units or millisecond?
μs means microseconds 😎
You mention Events being used for real time updates to users via websockets, is that use case one with an example in the SurrealDB docs? Is it something we can build out in Rust? Is it better to directly interface with SurrealDB's API?
Hi Sean. As mentioned on my other comment, we are working on the Rust client to connect Rust with a SurrealDB server. However you can already use the database library itself embedded in Rust. Not sure if that answered your question or not, but let me know!
Might be making a Rust integration tut for SurrealDB at some point...
looks amazing!
Glad it piqued your interest as well!
Can we get update on it since whole year passed. Maybe to cover updates and upgrades? Very nice video.
thank you! actually currently working on a new video that will feature SurrealDB (but won't focus specifically on it). I'd love to do another full video including all of the new stuff since this one, we'll see if it bubbles to the top of the video idea list 😎
Looks incredible but would love to see a perf / scaling follow up to really get a better feel of how SurrealDB stands up in real life scenarios
I'd love to do such a follow up, but I might wait until it goes 1.0 to do so!
Most production dbs are written in c, and tested throughout decades. It will be extremely difficult to beat performance but you never know.
@@nobytes2 well-written rust is potentially faster than C due to the rust compiler having more information to make agressive optimisations
@@eliottveyrier6253 show data or else stop making guesses, "potentially" lmao 😂
@@nobytes2 "potentially" because it's not automatically faster, it depends on what you write, but for the same code in c and in rust you will either have rust and C roughly equal (same llvm backend) or rust faster because the compiler has more information to prove more things about the program and use more agressive optimisation. If you want data you can look for it yourself you have Google too.
It’s pretty cool for a multi-model approach. Right now I’ve switched to graphs exclusively because moving away from SQL and tables has been easier for me. I haven’t seen the need to keep them.
Graph is definitely the future @Khari99! That is exactly why we wanted to merge Graph with traditional database concepts.
@@jaimemorganhitchcock6195 I understand the desire to combine different DB models. A better metaphor for how I see this is sticking with a hybrid vehicle instead of going full electric because it’s assumed that having both gas and electric together is more convenient. Convenient sure but is it a better design? I don’t believe so. Currently the database I mainly use is Dgraph. I’ll see how Surreal compares but I haven’t found anything that’s close that simplifies my architecture as much as Dgraph.
Dgraph is definitely a cool database @Khari99. With regards to multi-model, we wanted to ensure that SurrealDB was easy-to-use coming from all different database backgrounds. In addition, the design is such that the performance isn’t affected regardless of what data model you use in SurrealDB - obviously there are no JOINS, so SurrealDB is definitely closer to MongoDB/Dgraph as a concept. We have some really great things planned on our roadmap, so hopefully we’ll sway your mind in the not-too-distant future 😊!
@@jaimemorganhitchcock6195 While I like Dgraph a lot, I don't think it's perfect. Outside of the GraphQL API, I feel like there are a lot of ideas that you can pick up on to make SurrealDB better.
1. Automatically generated CRUD queries from a schema definition. I've only seen this implemented with GraphQL so I don't know if this is possible for your architecture. But this significantly reduces how much code needs to be written and maintained in the server.
2. Implement lambda functionality. Dgraph has a philosophy of moving the DB to the middle layer and allowing queries to trigger lambdas that either run custom computations or can reach out to a server externally for some request if needed. I really like this approach as it helps further reduce architectural complexity. But right now I'm limited to JavaScript lambdas. Rust lambdas would be amazing.
3. Auth & Access Control patterns. I looked through the documentation for Surreal and haven't seen anything around this topic. I understand the philosophy of leaving this up to server logic but if these features are baked into the database, it's one less thing for me to worry about. Even though I moved away from Fauna, they have this integrated into their DB, which is why I tried them out in the first place.
4. Logical Inference Rules. I've only seen these implemented in TypeDB. one of the things that Dgraph is missing that almost makes me want to develop a DB in Rust myself is the ability to create a knowledge graph intuitively. If you could look into how TypeDB accomplishes this and think of a similar implementation, this would be awesome.
5. Indexes. I also didn't see this in the documentation. Dgraph has indexes that enable full-text search (along with many other index types) out of the box. I would rather not have to implement elasitcsearch separately.
This is really cool, I haven't heard of many folks switching from pure relational to pure graph!
can i migrate stuff out of surreal db to a typical sql database or not ?
Hi Rea. There are close similarities with traditional SQL, but the way that record IDS and foreign keys are stored are different. We plan to work on a realtime sync-connector/importer so that you can import data easily into SurrealDB from Postgres/MySQL/MongoDB/RethinkDB in due course.
@@jaimemorganhitchcock6195 [EDIT] Thank you for clarifying! 👍🤓
This isn't possible at the moment Rea!
Sounds like it's not available now but I imagine there are droves of folks with the same need, I'd be surprised if this isn't one of the higher priorities of the team.
*Woah!!* Any plans to support full-text search?
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
I believe it is already supported!
@5:47 It works for me with SELECT math::sum(strength) FROM player GROUP ALL; (I'm using version 1.2) - so use GROUP ALL, rather than GROUP BY ALL
Personally I think I'm waiting tor the interface to be implemented. Currently using PocketBase and it's great to use - but I'm always keeping an eye out for alternative to broaden my horizons. SurrealDB looks great - but the lack of a UI right now is keeping me away still. Hopefully soon!
I haven't yet tried any of the "*bases", keep hearing good things about them. I agree the UI will be a game changer - the screenshots look really nice
Mylosp just a legend really aren’t ya
I try to be! 🦄
Good to know, looks very interesting, I will still use the rock solid postgres but this kind of project can have they use cases
hard to go wrong with Postgres!
I'm curious about the what kind of backup solution they have. Over the years many databases and come and gone, all promising to be easy turn key solution to all problems. But when you start to look at them more in detail, as you do when you use them for real project, you always find that the old databases have these "complexities" for reasons not apparent initially.
I think this healthy skepticism is a great way to approach new databases like this, and one that I had going in as well. During the time I immersed myself in it, I tried to uncover any gaps between its promises and what it delivers, and I couldn't find any. That's not to say there aren't any, just not any that I could find. The biggest issue I ran into was missing / incomplete documentation, which I'm confident will be addressed. YMMV
it's still full of bugs
What kind of font are you using? It looks so nice!
Very helpful, and surprisingly therapeutic
Nice, glad you liked it! "Therapeutic" is something I'm very happy to have achieved!
Can you please also tell us for what purpose it can be used. For example, RedShift can be used for data warehouses, MySQL can be used as you traditional DB, as well as your reporting DB, Dynamo DB can provide you fast partition based access, so it can be used where the application would like to get / set single partition data. In which category would you like to put Surreal DB in?
The category that wants more PR bc they have no real users or sane use cases. The one where something as important as a database is supposedly written and maintained by 2 brothers. The category you'd be insane to use for anything be it production or hobby or even just looking more into it than this. Choose a well established, tested, stable db like a sane person. Not one showcasing random coder's praise of them and every 3 star project using them on their site. Databases are not the place to try experimental software, look for tested and reviewed software instead.
Take CM with a grain of salt, small channel and recommending a lot of new projects with no user base or sane use case bc they were paid to do so. In fairness to CM, they have kept these sponsored videos separate from their more general fast tutorials and made it clear they are sponsored from the getgo.
In fairness to me and my criticism. CM did outright recommend it to their entire audience without explicitly pointing out its obvious week points like lack of a community or any large company production use and likely lack of testing due to such. They only covered competitors like PostgreSQL's weaknesses. Very obviously bias and thus inherently dishonest (as in not telling the truth they know) review which is very disrespectful to their audience
i looked for this comment, after all thus hype i decided to replace sqlite db in my backend...and omg... this surreal db has so many bugs and issues i recon it will not be ready to any serious prod deployment in next 1-2 years, seriously...
Keep up the good work!
thank you!
Interesting new DB, love Rust too.
I agree!
ti (the instrunt into the channel rack) and then it crashes the soft soft... Can soone help please?
sorry not an expert in "soft soft"!
Does it have some kind of migration tool to help migrations versioning?
Not just yet, but we are working on migration and sync connectors from Postgres, MySQL, MongoDB and RethinkDB.
@@jaimemorganhitchcock6195 I think this refers more so to schema migrations/versioning. Aside from the under-the-hood capabilities, I think this would be one of the top feature requests
Good question, for many people using Surreal will surely be dependent on this!
thanks for your stuff, do you have any recommendable rust course?
I haven't heard of any courses for SurrealDB yet - it's still very new, but I'm sure there will be one before too long.
Great video, thanks for sharing. SQL novice here, would you mind sharing the SQL/SurrealDB queries? Would love to look them over to better understand.
Thanks - unfortunately I don't have the queries posted anywhere, but you can check out README.md on the SurrealDB GitHub repository for some good examples!
I'm ready to marry this db
💍
😂
hah me too!
how would I host this and not have to worry about scalability? thanks for the video!
Hey Osej! Currently you can run SurrealDB on top of TiKV (tikv.org) which would enable you to scale your data to 100+TB if you need. We're going to be improving our documentation on our site soon so that it makes this easier to get started with, but for the moment head to the bottom of this page: surrealdb.com/docs/start/starting-surrealdb
Looks like you got an answer directly from Jaime 😎
Super interesting.. thanks for sharing!
Thanks for watching!
Is there a way to change the schema without going schema-less?
I believe you can do this, but I can't remember if I actually tried it.
onto the software. I'd like to make soft from my ukulele, keyboard, etc., but don't know how to transfer. Do I need a microphone or
SurrealDB thankfully doesn't require a ukelele, keyboard or microphone. Unless you're using it to store your band's music
Do you know how to get the attributes that can be SET in a RELATE?
not sure off the top of my head, maybe drop into the SurrealDB discord server and ask! They are really responsive.
This DB looks really interesting. I would love to try it for a future Rust project.
Does it cost anything to use? Is it available at any existing cloud hosting provider?
Nice! It's completely free to download and host however you like. But SurrealDB is aiming to also provide a managed cloud version of it that comes at a cost, which I imagine will be a pretty compelling value prospect.
You can run it yourself in a docker container on a VPS. Should be pretty simple.
Is it just the special query language and the websocket real time updates that makes it different from normal SQL databases? Everything it can do in this video can also be done in normal SQL databases. Also, is it ACID?
I think the main differences from traditional SQL databases are
* Under the hood it is a key-value store, which allows for basically infinite horizontal scalability because it lends itself well to sharding. Traditional SQL databases need something like Vitess to do this
* It supports graph operations
* Strict schemas are supported but not required
* Supports REST endpoints and end user auth out of the box
I'm probably missing something else but those are the first ones that come to mind. More broadly though, choosing a database paradigm is currently one of the most impactful decisions in any software project, and making the wrong choice can lead to immense amounts of pain. The main value of SurrealDB is that it is aiming to be aiming to be a jack of all trades, such that it can support any unforseen use cases that might arise after a database has already been chosen. For example - if initially a project just requires a KV store but a few years later a new feature requires a relational database. SurrealDB gotchu.
And I believe it is ACID, but check the docs and/or ask on the Discord server to keep me honest.
Can this access the file system from functions? Thinking in terms of writing files to other storage.
Would love a tutorial on the graph queries if you have them down. They can twist your mind when not familiar with the syntax.