Rust Powered Database SurrealDB (It's Pretty Ambitious)
Вставка
- Опубліковано 1 сер 2024
- A fast tour of the query language and capabilities of SurrealDB, the ambitious project that aims to blur the lines between major database paradigms.
00:00 Introduction
01:17 First Example
02:51 Schemafull Tables
03:45 SurrealDB CLI
04:02 Querying Multiple Tables Without JOIN
05:17 WHERE Clauses
05:33 Aggregation Functions
05:52 Graph Connections
09:23 Events
11:11 Conclusion
-
Stuff I use to make these videos - I absolutely love all of these products. Using these links is an easy way to support the channel, thank you so much if you do so!!!
Camera: Canon EOS R5 amzn.to/3CCrxzl
Monitor: Dell U4914DW 49in amzn.to/3MJV1jx
Lens: Sigma 24mm f/1.4 DG HSM Art for Canon EF amzn.to/3hZ10mz
SSD for Video Editing: VectoTech Rapid 8TB amzn.to/3hXz9TM
Microphone: Rode NT1-A amzn.to/3vWM4gL
Microphone Interface: Focusrite Clarett+ 2Pre amzn.to/3J5dy7S
Tripod: JOBY GorillaPod 5K amzn.to/3JaPxMA
Keyboard: Redragon Mechanical Gaming Keyboard amzn.to/3I1A7ZD
Mouse: Razer DeathAdder amzn.to/3J9fYCf
Computer: 2021 Macbook Pro amzn.to/3J7FXtW
Caffeine: High Brew Cold Brew Coffee amzn.to/3hXyx0q
More Caffeine: Monster Energy Juice, Pipeline Punch amzn.to/3Czmfox
Building A Second Brain book: amzn.to/3cIShWf - Наука та технологія
Wow! Thank you very much for this video Code to the Moon! Just the two of us at the moment, but my brother and I have huge plans for SurrealDB. We are working on lots of performance improvements, GraphQL integration, and our own key-value storage engine coming in due course. Thank you once again for this video!
Hey just wanted to say thank you for this amazing DB keep up the good work.
@@alandto3996 Thank you very much for the kind words. We certainly will!
Thank you very much! ❤️
Looks like a brilliant project. Would love to see key-values and websocket auth pubsub soon! 💛
@@fire17102 WebSocket auth already here Tami. Live queries and pub-sub coming very soon!
_Came for Rust, stayed because the unique abstractions in the design actually managed to make the idea of database structure actually appealing to me..._ 😹
Is it unique? Looks like OrientDB, which has been around for a decade.
Thanks Wolveric, glad it piqued your interest!
OrientDB is written in Java - I'm sure it's fast, but there's kind of an upper bound to how fast it can be...
Me too
This looks extremely interesting. I suspect this will grow very quickly.
Thank you very much @theherk! We have a lot planned on the roadmap!
Agree!
A directional edge connection is the first time I've seen an arrow operator feel appropriate
❤
In F# you use this operator
It's similar to the syntax of Cypher, the query language of Neo4j
Arrows for marking function output types and for dereferencing a pointer to a compound object both seem appropriate to me. Just my 0.002¢
I agree! Really the perfect operator for working with graphs
DEFINE, CREATE, RELATE, SELECT ... all six letter commands. I *love* it.
Wow, I hadn't noticed that!
keen eye
Ok, this is insane. I was dreaming of a DB that would do graph connections in trust to move away from neo4j , but this is amazing. An absolute killer.
I agree! It definitely has massive potential.
The content of your channel is real: great video + audio quality + thumbnails + transitions + rhythm and so much more.
Really inspiring!
Thanks for that super fast and amazing tutorial! 🙏🏻
Thanks for posting this! I've surveyed the DB landscape wayyy too much and still, this is my first time seeing this database! VERY impressed and intrigued. Also, great job on the overview! Really concise and thorough introduction.
Thanks for the kind words Mike, glad you found it valuable!!
Time to start playing with the new boy in the block. Thanks for sharing.
nice, thanks for watching!
at 9:15 you could of simplified the statement to be *SELECT armor:dragon
Thanks Dabbo will give this a try!
What a cool demo! It really showcases SurrealDB's simplicity and usability. Thanks!
Thanks and thanks for watching!
This looks really interesting. Having worked for many years with traditional SQL, and having migrated a few years back completely to NoSQL databases for speed and scalability reasons, I'm really curious to see how SurrealDB scales. If it combines the scalability of NoSQL databases with the ease and familiarity of SQL then could be a very exciting!
Agree, it seems like the team is aiming to do just that!
These awesome projects like SurrealDB are a result of developers using a language in which they don't have to care about memory and bang their heads when their application segfaults. Awesome work by SurrealDB devs!!
Holy crap. If this pans out to what it’s teaching for this could single handedly solve a lot of the database headaches.
I agree! Choosing a database paradigm is really challenging, the prospect of not having to make that choice upfront will be so liberating...
I'm sure this will become very popular quickly. It seems to be very well thought out and a great fit for a ton of use cases, it's written in Rust which is always a plus and it has super consistent and slick looking branding and all that. The logo looks so cool, I'm tempted to use surreal just so I have a reason for putting that sticker on my laptop :-P
I agree Flying Squirrel! To your point - In addition to being feature packed, it has incredible branding which I feel like matters more than most technical people would like to admit
I am brand new to DAW and soft soft - these tutorials are excellent an very helpful to get soone like up and running. Appreciate
Glad you found it valuable, thanks for watching!
I couldn’t believe you got that event to work just like that 👏
Thanks, I practiced a bit before recording the video 🙃
Those multi-cluster stuffs that are in "future" on surrealdb's roadmap are even more ambitious. If they can deliver on those and survive Jepsen, I'll tip off every hat to them!
100% agree, can't wait to see how things pan out. I actually hadn't heard of Jepsen, I'm glad you put this on my radar!
Fantastic video, loved it. I will be playing with it this afternoon.
Thanks Thomas, let us know how it goes!
The devs fpr this project should feel proud. Its not easy to create a whole new database architecture, let alone one that leverages the benefits of it predecessors. This will catch on
I agree that what they've done is really impressive! Seems like they have a ton of momentum already, hopefully it continues
I love it. Gotta test this
It's worth giving a try!
I'll definitely be trying this out on my personal app later!
Nice, let us know how it goes!
looking forward to learning from the rest of your videos. Subscribed!
Thanks Justine, very happy to have you onboard!
You guys are great and SurrealDB genuinely feels interesting. Can't wait to play with it.
Thank you very much for the kind words Onizuka ❤️
It's worth giving a try for sure!
This is such an informative video. Thanks for creating. I am going to try myself.
Thanks Singh! Fantastic that you are going to start creating! Please let me know if there's anything I can do to help.
@@codetothemoon Thanks for replying. Will definitely do 😊.
Very helpful, and surprisingly therapeutic
Nice, glad you liked it! "Therapeutic" is something I'm very happy to have achieved!
This is cool and interesting. I might try it out, but I’ll be keeping work with single databases for their specified purpose.
Nice, yeah definitely something to explore but probably not ready for production yet. Also, I see you are a fellow Babylon 5 fan!
This is absolutely the level up!
I agree, the SurrealDB team is moving mountains!
It’s totally worth learning new skills cause then you get excited and confident to create
It's true! 🔥
First vid I watched *exploring* SurrealDB.
Looks like something I gotta know, being a big PostgreSQL fan.
Thanks :)
Thanks for watching Tobias!
That's super cool; thanks for bringing this!
Thank you very much David
Thank you David!
Thanks for watching!
Thank you for this video,
You rock!
Thanks for watching Scott!
The event thing was crazy, amazing
agree!
This looks pretty interesting. I’ll have to play with it some
it's worth checking out!
As soon as LIVE queries are implemented, I am switching for sure :)
they do sound pretty handy!
I'm starting to see more Obsidian in rust dev videos... I'm inclined to think you guys have super meaty brains... 🧠
side note let me know if someone is making Obsidian in Rust... so I can meme about it.
@@leeroyjenkins0 Obsidian MD is not open source, so I don't know what it's written in and note-taking being called a fad is funny can't tell if you're trolling or legit.
Funny story about why I used Obsidian for this - there isn't any readily available SurrealDB SQL syntax highlighting, so I had to hack Obsidian to use the same custom JavaScript syntax highlighting that the SurrealDB website uses 👍
Really strong stuff you serving today!
Authors really taked their code to the moon 😄
lol thanks! 🚀
Good to know, looks very interesting, I will still use the rock solid postgres but this kind of project can have they use cases
hard to go wrong with Postgres!
tbh I can't wait what sorts of projects the community will come up with now that we have rust
agree, there are so many interesting Rust-powered projects popping up, even CLI tools to replace ones that have been around for decades...
This is the most interesting thing I have seen in a while in the tech world
Same, it seems pretty disruptive if the vision fully materializes!
@@codetothemoon yes, especially for small projects it's like having a firebase in a box, at least for me having something like this, if the security layers and event emission works properly, completely removes the need for a server between the client and the DB. And the fact that you can embed js functions into fields makes it even possible to have some business logic too (with limitations I guess)
Definitively not the way I would use for a well structured long-term project, but for everything else I don't know... on paper it kinda fits perfectly
looks amazing!
Glad it piqued your interest as well!
This is very interesting to me. The features are indeed ambitious but quite compelling. Would require a lot of testing before using it in a production product I suspect but I want to try it in my normal research and experiments code. As usual, well explained and informative.
I think SurrealDB is really interesting too, but yeah I wouldn't use it in production quite yet. I'm definitely looking forward to watching how the project develops!
Nice video and nice product!
For the curious, this db uses TIKV under the hood which uses RockdDB, an LSM tree based database. Cassandra uses the same technology.
Good to know, thanks
Thank you Ismaj. We are working on our own key-value store which will bring some additional benefits to SurrealDB in due course.
@@SurrealDB Do you think it’s more effective to build your own storage engine right now or have a pluggable architecture?
I ask because there’s several features that I think would be great if users had a choice and they seem complex to implement in that way out of the box, so if users can choose the right storage engine for their use-case, it would probably be better.
Some of the feature choices I’d like to see are LSM vs. b-trees, enabling/disabling record versioning for time-series data, enabling/disabling zero-copy for websockets (I really want something to dethrone Kafka that will integrate natively with a primary data store (Sorry, Redis. Keep dreaming)), and compression algorithm choices. Some of those aren’t even really in current storage engines, so I’d be really interested to see how they can be implemented.
I’ll certainly be keeping an eye on this project. Wish you the best of luck!
@@CharlesChacon We always intend to support lots of pluggable storage engines to give developers the choice. Our own key-value store will enable us to offer additional functionality down the line that we can't currently do with the current architectures out there.
Thanks, and thanks for the info - I wasn't aware of TiKV's relationship with RockDB!
This is super cool, would love to check it out
Thank you Zaha! Join the Discord or GitHub if you have any questions - surrealdb.com/community
Yeah I think it's worth a look!
This looks very promising!
Thank you very much @Elf!
I agree!
Looks like I found a new go-to databse for prototype projects (and non-prototype ones once it matures a bit)
same here!
this is amazing!! thanks
thanks, yeah I think SurrealDB is definitely worth keeping an eye on!
Keep up the good work!
thank you!
If they manage to do the websocket part, it will immediately become much more useful.
Agree, it will be a game changer!
It's pretty SURREAL that a database engine like this exists... Too bad I work as a front end dev and never have to deal with databases
Perhaps this tech could have some side-project potential for you?
It's ok, given that there seems to be a new JS framework every day, there's bound to be a SurrealJS at some point... 🙃
You can start creating your side projects with Nodejs just for fun :D and connecting with SurrealDB
Super interesting.. thanks for sharing!
Thanks for watching!
Thanks so much! I'm actually just beginning to get my feet wet in the backend development world, and I have to say it's a very exciting time to be doing so. I've got just enough experience with a couple other relational and graph databases to see the massive advantage of some of these features, but I've yet to really dive into Rust. I'm getting the impression that its time is coming soon, but according to the forum murmurings that's an impression that Rust has been pretty good at creating for quite some time without a lot of real progress. However, looking at the hard facts with this framework, it seems like they might finally have a real game changer on their hands. I hope it works as smoothly as it appears to.
Thanks for watching! Fwiw you don't need to have any Rust knowledge to use SurrealDB! I agree it does seem like a game changer - the current version does have some of the quirks that you'd expect in any beta software, but overall it seems to work really well in its current state!
@@codetothemoon I managed to get an instance served up locally and it worked great, but I'm wondering if it's possible to create a cloud-hosted instance through something like Google Cloud Functions or if that is something I will have to wait for until they release their cloud hosting features?
SurrealDB seems very interesting. If I ever am granted the luxury of time I'll gladly experiment a bit.
Nice, it's worth checking out!
Wow that looks bonkers good!
Thank you very much Ratstail91 ❤️🙌
I think "bonkers good" is a fitting description!
Super cool. I just need to figure out how to use this at work.
Hi Nathan. Thank you very much. Join the Discord or GitHub if you have any questions - surrealdb.com/community
Convincing others to buy in is the hardest part! Even for game changing technologies.
WOW, just so simple to use!!
I agree!
After web framework galore we now have database galore
For reals!! This is getting ridiculous!! 😂
Hah yeah, I'm skeptical of all new projects - JS, Database, or otherwise - but this one really seemed to stand out from the crowd.
This feels surreal. The web UI seems not ready yet but the project is interesting.
Surreal indeed! Yeah my understanding is that the UI will be available relatively soon.
This was helpful!
Thanks for watching!
There’s a JVM based multi-storage mode database called OrientDb. They have the same concepts/similar concepts. Distributed fault tolerant. Document store, relational, and graph, schema is optional, integration with kafka, and more. Uses SQL variant for query. The documentation is a bit lacking though.
Nice, this actually wasn't on my radar. How does it stack up performance-wise?
@@codetothemoon I haven't done any performance testing. The backend storage engine is very robust and a lot of integration with other systems like Kafka, MQTT, etc. It's a pretty "featureful" database.
Interesting new DB, love Rust too.
I agree!
Very interesting!
Absolutely revolutionary. Got a question about the performance of the edge queries - while most were in the 100 microsecond range, the edge queries were in the 500 microsecond range. Does that give anyone pause? I have no idea how this stacks up with other graph database tech so it could be a nitpick.
Great question - I hadn't delved too deeply into performance benchmarks. In the docs they openly state that the performance currently doesn't yet quite stack up to the more established databases, so I'm wondering if we should wait for a more stable release before reading too deeply into them. Definitely something to keep tabs on though, and it's not unreasonable to be cautious about types of queries that seem to be slow in the beta.
This is really cool and all, but when evaluating new things to work with, "version control" is the first thing that comes to mind... if this can be easily integrated into Git-like repos, it'd be a gamechanger for me. Also... localization... and transaction support across tables.
Agree, version control is huge, and I'm not sure if Surreal plans to support it. At one company we had implemented our own homegrown version control inside Postgres, and the number of bugs and general pain it caused over many years was hard to fully quantify. It'd be nice to have something like that built in to the database.
informative video good sir, Gracias
Glad you found it valuable and thanks for watching!
I really like what I'm seeing, I just hope it won't end up like rethink db
Thank you very much indeed @gorudonu ! We have lots planned for SurrealDB!
Agree it seems promising!
I'd really like a nice way to represent rust style enums in my database. I've never seen a database do that before. I would use Surreal for everything if I could do that
That would be cool!
Looks amazing!
As a data analyst/engineer the one thing I really want to know is: Does this support complex joins, subqueries & cte?
My understanding is that it aims to support all of the complex types of queries you'd expect to be able to perform on a traditional relational database. JOINS specifically are eliminated as you are expected to use record links, graph edges and FETCH instead, but you can achieve basically the same thing. I believe subqueries are currently supported (but maybe ask on the SurrealDB discord server to be sure). Not sure to what level CTE-like patterns are currently supported or what is planned there, again maybe check with Tiobe and Jaime.
I'm hooked
nice!
Nice tutorial,
Thanks and thanks for watching!
Great video, thanks for sharing. SQL novice here, would you mind sharing the SQL/SurrealDB queries? Would love to look them over to better understand.
Thanks - unfortunately I don't have the queries posted anywhere, but you can check out README.md on the SurrealDB GitHub repository for some good examples!
I'm ready to marry this db
💍
😂
hah me too!
Clever design.
I agree, I'm looking forward to see where they take this thing!
This looks amazing for web applications and small relational database. Is the ambition to be a SQL alternative for everything including bug data storage or data warehousing?
Excelente!!
thank you!
Great Video! Im new on soft soft, and tNice tutorials tutorial really helped
Thanks for watching!
great tutorial. at 10:36 maybe we should add player= $after.id field so we can fetch the player who levelled up. just an idea that I learned from previous 10 minutes. thank you very much.
Mylosp just a legend really aren’t ya
I try to be! 🦄
I checked everything is good to download.
Yeah SurrealDB can be downloaded for free!
A comparison with edgedb would be interesting. The benchmark, the facilities differents edgeql and the same with surealdb....
I'd love to see a video showing how to use it with Rust. The documentation doesn't seem to exist yet.
Good idea, I've added it to the video ideas list! There are definitely some gaps currently in the documentation. But the SurrealDB team is really responsive on their Discord server, you might try getting help there!
Can this access the file system from functions? Thinking in terms of writing files to other storage.
Would love a tutorial on the graph queries if you have them down. They can twist your mind when not familiar with the syntax.
What kind of font are you using? It looks so nice!
Can we get update on it since whole year passed. Maybe to cover updates and upgrades? Very nice video.
thank you! actually currently working on a new video that will feature SurrealDB (but won't focus specifically on it). I'd love to do another full video including all of the new stuff since this one, we'll see if it bubbles to the top of the video idea list 😎
thank you💯🥹😅🙂
thanks for watching!
RetroSun thanks to other comnts i gotchu; click on gms when you’re in the channel rack to open up the gms nu and in the light blue
?
So, if IDs are non-sequential (this is a good thing), is there a way to generate new ids? For example, a syntax like CREATE player:new SET name = "Bob"; ?
I can RELATE, Bob
I have a soft spot for databases and SurrealDB sounds very interesting. I must say that I have often problems with joins, so not needing it sound great. I am not proficient with rustlang yet, so I wonder if I cound use some other language with SurrealDB?
Kimmo - absolutely, I believe they have client libraries in several other mainstream languages. No Rust knowledge necessary.
I like the idea behind this. I'm just not sure if it'll fit any of my projects. I have things split into three distinct layers SvelteKit (Cloud Edge Deploy) -> FastAPI (Local Python Microservices) -> PostgreSQL (Local DB). Mostly I prefer the DB to be as plain as possible which is just storing the data and the Local Microservice layer handles Auth, Data validation, some caching, etc, etc. This means what I'd really be looking for if I was switching to a new GraphDB is a Rust or Python equivalent of FastAPI to connect to SurrealDB. It'll be interesting to watch this project grow and see what other utilities pop up from this.
Hey Matt! I understand your tech stack position. We have some awesome things planned for SurrealDB so hopefully we can sway your mind in due course 😀
@@jaimemorganhitchcock6195 If you get full-text search in there, I’ll bite!! 😄👍🏽
Just checked the Docs and I'm blown away by the full-text support. I'm blown away by everything actually. They actually have a bunch of drivers and great extensibility points. Wow!!
It's probably not a good fit for every use case, especially when one db paradigm can handle all of your access patterns and you don't need the built-in auth and REST APIs that Surreal has. But yeah I agree it'll be cool to see how the project develops!
Could we say that this is SURREAL
❤❤
Indeed!
Love it
Nice, thanks for watching!
thanks for your stuff, do you have any recommendable rust course?
I haven't heard of any courses for SurrealDB yet - it's still very new, but I'm sure there will be one before too long.
@5:47 It works for me with SELECT math::sum(strength) FROM player GROUP ALL; (I'm using version 1.2) - so use GROUP ALL, rather than GROUP BY ALL
It’s pretty cool for a multi-model approach. Right now I’ve switched to graphs exclusively because moving away from SQL and tables has been easier for me. I haven’t seen the need to keep them.
Graph is definitely the future @Khari99! That is exactly why we wanted to merge Graph with traditional database concepts.
@@jaimemorganhitchcock6195 I understand the desire to combine different DB models. A better metaphor for how I see this is sticking with a hybrid vehicle instead of going full electric because it’s assumed that having both gas and electric together is more convenient. Convenient sure but is it a better design? I don’t believe so. Currently the database I mainly use is Dgraph. I’ll see how Surreal compares but I haven’t found anything that’s close that simplifies my architecture as much as Dgraph.
Dgraph is definitely a cool database @Khari99. With regards to multi-model, we wanted to ensure that SurrealDB was easy-to-use coming from all different database backgrounds. In addition, the design is such that the performance isn’t affected regardless of what data model you use in SurrealDB - obviously there are no JOINS, so SurrealDB is definitely closer to MongoDB/Dgraph as a concept. We have some really great things planned on our roadmap, so hopefully we’ll sway your mind in the not-too-distant future 😊!
@@jaimemorganhitchcock6195 While I like Dgraph a lot, I don't think it's perfect. Outside of the GraphQL API, I feel like there are a lot of ideas that you can pick up on to make SurrealDB better.
1. Automatically generated CRUD queries from a schema definition. I've only seen this implemented with GraphQL so I don't know if this is possible for your architecture. But this significantly reduces how much code needs to be written and maintained in the server.
2. Implement lambda functionality. Dgraph has a philosophy of moving the DB to the middle layer and allowing queries to trigger lambdas that either run custom computations or can reach out to a server externally for some request if needed. I really like this approach as it helps further reduce architectural complexity. But right now I'm limited to JavaScript lambdas. Rust lambdas would be amazing.
3. Auth & Access Control patterns. I looked through the documentation for Surreal and haven't seen anything around this topic. I understand the philosophy of leaving this up to server logic but if these features are baked into the database, it's one less thing for me to worry about. Even though I moved away from Fauna, they have this integrated into their DB, which is why I tried them out in the first place.
4. Logical Inference Rules. I've only seen these implemented in TypeDB. one of the things that Dgraph is missing that almost makes me want to develop a DB in Rust myself is the ability to create a knowledge graph intuitively. If you could look into how TypeDB accomplishes this and think of a similar implementation, this would be awesome.
5. Indexes. I also didn't see this in the documentation. Dgraph has indexes that enable full-text search (along with many other index types) out of the box. I would rather not have to implement elasitcsearch separately.
This is really cool, I haven't heard of many folks switching from pure relational to pure graph!
What is the encryption at rest support?
I love it, except for one thing: is there a simplified schema definition syntax? I support defining strong schemas, but the code for doing it looked really verbose, too wordy vs normal SQL
yeah afaik this is currently the only way to do it, at least the only way that exists in the documentation. The team seems really receptive to feedback, maybe hop on their discord and shoot them a message!
Summary:
Surreal DB is a new database that aims to take the best aspects of each major database Paradigm and combine them all into one database. It is schema-less by default, has a sql-like query language, and supports record links to establish relationships between records in different tables. It also has a graphql-like query language for fetching data from multiple tables at once, and supports websockets for real-time updates.
Sounds about right to me!
Looks like a great product. Thanks. How to we do a full text search on a field? Is it efficient (unlike oracle search where % wildcard or any text partial search will kill the efficiency while search?)
The docs mention support for full-text indexing, but I'm not sure exactly the steps required to enable that for a field (maybe just create a specific type of index on it?). The docs do appear to have some holes in them at the moment, but the team is extremely responsive and helpful on their discord server, maybe drop in and ask them!
@@codetothemoon Thank you!!
Do you know how to get the attributes that can be SET in a RELATE?
not sure off the top of my head, maybe drop into the SurrealDB discord server and ask! They are really responsive.