Complex Schema Design with Drizzle ORM | Common Patterns

Поділитися
Вставка
  • Опубліковано 27 вер 2024

КОМЕНТАРІ • 85

  • @syntaxfm
    @syntaxfm  4 місяці тому +6

    03:30 Drizzle: A Different Kind of ORM
    04:37 Codebase Intro / Setup
    07:51 Database Structure High Level Overview
    09:20 Drizzle Studio Setup
    12:06 Creating the Schema with Drizzle
    14:10 Drizzle Foreign Key Constraints
    14:45 Drizzle Composite Key Constraints
    15:19 Drizzle Index Constraints
    16:38 Drizzle Migration Setup
    23:18 Seeding the Database with Drizzle
    32:38 Querying the Database with Drizzle
    34:45 Related Queries with Drizzle
    35:39 Creating Table Relations
    37:18 Drizzle Generated SQL Queries
    38:51 Drizzle Join Query
    40:34 Nested Where Queries with Drizzle
    44:00 Final Thoughts on Drizzle
    Listen to Wes and Scott talk about how they use Drizzle in their apps: syntax.fm/show/721/you-should-learn-drizzle-the-typescript-sql-orm

  • @anuzpandey3326
    @anuzpandey3326 2 місяці тому +6

    This is by far the best tutorial on Drizzle. Separate schemas, seeders. Awesome man.

  • @ddrweb_
    @ddrweb_ 4 місяці тому +4

    I started learning SQL, feels like this is exactly what I will need once fully understand what I am doing. Nice, clean patterns btw. Thanks for this you did an amazing job!
    This new role at syntax really suits you, everything you do has so much value to it, keep being amazing.

  • @TheHermitHacker
    @TheHermitHacker 4 місяці тому +6

    I'm totally stealing the use of the Zod for getting the env vars checked before doing anything else. This actually would have been nice a few days ago when I couldn't figure out why my DB connections were not working in production. I wasn't getting any errors in logs! This is such a great idea.

    • @iukeay
      @iukeay 3 місяці тому +1

      Legit, I never have never seen that done until this video!
      Clever. Clever

    • @fennecbesixdouze1794
      @fennecbesixdouze1794 3 місяці тому

      Yes validating your .env is good, but totally overcomplicated to use Zod for this use case just write simple js.

    • @TheHermitHacker
      @TheHermitHacker 3 місяці тому

      @fennecbesixdouze1794 I've implemented a caching mechanism that skips the check after the first successful pass. Each fresh deployment gets this extra step on first http request.

    • @AnishBishnoi-wf7lv
      @AnishBishnoi-wf7lv 19 днів тому

      @nicholas do you have any public repo how you are using zod in ur code. if you have then let me know please.

  • @chill-hot-stream
    @chill-hot-stream 4 місяці тому +1

    cj never stop releasing videos .Your coolify video from last week helped me manage my server effectively.Was using normal terminal but now everything is simple and easy to setup effectively

    • @jbphilippi
      @jbphilippi 4 місяці тому

      I agree and hope he will do more coolify videos!

  • @francescoimpellizzeri1345
    @francescoimpellizzeri1345 2 місяці тому

    This is great! Concise, clear, elegant patterns and implementation! We need more of this 🙏🏻

  • @Plaswin
    @Plaswin 4 місяці тому +24

    Couldn't follow him before in streams because I get distracted by that format. But this edited presentation are absolutely awesome

    • @ofeenee
      @ofeenee 4 місяці тому

      I could not agree more!

    • @Anbaraen
      @Anbaraen 4 місяці тому +1

      Stream vids felt second-best (this is pretty much always the way with stream vods unless you use Theo or CJs old approach of heavily editing them down). This is perfect.

  • @paulclarke4099
    @paulclarke4099 4 місяці тому +2

    Your videos are fantastic and always so well explained, thanks so much CJ 💯😃👍

  • @Stoney_Eagle
    @Stoney_Eagle 4 місяці тому +2

    Nice job! I have watched many videos to make sense of the chaos in de docs and all of them combined missed a lot of information that I needed and is shown here.
    I wish I had this video when I started using it.

  • @joelkuijper
    @joelkuijper 4 місяці тому +2

    This is awesome, thanks for the hard work your putting into these videos CJ!

  • @eleah2665
    @eleah2665 2 місяці тому

    I just noticed this. Really good. ("We can get more complex than that!") Thanks.

  • @IsakFilms
    @IsakFilms 4 місяці тому

    Great video! I've been learning Drizzle/SQL the past months and this was a great refresher. It really helped me understand relationships better

  • @kuldar
    @kuldar 4 місяці тому

    Absolutely love this new content from CJ. Thank you for making Syntax even better!

  • @czhablog9559
    @czhablog9559 3 місяці тому

    Bro this video is pure gold! Thanks for sharing your amazing technical knowledge to the world.

  • @MrAbuYT
    @MrAbuYT 4 місяці тому +1

    Awesome! Thank you for this very helpful explanation. Good job CJ.

  • @truthzp
    @truthzp 4 місяці тому +1

    Thanks for the brilliant content! It's exactly what I need now.

  • @prashlovessamosa
    @prashlovessamosa 4 місяці тому +1

    Great explanation CJ thanks
    Please keep uploading good stuff.

  • @fennecbesixdouze1794
    @fennecbesixdouze1794 3 місяці тому +1

    @39:40 You could use postgres array_agg and add a group_by clause if you want to aggregate up the menu items in the sql syntax.

  • @VincentFulco
    @VincentFulco 4 місяці тому

    This was an outstanding tutorial. Will watch a few times. Thanks!

  • @zhanezar
    @zhanezar 4 місяці тому

    another great video thanks i really liked the idea of using Zod for Environment variables instead of adding another libray , thanks CJ

  • @vitalikda
    @vitalikda 4 місяці тому +1

    Omg! It's pure gold! Thank you

  • @brennenrocks
    @brennenrocks 3 місяці тому +2

    How do you run migrations on your Production DB? When in a production environment typically you won't have your devDependencies which means drizzle-kit won't be there which then means the migration script will not run because the defineConfig is coming from drizzle-kit

  • @yantakushevich1014
    @yantakushevich1014 3 місяці тому

    Thanks for the video! Would be great if you also show some examples of inserting data.

  • @bashiryousufy1
    @bashiryousufy1 4 місяці тому

    CJ is the best, been watching him on Coding Garden

  • @stephendeese97
    @stephendeese97 Місяць тому

    This was sooooooo Good!

  • @AlexanderBelov-y8o
    @AlexanderBelov-y8o 7 днів тому

    Professional course!

  • @mehmethanifierenler3612
    @mehmethanifierenler3612 2 місяці тому +1

    thanks to you very very

  • @petermckeever2360
    @petermckeever2360 4 місяці тому

    Well this was incredibly helpful

  • @devinschumacher
    @devinschumacher 11 днів тому

    would you ever manage multiple repos/projects schemas from a central repository for convenience?

  • @nuancetone
    @nuancetone Місяць тому

    What a great video. Trying to understand Drizzle, coming from Supabase and Prisma, this was a great explanation of the schema structure. Have a question, I'm using vercel/postgres and the docs from Drizzle are light on setup, I'd like to use a similar approach to yours here but the vercel adapter doesn't surface the same function layout of the Postgres one here, have you done this with Vercel?

  • @paoloricciuti
    @paoloricciuti 4 місяці тому

    I think i actually did the same error when i used Drizzle with sqlite but from what I've researched closing the db connection is not handled by drizzle so with PG you can use the Pool to use a connection pool but you have to find ways to do the same with other drivers otherwise you will either clog up or all your connections will pile up. This is obviously less of a problem with serverless but i think is worth mentioning.

  • @pavank4466
    @pavank4466 4 місяці тому

    Great video, Thank You 👏

  • @paulorodriguez6288
    @paulorodriguez6288 5 днів тому

    I have been doing relations the wrong way I'm drizzle 😔, thank you for the video, I have some code to fix

  • @haziqali6024
    @haziqali6024 3 місяці тому

    great video! I have just started to look into drizzle to integrate with expo-sqlite. I had a question:
    Is there any way to generate schema.ts file automatically from an existing sqlitedb which already has sql schema defined?

  • @Trager95
    @Trager95 2 місяці тому

    Why do you use lower case underscored for columns but camelcase on drizzle side?

  • @joaquimley
    @joaquimley 4 місяці тому

    Cj you are amazing ☘️

  • @thomastaylor1799
    @thomastaylor1799 Місяць тому

    Have you been able to generate a table with an enumerated field. I have tried both PGenum and PGSchema and have not gotten this to work.

  • @ArionKosturi
    @ArionKosturi 4 місяці тому

    Thank you CJ

  • @ayushgogna9732
    @ayushgogna9732 4 місяці тому

    18:07 actually in the version @0.21.0 there is a way to handle migrations in drizzle without a migration file with "drizzle-kit migrate"
    No more push:pg, generate:mysql. Just use push, generate and specify dialect in config once!

    • @syntaxfm
      @syntaxfm  3 місяці тому

      I have updated the repo with a note for anyone that comes across this: github.com/w3cj/bytedash/#drizzle-kit-updates

  • @phantazzor
    @phantazzor Місяць тому

    Noob question, why would use an ORM as much as easy and performant it can be over a query builder like Kysely? Adding an ORm will just make it slower no? Is it only for the schema building part?

    • @syntaxfm
      @syntaxfm  Місяць тому

      This will really depend on the ORM and it's internal implementation. Drizzle is very fast and has some performance metrics posted here: orm.drizzle.team/docs/perf-queries
      Usually it is a trade off between ease of use / developer experience / maintainability and performance.

  • @brennenrocks
    @brennenrocks 4 місяці тому

    What are your thoughts on using UUIDs for primary keys instead of `serial` in Postgres?

    • @danfascia
      @danfascia 3 місяці тому

      Just add another column for uuid as well as id(int) because indexing uuids is expensive

    • @brennenrocks
      @brennenrocks 3 місяці тому

      @@danfascia what's the point of having the uuid if you aren't going to do lookups on it or index it?

    • @danfascia
      @danfascia 3 місяці тому +1

      @@brennenrocks not what I said. Use integer as your primary key index and use that to carry out internal relationships as indexing integers in this context across relationships faster. No issue having a uuid index for direct item lookups

  • @officialdanieldsouza
    @officialdanieldsouza 4 місяці тому

    Hey awsome video, this video really clarified organization of my files in drizzle. I was wondering what is the best field type to use for the primary key, I know in your example you have used serial but I was just wondering in terms of security, performance, running out of integers. Would love to hear your thoughts on this dude!

    • @syntaxfm
      @syntaxfm  4 місяці тому

      There's always a trade off in performance / usability.
      string ids (guids) are good for large distributed systems where you might be inserting many many rows all at once across several nodes. guids can also be generated outside of the DB, so you could generate them before insert.
      auto incrementing integer keys might be vulnerable to enumeration attacks if your application code does not protect against it. e.g. someone might try requesting /users/1, /users/2, /users/3 etc. but this is only vulnerable if you have not locked down the API.
      postgres serial / int maxes out at 2,147,483,647 but if you are worried about running out, you could use bigint / bigserial which maxes out at 9,223,372,036,854,775,807
      I have a lot of apps still in production that used int / serial keys, but those apps only have a few thousand users. Worrying about running out of primary keys / insertion performance is likely only needed if your going to be working on things at a much bigger scale.
      -CJ

  • @w3mw
    @w3mw 4 місяці тому

    Great video as always CJ! I’m building a project right now with nest js and prisma as backend and I thinking of next js with drizzle in front end is that possible tough or should I use the same ORM in both ways? Never tried drizzle before so I’m not sure if it works good with nest js but I guess it does 😄 what do you recommend?

    • @syntaxfm
      @syntaxfm  4 місяці тому

      Drizzle is just typescript, so it will work just fine with nest js.
      It is possible to use 2 different DB libraries to connect to the DB, but if you can consolidate to a single one, that would be ideal. If you are working in a mono repo, you could also have a library specifically for the db / queries and share this between projects.
      -CJ

  • @tobbymarchal3140
    @tobbymarchal3140 2 місяці тому

    Can we use drizzle in deno

  • @charlesokorobo508
    @charlesokorobo508 Місяць тому

    Dear CJ,
    I am trying to use Drizzle ORM, but many things are not clear to me:
    I want to create 4 Database Tables:
    a.) Users Table: a table that will have normal user information.
    Then there will be 3 additional tables which records and stores a user's interaction on the website.
    These are:
    b.) Posts Table: a table with a One-to-Many Relationship -- where a user will have many Posts, many Comments, many Likes, many Thumbs Down, many Friends, many Groups, and many Pages, stored into this table.
    c.) Purchases Table: a table which is a One-to-Many Relationship Table -- where a user will have many transactions listed to their name.
    This table only keeps stock and inventory of all the Purchases that a user makes on my website.
    d.) Points Table: a table which stores all the many Points, many Awards, and many Badges that a user will earn on the website.
    QUESTION 1
    In creating a Schema, I'm I going to create One(1) Schema per Table? -- in which case I will have 4 different Schemas?
    Or
    Is it one(1) Schema I will use to declare all the structure of the 4 Tables?
    2.) How do I create a Schema that will take note of all these four(4) Tables, and is able to link them to work together?
    This is a big problem for me, and I am hoping that you can guide me on this, because I didn't see how to do it in the Documentation, and I've spent 8 days on it without hope on what to do exactly.
    Regards.

    • @syntaxfm
      @syntaxfm  Місяць тому +1

      A schema lists all of the tables and relationships in your database.
      In the example in this video I have 1 schema defined here: github.com/w3cj/bytedash/blob/main/src/db/schema/index.ts
      Notice how the schema is the combination of all table definitions. I defined each table in it's own file and re-export it here. But you could also define all of the tables in a single file and then export an object with each of the tables in it like this:
      const schema = { users, posts, comments, likes, groups };
      export default schema;
      The relationships between the tables are defined by the foreign key constraints in each table definition. For instance, a comment has a userId. We link this to the user table with the references constraint:
      {
      // rest of the table
      userId: integer("user_id").notNull().references(() => user.id)
      }
      This is all that is required to define the relationship between the tables at the DB level. You could then perform joins across tables with foreign key constraints. orm.drizzle.team/docs/joins
      To use the ORM query syntax for easier linking of data, you have to define the relation separately. This lets us use "with" during ORM querying to easily join the tables together. orm.drizzle.team/docs/rqb
      See the example here that allows comments to be queried with users: github.com/w3cj/bytedash/blob/main/src/db/schema/comment.ts#L29

    • @charlesokorobo508
      @charlesokorobo508 Місяць тому

      @@syntaxfm
      Dear CJ,
      You are the best there can be. Oh my gosh 😱 You made my day!
      Thanks so much.
      HEARTFELT REQUEST
      There are many newbies and inexperienced people like myself out there, who are just coming into the Web Development space (I finished learning JavaScript in June 2024, learned Svelte and SvelteKit in July 2024, and learned PostgreSQL this month in August 2024 ).
      There are many newbies like me out there, who would love to use Drizzle because they want to use SQL-like Database like MySQL or PostgreSQL.
      While they, like me, have learned SQL (either through MySQL or PostgreSQL), it is (kind of) true that Drizzle is a different implementation of that SQL we learned.
      Drizzle has features, rules, structures, and layers that are on top of SQL, making Drizzle a tool that we have to learn (far different) from the base SQL that we've learned.
      THE REQUEST
      Can you do a Drizzle series -- where you will teach(not show) new Drizzle users, the things they need to know to set up Drizzle, and the steps they need to follow to get Drizzle running on their local development environment?
      The series will touch on:
      -Setting Up Postgres.
      -Create Schema.
      - How to create a Table.
      - How to create multiple tables.
      -Table relationships (how to connect multiple tables)
      -Migrations
      -Push Migrations to Database.
      -Basic Crud Operation in Drizzle.
      -Basic Queries in Drizzle
      Inserting Data into Tables.
      Hosting/Deployment (Self Hosting or platform hosting like Supabase, Vercel, Neon and so on).
      IMPORTANT NOTE
      I am a novice in Web Development and Drizzle. This means that my curriculum is still novice in its contents for what beginners need to know.
      You are the expert, and can complete it with the missing things about Drizzle that beginners need to know.
      I am able to create this curriculum because this is almost everything I am struggling with, in being able to use Drizzle.
      MOVING FORWARD
      Now, I'm not in a hurry to complete my Application, because it is my personal project which I hope to launch for my future endeavour whenever it completes.
      The meaning here is that, while I'm figuring things out in Drizzle (little by little), your creating of the series could still be useful to me.
      Highest Regards to you:
      Golden crown 👑 presented to you, and a thousand hugs 🤗

    • @charlesokorobo508
      @charlesokorobo508 Місяць тому

      @@syntaxfm
      Dear CJ,
      You are the best there can be. Oh my gosh 😱 You made my day!
      Thanks so much.
      HEARTFELT REQUEST
      There are many newbies and inexperienced people like myself out there, who are just coming into the Web Development space (I finished learning JavaScript in June 2024, learned Svelte and SvelteKit in July 2024, and learned PostgreSQL this month in August 2024 ).
      There are many newbies like me out there, who would love to use Drizzle because they want to use SQL-like Database like MySQL or PostgreSQL.
      While they, like me, have learned SQL (either through MySQL or PostgreSQL), it is (kind of) true that Drizzle is a different implementation of that SQL we learned.
      Drizzle has features, rules, structures, and layers that are on top of SQL, making Drizzle a tool that we have to learn (far different) from the base SQL that we've learned.
      THE REQUEST
      Can you do a Drizzle series -- where you will teach(not show) new Drizzle users, the things they need to know to set up Drizzle, and the steps they need to follow to get Drizzle running on their local development environment?
      The series will touch on:
      -Setting Up Postgres.
      -Create Schema.
      - How to create a Table.
      - How to create separate multiple tables.
      -Table relationships (how to connect multiple tables)
      - Meaning and Explanation of Composites.
      -Migrations
      -Push Migrations to Database.
      -Basic Crud Operation in Drizzle.
      -Basic Queries in Drizzle
      Inserting Data into Tables.
      Hosting/Deployment (Self Hosting or platform hosting like Supabase, Vercel, Neon and so on).
      IMPORTANT NOTE
      I am a novice in Web Development and Drizzle. This means that my curriculum is still novice in its contents for what beginners need to know.
      You are the expert, and can complete it with the missing things about Drizzle that beginners need to know.
      I am able to create this curriculum because this is almost everything I am struggling with, in being able to use Drizzle.
      MOVING FORWARD
      Now, I'm not in a hurry to complete my Application, because it is my personal project which I hope to launch for my future endeavour whenever it completes.
      The meaning here is that, while I'm figuring things out in Drizzle (little by little), your creating of the series could still be useful to me.
      Highest Regards to you:
      Golden crown 👑 presented to you, and a thousand hugs 🤗

  • @russpalermo
    @russpalermo 4 місяці тому

    cool CJ. On seeding the db have you thought about or tried running the seeding as a single transaction in postgres? I think I did this in a prior project using knex where I set the table fk contraints to 'deferred' the ran a promise.all on the seed files where the commit did not happen until all data was stuffed in the tables.

    • @syntaxfm
      @syntaxfm  4 місяці тому +1

      Yes this should be easy enough with drizzle, just wrap the insert code into a db.transaction call: orm.drizzle.team/docs/transactions

  • @premdasv8
    @premdasv8 4 місяці тому

    What's the font you are using in your vscode?

    • @syntaxfm
      @syntaxfm  3 місяці тому

      It's called anonymous pro - www.marksimonson.com/fonts/view/anonymous-pro

  • @fennecbesixdouze1794
    @fennecbesixdouze1794 3 місяці тому

    "Run as a single statement" doesn't really mean much in terms of performance, if you're doing nested selects you should take the produced sql statement and put it through a PLAN analyzer to make sure it's performant and getting all the joins etc it needs without full table scans.

  • @mrlectus
    @mrlectus 22 дні тому

    For development to production with all this ORM. Cause with Prisma in production i nearly kill myself 😅

  • @polioann
    @polioann 4 місяці тому

    How about recursive schemas? (like the comment section)

    • @syntaxfm
      @syntaxfm  4 місяці тому

      Self referencing keys is possible and set up in the same way: orm.drizzle.team/docs/rqb#one-to-one

    • @polioann
      @polioann 4 місяці тому

      @@syntaxfm Nice! Thx

  • @mzerone-g6m
    @mzerone-g6m 4 місяці тому

    For me drizzle sucks and prisma sucks sorry for that i prefer kysley

    • @syntaxfm
      @syntaxfm  4 місяці тому +2

      What do you like more about Kysley? Also what makes you say Drizzle and Prisma sucks, in my experience they are both great options

    • @mzerone-g6m
      @mzerone-g6m 4 місяці тому

      @@syntaxfm i worked with prisma and drizzle before but for very easy jobs but sometimes you pull data from alot of table and do aggregation and window function and lots of stuff okay sometimes we make a view on database but when i write new feature i need a tool to provide me most of things and drizzle already lack alot of sql features and prisma have the same problems i work on my job with ecto it is lib written in elixir and man working with so much easier.
      I used alot of orm like prisma awesome until you start making joins and take data from alot of tables and make aggregation you just lost the way for prisma.
      Drizzle just low level implementation of sql if we have versions for sql i will give drizzle like 0.1
      I sometimes even work with laravel and for meduim things it is okay but after that you will write sql in your hand.
      so i think the only true orm i used until ecto and linq give you all the power it make the write mapping for objects and alot of stuff.
      Why kysley i was have a project it is scrapper and i need to feed this data to db so i tried prisma and drizzle does not fit but when i tried kysley and make inspect db i just get the completion type safety and do not need to deal with writing my geolocation types i think that a joke you have orm that does not support goelocation 😦

  • @adamcollier1907
    @adamcollier1907 4 місяці тому +1

    Great video as always CJ! I know you touched on migrations but I’d love to know how to handle migrations for prod, e.g you already have an app running and need to run a migration without breaking anything. Also worth mentioning Drizzles proxy driver, was great for my move away from Planetscale and still allowed me to use Vercel edge functions

  • @cpakken
    @cpakken 4 місяці тому +1

    What is the vscode theme you're using?

  • @haim3689
    @haim3689 Місяць тому

    hi great video i have little question,
    you made 1 postgres connection and you check if its migration/seed based on env, if it was in production and you would like to migrate on every deploy doesn't it means you had to make a whole server instance with its own env of migrate=true just for migrating ? is it a good practice ?

  • @talensjr
    @talensjr 4 місяці тому +1

    Yet another one of this amazing tutorials!! Keep this going, super helpful from a fullstack developer point of view.
    Thanks! 🙏

  • @Michael291288
    @Michael291288 4 місяці тому

    Amazing! Would love to see the same implementation done with Prisma 🤩

  • @Loui3Hunna
    @Loui3Hunna 2 місяці тому

    You are a great teacher CJ! I learned a ton :)

  • @sasquatch_devs
    @sasquatch_devs 4 місяці тому

    Would love your thoughts on Lucide ORM by the creators of AdonisJS!

    • @syntaxfm
      @syntaxfm  4 місяці тому

      I'm not sure how I feel about the lack of type safety 🤔
      The author talks about it here: github.com/thetutlage/meta/discussions/8
      I think I would rather have a single source of truth with drizzle for the table schema / inferred insert / select types, inferred validators etc. than worry about a query not having a typescript error when it should. I haven't run into the type issues during querying the author brings up in that post.
      With lucid you need to define your models separately from creating the tables. Migrating needs to be done manually for every column / change and then you need to update the model to match. This is very similar to my experience with sequelize.
      With drizzle, you update the schema type and then the migrations are auto generated from the schema change and the inferred types continue to flow through.
      Seems like a trade of better type safety during querying vs better type safety / single source of truth from model definitions. In my experience with other libraries, keeping the models in the code base up to date with the table schema is one of the biggest pain points, not querying.
      -CJ

  • @dummy_code
    @dummy_code 4 місяці тому

    Does it support raw sql queries?

    • @syntaxfm
      @syntaxfm  4 місяці тому

      Yes and it parameterizes them by default: orm.drizzle.team/docs/goodies#raw-sql-queries-execution
      If you don't want parameterization you can use sql.raw

  • @yiannis_p
    @yiannis_p 4 місяці тому

    Amazing video as usual!
    One thing I would also add here is that drizzle offers a db:push functionality so you don’t have to deal with any migrations!
    Also the foreign keys are optional, as you can still use their orm style query builder without foreign keys just using the relationships you defined in the schema! Works really well in an environment like planetscale which only has online ddl and has no foreign keys!