Should you use the Repository Pattern? With CQRS, Yes and No!
Вставка
- Опубліковано 12 чер 2024
- The repository pattern is polarizing. Some developers swear you should always use it to abstract data access logic while others think it's unnecessary if you're using an ORM. So should you use it? My answer is Yes and No! If you're applying CQRS and Vertical Slice Architecture you'll likely want a repository to build up Aggregates. However, for a Query, you may want to just get the data you need rather than an entire aggregate (or collection of aggregates) to build a view model.
🔗 EventStoreDB
eventsto.re/codeopinion
🔔 Subscribe: / @codeopinion
💥 Join this channel to get access to source code & demos!
/ @codeopinion
🔥 Don't have the JOIN button? Support me on Patreon!
/ codeopinion
📝 Blog: codeopinion.com
👋 Twitter: / codeopinion
✨ LinkedIn: / dcomartin
0:00 Intro
0:56 CQRS & Vertical Slices
4:23 Query
7:16 Command
#softwarearchitecture #domaindrivendesign #cqrs - Наука та технологія
5:43 You actually don't need AsNoTracking if you are doing a Select mapping to different models. Tracking is only for the entities added as DbSets.
Also you don't need Include or ThenInclude when doing Select as it will automatically Include the navigations you need for your projection.
One little note, Include and AsNoTracking are not necessary when using Select, EF only tracks entity types, and the Joins are inferred from the projection :)
You're correct. I had another video and someone pointed out to use AsNoTracking and for some reason while I was editing I added that without really thinking too much about that I was doing a projection and it wouldn't be tracking. Thanks for the comment.
Is there somewhere officila documentation on this?
Just thought about it. That is a great point. A DbSet in EF is a Repository. If you want to add behavior for fetching an Aggregate you can just create an extension method - which is an established pattern. So I don’t see a need for implementing another abstraction. If you use CQRS then you also ger boundaries from that. The point is not to introduce complexity that is hard to and not worth to maintain
DbSet is a Repository. The DbContext is a unit of work.
I althoug theoretically, I agree with the conclusion that "it depends", I would argue that the levels of abstraction got mixed after removing the repository. A Repository is a data access pattern while the DB context is an ORM gateway (in this case EF) instance. Also the unit tests were affected similarly: replacing a Repository is simply replacing a class instance while with the presented solution you must finish replacing a DBMS. Furthermore, I would argue that consistency helps to reduce bugs and misunderstandings.
Good stuff, keep it up buddy. Will be seeing you on memberships channels
Great stuff! Thank you for the time you put into these videos.
Glad you enjoy it!
It always surprises me how you can put so much information and explain it so concisely. Keep up the good work!
Regarding the topic: I’m usually afraid “somebody” might tamper with the DB (manually), and therefore am also inclined to use my aggregates validations before I pass them to the UI in some form or another. But it’s like you say, consistent state is important when you’re mutating this state, so that would be AFTER a user sent some instruction, based on just data that was sent to their screen.
Thanks! Appreciate the feedback and you watching the videos. Hopefully they help!
In java nobody would even think about "removing the repository" for some use cases because it's a generated layer using "Spring Data". You can have custom simplified queries for some use case though. What is true is these custom queries of the "repository" may not fit anymore the strict "repository" definition, it's not more collection of objects, it may be collection of aggregated data using native queries.
Spot on. 100% agree, my understanding is that repository is a patern from DDD and it should be used together with as small as possible (to protect the business rules) agregates loaded eagerly. Every other similar abstraction is just a data access. The problem is that everyone seems to call those data access objects a "Repository". Thanks for the great content.
It doesn't help that often folks refer to "Entities" that are just purely data models without any behavior.
"The problem is that everyone seems to call those data access objects a "Repository"". That's why i'm confused now, since everyone is giving this as an example. So how would you call and integrate those data objects and be independent of the underlying data storage/database?
@@ramonennik2536 There is many approaches you can take depending on your preferences and the nature of the project. Myself for example, if the project is more than just a CRUD I prefer DDD approach with repositories and EF by default. Repositories with Unit of Work use only on the "write" side of the system which works ONLY with agregates. In most of the cases my repositories contains just 2 methods Save (when create new) and Load (gets aggregate by ID including the whole graph of objects) Agregates should be as small as possible to protect the business rules that have to be atomic, so your ORM should never drag 20% of the database even with lazy loading which I don't need to use with this approach. For the "read" side I usually use an Interface "IDatabase" or "IDataAccess" on top of DBContext and exposing db sets. I craft my queries inside the query handlers and project for the specific view or use case, trying not to pull more data that is needed. Some people use just raw SQL for reads. This way you can shape your agregates to handle complex business logic and have seperated read models that looks completely different and display the data the way user wants. This is very important because having one model doing both quite often makes your models wrong because it forces you to think about the UI, how to present it to the user so agregates ending up fat and problematic. All I said in here does not refer to the systems where simple CRUD is enough and DDD is just an overkill.
You have introduced tight coupling with the DB implementation instead of adding an extra method in the repository pattern that would do exactly the same thing. Thus keeping all the data mapping, retrieval in an abstraction. Effectively the Repository pattern… Imagine having to migrate from Postgres to MySql like Uber did… Good luck
Imagine having to move from any database to any database but since your aware of coupling and defined focused boundaries, it wasn't so much of an issue. Check out ua-cam.com/video/_IXEfi1B-lA/v-deo.html
Great video as usuall. Thanks Derek!!
Glad you enjoyed it!
Great video! Though I've found that using entity framework directly in ANY business logic, even in queries reduces maintainability. I can't tell you the number of times we've had to refactor queries that were duplicated. If we had used a repo, we just modify the logic of the repo method, but because we used EF directly everywhere, I had to change the logic itself every place we made that query which made it way more work and was much more prone to create new bugs.
In this video, you addressed that by using an extension method, but at that point, you're basically just making a repository method but without the added benefit of dependency inversion or mock-ability for testing. You can't change the implementation of an extension method (at least not easily) so why not just use a repo that does the same thing while hiding the technical dependence on entity framework? I'll admit I've been so burned by shortcomings of Entity Framework that, by this point, I basically take a dogmatic approach to hiding it everywhere... if I use it at all.
I did a video about vertical slices where I touch on this about using EF directly. To your point, having a similar query in a lot of places is the issue. ua-cam.com/video/cVVMbuKmNes/v-deo.html
IQueryable is mockable, support is just not that extensive outside of EF. I get the EF hesitation, but if you're using it the extension method approach works great for query reuse.
I had the same experience. Not just in C#, but in other big projects including Ruby on Rails based and Java/Spring Boot based. Writing queries directly coupling ORM layer into business logic, always bites us. Honestly, in Java world, this idea of repositories is the norm, not a polarizing topic at all. It's good that the C# community challenges these norms and try other things, but giving advice based on toy implementations to people writing large codebases, might not be the most effective thing to do.
Great view on repository pattern. With event sourcing the write side is pretty simple for storage interface requirements. Therefore I don't use EF there at all and use a custom repository instead. For read side EF would work better, but I tend to avoid it in my projects due to high technology coupling. Opinions...
I don't have much issue with EF when used "simply". It's a loaded gun that you can pretty easily cause a whole load of issues with. Simple mappings, simple queries. A lot of that also comes down to how your modelling data and boundaries.
Thank you, Derek, for your thoughts and examples, it was always helpful to me!)
After watching this video I have a question of how to map an entity to an EF model in your repository? How to do this if most of entity fields may have a private accessibility level?
I can't say I'm a fan of how most examples try and use EF as a data and domain model. Check out this video that I kind of alluded to it. ua-cam.com/video/GtWVGJp061A/v-deo.html
Do you think unit of work is an anti pattern? How do you usually do transactions with different repositories queries? Thanks
But I think application layer should be database independent. İf We inject directly dbContext. That makes layer coupled with ef core. Database changing should not effect application layer.
If you create your own abstraction over EF, then you're coupled to that. Sure it's your own type rather than EF. If that abstraction has a high degree of afferent coupling to it, then yes, I can understand wanting to own the abstraction. However, if you're using CQRS and a Vertical Slice Architecture, it shouldn't have a high degree of coupling. As an example, if you were using the dbContext directly in 5 query handlers, how much work is required if your database changes? It's all about coupling and limiting it by creating as small of boundaries as possible.
In case the data comes from another application, like from another REST API.
Can the implementation of the Repository, be the consumer client of the REST API?
Or Repository Pattern only applies to something like Databases?
This is indeed a very sensitive topic for a lot of people.. :)
People sure love or love to hate repositories.
Interesting! Never thought of this. Repositories are actually needed in writes because of aggregates and state consistency but not in reads. Aggregates are useless in reads so using extension methods on the db context is a good idea for testability. Interesting!
Exactly!
did you mean use extension of custom dbcontext class for testing?
Hey great video. In my opinion, you wouldn't lose anything if you remove the repository on the write side also, and use EF directly in this case. What do you think about that?
Well in a real world situation, you'd likely be making the exact same EF LINQ call in every command. Because of this, it makes sense to abstract that into the repository so you can simply get out the Shopping Cart the same way every time.
I always had the question of "how do you paginate using a Repository?" and finally found the answer. It seems it's not even the concern of a Repository!!
I'm reading Fowler's EAA book, but haven't arrived at this section yet. Thanks for clearing this up, appreciated! Greetings from a PHP dev.
Thanks for watching! There are several situations where the solution/answer to a problem might be to simply not have the problem in the first place.
@@CodeOpinion Indeed! The best solution is to not have the problem in the first place as you said 😀
You express my thoughts really well in this video! I've sent this to my colleagues at work. Background: In recent years I haven't really seen a good application of repository pattern in the dev industry in Sweden. When I reflect on it, I think the repositories are ill-designed (at least at most places I've been working). So the repositories end up as a collection of query methods, most methods only used by one path. IMO the repositories have low cohesion.
This has also led to me being a bit anti against repository pattern, so you definitely gave me some food for thought: about domain models and consistency boundaries.
Follow up question. A lot of these onion architecture (clean) uses an infrastructure library which has the persistence logic.
Some examples have a leaky abstraction over DbContext and some use repositories for everything. If you were to use DbContext directly with queries it has to be accessible from your application logic, which for feature slices makes sense. How would you do if you followed a clean architecture style?
You end up having data access in infrastructure that's either generic and usually over fetching data. Or you have very specialized data access that lives in infrastructure but is only used by a very limited number of use cases.
I personally use repositories for queries. The reason is because I do not want query or command to be aware of database context. Database context is present only in infrastructure part of project.
Forgot to mention, we have no aggregates, we have classic use-cases with request DTOs and queries for GET requests.
Hey Derek, I don't use C# or Entity Framework etc. so the whole "DB Context" doesn't mean much to me in terms of abstraction. For languages that aren't C#, if you were creating an abstraction on the query side akin to the "Repository" on the command side, what might you call it? Just a DAO or something? I know you talk about not abstracting the database sometimes, but presuming you were - typically I have an interface like `OrderRepository` and implement that via `PostgresOrderRepository` and that abstraction allows for some unit-testing niceties, but presuming I was writing something purely for reads on the query side of CQRS like you talk about, what pattern might you have with naming the interfaces? `OrderDAO`? `OrderQueries`?
I just call them Queries generally. I often don't abstract anything on the query side because I'm limiting what I'm actually querying and where. Meaning, there may only be a dozen places where I'm fetching certain pieces of data. So if I ever needed to switch DBs or for whatever reason I need to change the queries, I'm only doing it in those 12 places.
@@CodeOpinion Not even like an `IQueries` interface for test stubbing or something?
7:26 isn't this then an integration test? Wouldn't it make more sense to test a call to an appropriate endpoint (e.g. GET /api/basket/{id}) and test the entire stack rather than just testing a single query?
6:08, I think it works without Include and ThenInclude too.
Amazing video! I was wondering, what if you need to start a transaction in a Command because you need to publish an integration event, would you expose a BeginTransaction function in your repository and wrap the transaction object with your own?
I think what your asking is about needing the outbox pattern: ua-cam.com/video/u8fOnxAxKHk/v-deo.html
@@CodeOpinion Yes I'm talking about the outbox pattern but in that video you are using the dbcontext directly in a controller to start a transaction.
I was asking if you would expose in your repository a function to start the transaction and wrap the original transaction object with your own. (Usually transactions are needed in commands so you use repositories there.)
@CodeOpinion Thanks for your video,
Should we avoid using repository pattern?
I've read your blog post that we should avoid using repository pattern with ORM
I wouldn't avoid it. As the video mentions, I don't view them as an abstraction for "data access". I view them as an abstraction around retrieving and saving Aggregates.
Won't be more suitable on the query side to get rid of the orm entierly and instead use something lighter as (dapper or ado net),with simple DTO objects as read model.As far as i understood you used the EF on read side because of testing (in memory database).Thank you!
To me it's about using whatever is best suited for the situation. If that means using something else, as your example on the query side, by all means. That's the benefit of the separation. You get to decide on each side (read & write) how you want to that data access.
how do you think about "a Command for creating entity can return an entity id", so that in the frontend, we don't need to query again for that created entity's id to make the UI up-to-date (I did this from time to time for creating entity from a popup & render it immediately to the grid without re-querying the generated database id). Does it violate "Command returns nothing" rule? Thanks.
Make the client provide the ID, then they don't need a response. Or have the client provide an identifier that can then be queried by it. Beyond that, if you must return an ID, so be it. I don't view it as a rule but it has benefits if you can follow it.
Great video! One question, though: In eShopOnWeb (Steve Smith's approach), Queries and Handlers are placed in the Web/Presentation layer, and the Queries return ViewModels. I've also seen Queries and Handlers placed in the Application/Core layer -- for example, Jason Taylor's CleanArchitecture repo -- where ViewModels are also returned but from the Application layer. I'm confused as to which layer the Queries and Handlers belong in. IMHO, ViewModels/ApiModels should be in the Web/Presentation layer. Would it be incorrect to place the Queries and Handlers in the Application/Core and have the Queries return Entities? Then in the Web/Presentation layer, a controller could use AutoMapper to map the Entity to a ViewModel contained in the UI layer? The reason I think ViewModels shouldn't be placed in the Application/Core layer is because the Application/Core should be able to work with, say, a Razor Pages client and/or a Web API client without knowing too much about the presentation details. Am I wrong? Thus far, I haven't been able to find any examples of the structure I'm proposing.
Or depends on definition and intent. I have no problem with query/commands and handlers living in the application layer. I view them as application concepts not infrastructure concepts (web). An http request is translated into an application request (query/command). If you want to return a view model out directly to the web, so be it, but realize it's a contract that now has versioning constraints.
@@CodeOpinion Thanks for answering my question. Just to clarify, you're saying the queries and commands can go in the Application layer but it would be preferable to keep the ViewModels in the Web layer? If so, that approach makes sense to me. On the other hand, I see people returning ViewModels from the Application layer and that seems counterintuitive, like coupling the Application layer to the UI layer. Or am I just misunderstanding the purpose of the Application layer?
If you return view model from application layer and pass that thru to web and serialized out, that's fine but as mentioned your end clients are coupled to that schema/structure. It means if you want to change it, you'd have to version it unless you control the clients.
@@CodeOpinion That makes perfect sense. Thanks!
entityframework or dapper? which way you can advice sir derek? Thank you.
It really depends. If you want a full blown ORM then EF. If you want simple mapping between results into objects as a layer over top of ADO.NET, Dapper it is.
IMO Repository pattern should be part of CQRS. With Repository pattern, u get to abstract the persistence layer, that you can for eg easily swap an ORM with a NOSQL. In the example shown here, EF is strongly coupled into CQRS.
Being strongly coupled to something isn't evil in itself. It's the degree to which you're coupled.
Is there a way to get the code of the video?
Do you need to use AsNoTracking() when you project the result to another class, in this case OrderSummaryView?
(at 5:42)
No
Any specific reason for not using Repositories instead of accessing db directly? The reasons I'm more inclined towards using repository as it makes it single point of source to have db operations instead of accessing db contexts all over.
I understand its not required to access the aggregate root as there is no kind of validation in query commands but didn't get why not to use Repository directly to query. Can you please explain?
Because it's over fetching data that likely isn't needed. Because the DbContext is abstracted, you're limited to how you select data.
Re: " using repository as it makes it single point of source to have db operations instead of accessing db contexts all over."
This is a concern only if you were using the dbcontext "all over". Don't. Define boundaries will limit the coupling.
@@CodeOpinion Thanks for your comment.
I'm still missing this part "Because it's over fetching data that likely isn't needed" => I understand this applies if I access through aggregate root, but here I'm talking about accessing through repository. Anything I'm missing? I'm thinking these hold good in general
1. An Aggregate Root will be seeded with data from a repository
2. A repository is a wrapper for a database or persistence
3. A repository is a wrapper for CRUD actions for persistence
@@ayyappaj8111 I think because he tries to follow the repository layer which will return domain objects, but since your queries might need dynamic formats (no over fetching etc.) he decided not to use repository layer, but he uses extension method. In my opinion you can still use repository layer as single source of managing the data access logic although it violates the original intention of repository layer, you always make little twists around architecture and principle to what you want to achieve and the use cases of your system.
And tbh, if you don't even need the CQRS if over-fetching and performance aren't the important requirements, and you can just follow some guidelines of clean architecture, DDD whatever just depends on your needs.
So if the repository should be used for aggregates but the DbContext is still exposed for the developers to use for queries.
How do you make sure the developers actually use the repository to load the aggregates and not the DbContext?
Does this also mean your repository exposes a Save method or do you need to inject both the repository and the dbcontext to apply your changes? If so, why not use an extension method on DbContext to load your aggregates that way?
You can't force developers not to access something they shouldn't per se. You can't stop them from adding ADO.NET and making an underlying SQL call directly to the database. Same type of situation, the team needs to understand the concepts you're implementing. The repository would implement a Save method. You could create some extension method on the DbContext to load the aggregate just like the Repository is doing. I often times don't use the same DbContext for both Commands and Queries, but rather create different DBContext for each.
Hi, thank you for your video.
I have one question:
I have a three layer architecture with presentation layer, business logic layer and data access layer, i have put my repository in dal with entities. For sending data from dal to UI I created a DTO to send from dal to service class in business layer, and in UI i have created models for UI and DTOs to send to business layer that will send to dal for operations in database.
Is a good thing to separate models of each layer and create DTOs for specific layer data?
I hope I was clear.
Thanks in advance.
You don't want to expose/leak your data models but rather compose them into something meaningful for the UI. At the same time, I find little value in having each layer pass around it's own DTO and constantly doing translations between layers. Keep it simple.
@@CodeOpinion Thank you for response. Have you any suggestions?
Are there never consistency boundaries that has to be enforced with regards to querying data too?
The purpose is to enforce invariants. To do that you require all the data in an aggregate (or whatever data is required to enforce them). This is only required when calling a command. Check out this video for reference: ua-cam.com/video/64ngP-aUYPc/v-deo.html
@@CodeOpinion Thank you very much for the answer. I have watched that video more than one time, takes some time for the concept to sink in :)
If all layer is separated and uses DTO objects , to connect dbContext in a layer which is close to client side sounds not a good approach. "Stay far, as much as you can to dbContext " that what my teacher said me. I am confused now ... :(
An EF Core DBContext implements a Unit of Work pattern. A DbSet implements the repository pattern. Again, if you want to own the abstraction and not depend directly on EF Core in your application code, that's your choice to do so, but it will come at the cost of adding a layer of indirection.
Thanks for the great video and explanation
I have once concern about this
If i i used dbcontext in the application layer and later we decided to move from ef to something like cosmos db
Now we have to change the code in application layer also who is refernceing the dbcontext
So i think that is why we are creating repository here
May be am wrong
It's about limiting the coupling within a boundary and/or vertical slices. If you you only reference the DBContext directly in say a dozen places, and you switch to cosmosdb, how difficult is to change 12 usages?
@@CodeOpinion you are right it is easy to change but i just wanted to mention it
Cool
Extension methods? I wish we had them in php
They are a nice way of extending functionality without touching the primary API.
Php has them!! They’re called traits in laravel
@@evilroxxx They are not exactly the same. Extension methods allow you to add functionality to a standard library class, say the String class for example (which we don't have in php as it is a primitive type). To do the same with a trait you need to write a class in your own namespace, inherit from the one you want to extend, add the trait and then replace the OG class with the new one in all your code. Let's not talk about the classes that return the libraries you're using or we'll spend the whole week writing proxy classes for them.
I like traits but they are not as powerful as extension methods.
Coming from a system software background, I'm doing some research in DDD and your channel is a great source of information.
I can't help but to keep being skeptical about aggregates as a concept. It seems like they are mostly there to help maintaining inter-entity business rules ("consistency"). But that's a whole additional layer with a lot of downsides (like having to over-fetch a lot of data) etc. for something that can't even be fully solved anyway. I mean - it's not uncommon to have rules and consistency requirements between multiple aggregates anyway and one can only expand aggregates to a point. And then the data model itself possibly enforce 80% of consistency right away anyway, and the rest would have to depend on testing and well... just being mindful about it.
Aggregates seem like just an extension of OOP's mindset of focus on "modeling" the real world (inheritance hierarchies, taxonomies, etc.), instead of focusing on the core issue of manipulating data to achieve the desired result in a maintainable package.
The make it more concrete: when subtracting a count of stock on an SKU, with aggregates (well, one entityt in this simple case) one would have to fetch the whole SKU entitiy, decrement the count, and if there was no error, write it all back. While without it, an interface on a SKURepo can have a `decreseStock` method, that executes a query that checks the count and decrement it. The performance difference is going to be huge (and gets even larger with large aggregates). Sure, with "models" (aggregates) it's possible to have one central place that enforces "stock >= 0", but is it really so hard to enforce in the couple of methods that touch it in the SKURepo? As a system software such amount of performance waste seems hard for me to accept.
At very least, you mention "fetching only the data you need" on the query path here, which is exactly addressing my concern, and that on the most common side of the system (reads). On the write side, I have a much easier time to accept the performance overhead for consistency benefits.
Thanks for the comment. On the write/command side, using an aggregate as a consistency boundary is key. If you design them so that you only have the data needed that relates to each other to enforce invariants, then you will never be over fetching data for writes. Meaning if you were subtracting stock count, your aggregate will only contain anything that needs to enforce that. Aggregates don't need be everything, just the data that's needed for the behaviors. In other words, you may have multiple aggregates for the same concept of a "product" with a shared SKU, but they each have different behaviors and concerns.
@@CodeOpinion Oh. Thank you. That makes sense, and I did not realize it.
When you're selecting into another model AsNoTracking is not needed
Correct. It's not tracking a projection. Code changed a few times recording and added that note incorrectly.
Thanks for sharing your knowledge. That's awesome!
But I have to share notes related to Enumerable.All method.
In your Basket class on line #20, you are using the 'All' method. And it looks weird. It is weird for a couple of reasons.
1. Even you expected 10 or 20 items in a basket, not thousands, the 'All' method will iterate through all items. Using the 'Any' method might have a better performance just because it will stop iterating as soon as it finds the first element.
2. Then on row #25, you are using the 'First' method, which is doing iteration again and in the worst case, it will iterate all items.
I think it is much better and cleaner to use the 'FirstOrDefault' method and check if the result is 'Null' then add a new item else add quantity for the item.
I have seen using the 'All' method only once in a real project and that was a bad idea.
Thanks for the comment and feedback. Good info. Originaly I was using Any and for some reason at some point changed it to All. Regard the FirstOrDefault(), "Better and Cleaner" is subjective since I absolutely hate null.
@@CodeOpinion Interesting. Do you have video how do you avoid NULLs?
All the things you showed only screamed "Use Repositories if you don't want a stinky codebase or a stinky test suite". Repositories also help keep all queries going to a model in one place. This comes in super-handy when you need to re-visit DB indexes, schema changes for query performances, all of that (A reason I avoid Generic Repository for read accesses, Updates and Add are object changes so they are generic). Plus, when using cloud native databases, if I write test suites setting up data every time and tearing it down, it will just take hours in my CI.
Congrats nw you have a Depedency to the Whole DBCOntext all its DBsets and all the EntityFRamework and SQL SERVER .Stick to the principles man. Specification + Repository was just the perfect abstraction . Dont mess with good code please
Principles aren't rules. I'd advocate being pragmatic in any given context. Indirection is useful but shouldn't be blindly added because of principles. If you understand coupling within your context, you'll understand the tradeoffs.