Regarding projections, this deserves its own video. All demonstrations of it that I have found are pretty trivial queries. But if your query is complex, with many joins (Include/ThenInclude) one-to-many relationships, and a lot of columns, you run into problems that are not discussed in these demonstrations. I did not quite catch your mention of AutoMapper, but if it makes this projection technique easier for complex queries, then I am definitely interested.
There seems to be a lot of discussion about IQueryable. They say that we shouldn't expose it because it reveals to malicious user that he is in fact in direct contact with database. They also say that it's "leaky", I am not sure what they mean by that. What's your take on this?
There are indeed a lot of discussion points on this topic and it's one of the debates that will never have a real end. From my point of view, like in most cases in the engineering world, it's about balancing pros and consi when taking a decision to use something or not. As a general guideline (from my point of view) is that we should stream data, not buffer it. We can use IEnumerable for that purpose. Actually in my code I'm returning mostly IEnumerables. However, there are practical situations when you don't have all the information to build the entire query in one go and you might want to add to your query some new thing in different places. In this case I'd prefer the IQueryable since it allows you to do that and execute the entire query on the server, not in memory. If you return IEnumerable in the first go, then everything you add afterwards (further filtering, ordering, grouping) will be performed client side, therefore in memory. On the other hand, I don't agree that it's "leaky". People that say this usually tend to bring the argument that it's leaky because it exposes your database implementation. I think this is wrong, because IQueryable is not database specific. That;s mostly my take on this topic. Hope it helped.
@@TommiLipponen I guess so. However, if you talk about input validation then that's for updates in the database and for that purpose there's really no discussion about Iqueryable or not. That;s for retrieving data.
Dan, I enjoyed your video! It would be even better if you could cover Transactions for Insert/Update operations across multiple Entities and discuss business logic that involves data reading. Thanks for your content!"
Thank you for your comment. Regarding AutoMapper, I have published the video today: ua-cam.com/video/6U4rQhKQq1I/v-deo.html Regarding LINQ I would suggest to go to the "C# for total noobs" playlist. There I have two videos on LINQ.
Hi, thanks for video. Give a advise please. If i have a big count of rows in database and i get them in for each cycle like foreach (var item in context.Items) it is need a split it by chunks for avoiding sql query like "select * from table"?
Thank you for this video, Dan!! I recently an interviewer asked how to handle millions of records when using EF, my experience is kind of limited to only few thousands and was not able to respond. Do you have experience handling these kinds of scenarios?
Millions of records is a little bit much. These things here are just fundamentals so that's surely the way to start. But then again, when it comes to large datasets it's also about how to design the query itself, limiting result sets by using paging, using split queries where it makes sense, using lazy loading where it makes sense and so on. Another approach is to work with pre-defined views and work with those. Another approach would be to even run raw SQL. Lastly, for queries I'd use a different ORM called Dapper. But there's surely much more to it.
Well, I guess you need to materialize a query in order to return a collection, am I wrong? In this case I'd say we should avoid this, like stated in the video. As long as we materialize the query, it means we bring all data into memory.
Thanks helpful tips, Ive never used AsEnumerable. Would be good to have seen examples in real services in addition just for clarity. I've found lately i often cast direct to a dto from the dbconext.... E.g context.People.Select(p => new PersonDto() ... And have found myself not using auto mapper as much, so interested to see a vid on that.
It's good that you cast directly to a DTO. I still think that the streaming approach should be used where possible when working with lists/arrays. We tend to materialize them a lot and that causes unnecessary allocations.
I have seen this too, specifically with DB first development, it is the easiest way to go, however when using Code First it is better to map correctly the entity models, it saves a lot of headaches when joining the tables.
Regarding projections, this deserves its own video. All demonstrations of it that I have found are pretty trivial queries. But if your query is complex, with many joins (Include/ThenInclude) one-to-many relationships, and a lot of columns, you run into problems that are not discussed in these demonstrations.
I did not quite catch your mention of AutoMapper, but if it makes this projection technique easier for complex queries, then I am definitely interested.
Thank you for your videos! Not everybody explains these concepts as clear as you do. Great work.
There seems to be a lot of discussion about IQueryable. They say that we shouldn't expose it because it reveals to malicious user that he is in fact in direct contact with database. They also say that it's "leaky", I am not sure what they mean by that. What's your take on this?
There are indeed a lot of discussion points on this topic and it's one of the debates that will never have a real end. From my point of view, like in most cases in the engineering world, it's about balancing pros and consi when taking a decision to use something or not.
As a general guideline (from my point of view) is that we should stream data, not buffer it. We can use IEnumerable for that purpose. Actually in my code I'm returning mostly IEnumerables.
However, there are practical situations when you don't have all the information to build the entire query in one go and you might want to add to your query some new thing in different places. In this case I'd prefer the IQueryable since it allows you to do that and execute the entire query on the server, not in memory. If you return IEnumerable in the first go, then everything you add afterwards (further filtering, ordering, grouping) will be performed client side, therefore in memory.
On the other hand, I don't agree that it's "leaky". People that say this usually tend to bring the argument that it's leaky because it exposes your database implementation. I think this is wrong, because IQueryable is not database specific.
That;s mostly my take on this topic. Hope it helped.
@@Codewrinkles If we implement repository pattern, DTOs and input validation, we should be quite safe, right?
@@TommiLipponen I guess so. However, if you talk about input validation then that's for updates in the database and for that purpose there's really no discussion about Iqueryable or not. That;s for retrieving data.
Dan, I enjoyed your video! It would be even better if you could cover Transactions for Insert/Update operations across multiple Entities and discuss business logic that involves data reading. Thanks for your content!"
Very useful EF Core tricks. Thanks a lot for sharing.
Glad you liked it
Nice video, I am interested in more EfCore performance videos.
Thanks for the suggestion. I'm certainly considering it.
Thank you so much. I would like to intrested to watch AutoMapper. Can you explain about indexes and Linq tutorial?
Thank you for your comment.
Regarding AutoMapper, I have published the video today: ua-cam.com/video/6U4rQhKQq1I/v-deo.html
Regarding LINQ I would suggest to go to the "C# for total noobs" playlist. There I have two videos on LINQ.
Thanks I really enjoyed everything you said.
Glad you enjoyed it!
Thanks for the tips!
Glad you find the video useful.
Thanks Dan !
my pleasure!
Which IDE this is?
Hi, thanks for video. Give a advise please. If i have a big count of rows in database and i get them in for each cycle like
foreach (var item in context.Items) it is need a split it by chunks for avoiding sql query like "select * from table"?
This sounds like a scenario where you should use paging.
Thanks for this tips
My pleasure
Thank you for this video, Dan!! I recently an interviewer asked how to handle millions of records when using EF, my experience is kind of limited to only few thousands and was not able to respond. Do you have experience handling these kinds of scenarios?
Millions of records is a little bit much. These things here are just fundamentals so that's surely the way to start. But then again, when it comes to large datasets it's also about how to design the query itself, limiting result sets by using paging, using split queries where it makes sense, using lazy loading where it makes sense and so on. Another approach is to work with pre-defined views and work with those. Another approach would be to even run raw SQL. Lastly, for queries I'd use a different ORM called Dapper. But there's surely much more to it.
i have one question what about if we return from the repo a collection ?
Well, I guess you need to materialize a query in order to return a collection, am I wrong? In this case I'd say we should avoid this, like stated in the video. As long as we materialize the query, it means we bring all data into memory.
thanks man
Any time
Thanks helpful tips, Ive never used AsEnumerable. Would be good to have seen examples in real services in addition just for clarity. I've found lately i often cast direct to a dto from the dbconext.... E.g context.People.Select(p => new PersonDto() ... And have found myself not using auto mapper as much, so interested to see a vid on that.
It's good that you cast directly to a DTO. I still think that the streaming approach should be used where possible when working with lists/arrays. We tend to materialize them a lot and that causes unnecessary allocations.
I have seen this too, specifically with DB first development, it is the easiest way to go, however when using Code First it is better to map correctly the entity models, it saves a lot of headaches when joining the tables.