Glad to hear that I've helped in a small way 😊 If there's any Cosmos DB related content you'd like to see a video on, let me know and I'll add it to my backlog!
A clear explanation... Awaiting for next video on throughput :-) One query, What would be the partition key in case of a IoT based solution with components in the order as Tenant -> Project -> Venue(Location) -> Device -> Parameters. A project could be spread across multiple areas within a location or different locations. A venue may contain large number of devices. Each device may have multiple parameters. I am looking at initial thought that might come to your mind and a reasoning over it. Let me know if you need more information.
Hi Praveen, thanks for watching, With this type of scenario, say if you have devices all around the world producing telemetry data, you could use an event sourcing model, where you have a Container that's partitioned by /id for your writes. Which will balance your write containers storage and throughput distribution nicely: docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#using-item-id-as-the-partition-key You could then implement the Change Feed feature in Cosmos DB to listen to that container and then move that data into a container optimized for read operations with a different partition key on a property that will be heavily queried on (For example, location): docs.microsoft.com/en-us/azure/cosmos-db/change-feed You could even take a look at the Analytical Store feature (in preview at time of comment) if you need to do more sophisticated analytical jobs on your data using Azure Synapse: docs.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction Hope this helps you out!
Nice detailed explanation. Just a quick query. Whether throughput be evenly distributed between logical partition if only one physical partition is there ? In that case can problem of hot partition also occur for a given logical partition ?
Yes hot partitioning occurs at the logical partition level, even if you have only one physical partition. If your logical partition's storage is unevenly distributed compared to other logical partitions on your single physical partition, this logical partition will be considered 'hot'
It depends on how often you are performing x-partition queries. If you do it occasionally, then it's ok. If you're executing that query numerous times, it's better to reevaluate that partition key and use the change feed to write to another container (or use a different database entirely).
Is it okay to use the id of a document/item as the partition key? I understand that's it's probably not ideal, but exactly how bad is it? The ID will be unique for every document/item.
Using the ID of a doc/item as partition key is great for write heavy workloads, where you may not care about querying the id later and you just want to write it. In situations where you need to support heavy writes BUT query that item later, this is where the change feed can come in. That will help move that item from a Cosmos DB container to an alternative store (or another container) that's better suited for queries.
How to find the size of the particular partition key? I am facing partition key reached maximum 20gb limit what are possible solutions for this scenario. Please response
Hi there, you can view this information using the PartitionKeyStatistics log category in Diagnostic logs. The product team have a doc on how to create alerts when PK size gets too large: learn.microsoft.com/en-us/azure/cosmos-db/how-to-alert-on-logical-partition-key-storage-size A possible solution for this is to re-evaluate your data model and see if there's a more granular partition key property that you can use. You could also use hierarchical partition keys to subpartition your data: learn.microsoft.com/en-us/azure/cosmos-db/hierarchical-partition-keys?tabs=net-v3%2Cbicep
I am so glad I've found your channel , I was having a hard time understanding cosmosdb, Thanks Will !!
Glad to hear that I've helped in a small way 😊 If there's any Cosmos DB related content you'd like to see a video on, let me know and I'll add it to my backlog!
This is a good explanation of partitions.
Thank you!
Thanks Will, this is great, understand this now better :)
Thanks for watching!
A clear explanation... Awaiting for next video on throughput :-)
One query, What would be the partition key in case of a IoT based solution with components in the order as Tenant -> Project -> Venue(Location) -> Device -> Parameters. A project could be spread across multiple areas within a location or different locations. A venue may contain large number of devices. Each device may have multiple parameters. I am looking at initial thought that might come to your mind and a reasoning over it. Let me know if you need more information.
Hi Praveen, thanks for watching,
With this type of scenario, say if you have devices all around the world producing telemetry data, you could use an event sourcing model, where you have a Container that's partitioned by /id for your writes. Which will balance your write containers storage and throughput distribution nicely: docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#using-item-id-as-the-partition-key
You could then implement the Change Feed feature in Cosmos DB to listen to that container and then move that data into a container optimized for read operations with a different partition key on a property that will be heavily queried on (For example, location): docs.microsoft.com/en-us/azure/cosmos-db/change-feed
You could even take a look at the Analytical Store feature (in preview at time of comment) if you need to do more sophisticated analytical jobs on your data using Azure Synapse: docs.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction
Hope this helps you out!
@@willvelida Awesome, will check the referenced links.
Great video. I expected that the RequestCharge of the x-partition query will be the highest, but if was 0. How so?
Nice detailed explanation. Just a quick query.
Whether throughput be evenly distributed between logical partition if only one physical partition is there ? In that case can problem of hot partition also occur for a given logical partition ?
Yes hot partitioning occurs at the logical partition level, even if you have only one physical partition. If your logical partition's storage is unevenly distributed compared to other logical partitions on your single physical partition, this logical partition will be considered 'hot'
Great video. Shall we fire a x-partition query or one query per partition or create a change feed with another container?
It depends on how often you are performing x-partition queries. If you do it occasionally, then it's ok. If you're executing that query numerous times, it's better to reevaluate that partition key and use the change feed to write to another container (or use a different database entirely).
Good explnation
Thank you!
simple and good videos
Thanks!
Is it okay to use the id of a document/item as the partition key? I understand that's it's probably not ideal, but exactly how bad is it? The ID will be unique for every document/item.
wondering the same
Using the ID of a doc/item as partition key is great for write heavy workloads, where you may not care about querying the id later and you just want to write it.
In situations where you need to support heavy writes BUT query that item later, this is where the change feed can come in. That will help move that item from a Cosmos DB container to an alternative store (or another container) that's better suited for queries.
How to find the size of the particular partition key?
I am facing partition key reached maximum 20gb limit what are possible solutions for this scenario.
Please response
Hi there, you can view this information using the PartitionKeyStatistics log category in Diagnostic logs. The product team have a doc on how to create alerts when PK size gets too large: learn.microsoft.com/en-us/azure/cosmos-db/how-to-alert-on-logical-partition-key-storage-size
A possible solution for this is to re-evaluate your data model and see if there's a more granular partition key property that you can use.
You could also use hierarchical partition keys to subpartition your data: learn.microsoft.com/en-us/azure/cosmos-db/hierarchical-partition-keys?tabs=net-v3%2Cbicep
Thank you so much@@willvelida
Where is the Modeling Cosmos first video link ?
Hey Danish, here's my video on Data Modelling in Azure Cosmos DB: ua-cam.com/video/p5rRGlKxNtk/v-deo.html