A deceptively simple demo of something that is bound to fundamentally boost adoption of Fabric. I'm thinking of all the pain points this eliminates, from data plumbing side to being able to reassure users that the data is available. Excellent to see the querying info on the Snowflake side as proof too, will be interesting to see what the admin side of this looks like over time within Fabric.
Hey there Reza... You've made a comment in this video about Direct Query being slow... and yes - that was my perception. Last Friday I loaded 45M rows into a table in Snowflake and connected to it with with a Published Power BI report via DQ. I've been completely below away by the outstanding performance when producing a Matrix visual across 9 months of data. IN my view - with this example the outstanding performance of Direct Query completely and utterly out weighs the benefits of refreshing the data on a regular basis. The refresh alone is 40+ minutes... and will only grow every month. Yes I could use incremental refresh - but why when the DQ performance is so outstandingly fantastic.
Hi David That is interesting indeed, I haven't seen DirectQuery performing fast unless a good database table optimization is done on the data source, maybe that is the case? some column store clustered indexing etc on the source?
@@RADACAD No tuning has been done at all. I did create a view with a select *, cast( to numeric for one column) from db.schema.table The data has been loaded from csv files from Azure BLOB storage. Very stunning, happy to do a demo for you..
They still have a few bugs to work out, i think larger snowflake databases seems to have some issues with mirroring, i've raised these through a support ticket and they are currently with the product team. I'm a little disappointed as i can really use this feature now, i hope it's fixed soon.
At the moment Mirroring is only possible for 3 sources (Snowflake, Azure SQL DB, and Azure cosmos db). for other sources, you have to develop the data integration using Data Pipelines and Dataflows yourself.
Hi Reza, thanks for the information. I’ve checked in my P1 capacity (in Australia) and Mirroring options haven’t been available yet. Moreover, will it be possible to access through snowflake SSO login?
That is strange. Because Australia supports the mirroring based on this link: learn.microsoft.com/en-us/fabric/database/mirrored-database/snowflake-limitations If by Snowflake SSO you mean users to be able to access the part of data in Snowflake that they are authorized to see (the security to flows through from Snowflake into Fabric); that is not possible at the moment. the security rules have to be re-created in Fabric. But if you mean that we use Snowflake user account to set up the mirroring as a stored credentials, it is possible, like what I showed in this video
A Game changer .Nice explanation Reza. By this approach SF can serve as a storage layer and for compute,etc customers can use Fabric..
Yes, you are right. Customers can keep their data in Snowflake and still leverage the powerful analytical features of Fabric.
A deceptively simple demo of something that is bound to fundamentally boost adoption of Fabric. I'm thinking of all the pain points this eliminates, from data plumbing side to being able to reassure users that the data is available. Excellent to see the querying info on the Snowflake side as proof too, will be interesting to see what the admin side of this looks like over time within Fabric.
Thanks Gina! Good to see you here. Indeed this feature makes a difference
Hey there Reza...
You've made a comment in this video about Direct Query being slow... and yes - that was my perception.
Last Friday I loaded 45M rows into a table in Snowflake and connected to it with with a Published Power BI report via DQ.
I've been completely below away by the outstanding performance when producing a Matrix visual across 9 months of data.
IN my view - with this example the outstanding performance of Direct Query completely and utterly out weighs the benefits of refreshing the data on a regular basis. The refresh alone is 40+ minutes... and will only grow every month. Yes I could use incremental refresh - but why when the DQ performance is so outstandingly fantastic.
Hi David
That is interesting indeed, I haven't seen DirectQuery performing fast unless a good database table optimization is done on the data source, maybe that is the case? some column store clustered indexing etc on the source?
@@RADACAD No tuning has been done at all. I did create a view with a select *, cast( to numeric for one column) from db.schema.table
The data has been loaded from csv files from Azure BLOB storage.
Very stunning, happy to do a demo for you..
@@RADACAD No special treatment or optimisation anywhere. And the warehouse within snowflake is a small one.
Thanks for the video! We use Snowflake and can't wait to start testing it. Next great thing will be Direct Lake in Power BI desktop
You're welcome. Let me know your experience with it.
They still have a few bugs to work out, i think larger snowflake databases seems to have some issues with mirroring, i've raised these through a support ticket and they are currently with the product team. I'm a little disappointed as i can really use this feature now, i hope it's fixed soon.
Hi, where is the snowflake environment for this demo. Is the snowflake environment hosted in aws or azure? does it work across cloud?
It won't matter what Snowflake environment for Mirroring
It uses the APIs and CDC for it
Thanks for this update!
You're welcome!👊👊
Hey Reza, can you please suggest if there's a way to mirror data from Athena and BigQuery, need an urgent solution for optimising the cost
At the moment Mirroring is only possible for 3 sources (Snowflake, Azure SQL DB, and Azure cosmos db). for other sources, you have to develop the data integration using Data Pipelines and Dataflows yourself.
Hi Reza, thanks for the information. I’ve checked in my P1 capacity (in Australia) and Mirroring options haven’t been available yet. Moreover, will it be possible to access through snowflake SSO login?
That is strange. Because Australia supports the mirroring based on this link: learn.microsoft.com/en-us/fabric/database/mirrored-database/snowflake-limitations
If by Snowflake SSO you mean users to be able to access the part of data in Snowflake that they are authorized to see (the security to flows through from Snowflake into Fabric); that is not possible at the moment. the security rules have to be re-created in Fabric. But if you mean that we use Snowflake user account to set up the mirroring as a stored credentials, it is possible, like what I showed in this video
Have you find any solution for the SSO, does it start supporting or still we have to recreate the roles is fabric?
I'm not sure your microphone works
What do you mean by not working? I can hear the sound in the video