This is excellent, best explanation I have seen so far that it simplified and helps break things down. I can't count the number of times I have to explain this to customers.
I`m extremely happy to finally find a channel with meaningful info on latest Azure Data stuff and little-to-no ''entertainment content' and no lootbox unboxing.
you all probably dont care but does someone know a trick to get back into an instagram account? I somehow forgot my account password. I appreciate any assistance you can offer me
Thank you for this video, it was really very informative and gives good understanding on this new Azure Synapse Analytics experience especially for people like me who are new to it and are coming from earlier SQL Data warehouse with lot of confusion between what was earlier and what is now
True! However - although you can link a workspace, create models/reports from the Synapse Studio workspace, the actual data, licences, compute etc still lives within PowerBI. In this case I see it as more of a convenient integration than a new service. Unless there are areas I've not seen yet! I definitely should have mentioned it, but it's not one of the key areas of confusion I was tackling! Thanks for pointing it out! Simon
MS should be paying you if they aren't already. Great video mate, thank you. Do you have one stepping through a mini example of how to org your lake and basically run through an ELT end to end ?
Awesome presentation. Thank you very much. One (probably dumb) question only, if the provisioned SQL pools are just computation power (and paid by the hour!), where would the so called Multidimensional Model go (the good old DWH cubes with facts and dimensions etc) in the "modern" big data approach? Because the Data Lake does not support this relational model does it? The Provisioned SQL Pools sound really expensive to be in use 24/7, doesn't it? Thanks again for the great content, count another subscriber.
Hi. Can we use synapse analytics to perform ad-hoc query on the tabular model deployed on azure analysis services without moving the data from analysis server to data lake. Basically the idea is to query data on analysis server. Is the ADOMD only way out bcz that would require too much of scripting. Waiting for you response.
Hi, is it possible to just store the data in an Azure SQL Database, instead of the SQLDWH? I don't think our datasets would benefit from the SQLDWH (MPP) part, but the available tools (that Workspaces offers) would help us.
Yep, you certainly can - you don't have the ability to read directly from a lake with SQLDB, but you can certainly do some lake prep then copy into the SQLDB using data factory. It's also possible to prep the data in the lake and just use the Serverless SQL endpoint directly over the lake - depends which database features you need! Simon
Thanks , for the on demand SQL - is that also an 'MPP' SQL Server? If not , can i still use it for scenarios in which there may be less than TB's of data for analysis?
Hey! So from what we can tell, it's neither MPP or SMP, it's somewhere in the middle. Given it's charged by how much data is read (in some increment of mb although you tend to see prices in TB), it's REALLY good and cheap for small amounts of data, especially for ad-hoc analytics queries. I'd imagine the costs would rack up pretty quick if you tried to treat it like a high-concurrency transactional engine - single record selects etc. But if you have a small-ish dataset, let's say your whole data mart is ~300Gb, and you're not hammering it constantly, SQL OD might be way cheaper than Provisioned Pools!
@@AdvancingAnalytics Thanks so much . Really helps clarify things. While i believe the stack of services for Azure is really powerful, sometimes a little clarity from MS (which you have done) would be really helpful!
How to build a tabular model for power bi in synapse, or we have to use Azure analysis services integrated with synapse. and how power bi use Synapse with data modeling?
Hey Bassem - you can't build the tabular model directly within Synapse, but you can use Synapse as a source for the data model, and build reports within the workspace. The data model you need to build using the usual tools (Power BI Desktop for most). I ran through the setup in this video: ua-cam.com/video/ldN6D2lhNyA/v-deo.html In terms of Azure Analysis Services - I've not looked into hooking it up to use Synapse as an endpoint, but you'd be find with the SQL OD / Provisioned Pool endpoints, as with any other SQL DB. Simon
@@AdvancingAnalytics if do not use Azure analysis services, where is data saved while using power bi, do we have the same performance of data in memory (tabular)? I mean do I need to add AAS in my architect for performance in dashboards or no need? considering not to build models inside power bi.
@@bassemamer4893 you have a choice - you can either store the data as a power bi tabular model, which has size limitations / cost implications for premium. Performance-wise, the tabular model is the same engine as Azure Analysis Services Alternatively, you can use a direct query against either SQL On-Demand or Provisioned Pools, depending on data volume/query frequency. If you have a lot of data & high throughput, then there's the hybrid option of aggregations & composite models in Power BI which selectively uses Direct Query only when users drill down past a certain grain. Lots of different patterns, depends on data/budget!
Great video - but one slight correction - R is not currently directly supported in synapse workspace. Apparently that visual from Microsoft was not correct. stackoverflow.com/questions/65617969/is-azure-synapse-analyitcs-supporting-r-language
Yeah, this video was quite some time ago, while still in the early public preview phases - we weren't sure whether R would be included or not by GA. Guess we know the answer now :)
when we provision SQL Pool DW then we don't select the Server on which is hosted. Unlike in SQLDW earlier where we provisioned in on a given SQL server. Is my understanding correct? Thanks
Yep! Essentially the Synapse Workspace itself is the logical SQL Server, then any databases you create underneath are associated to it (Provisioned SQL Pools, SQL On-Demand metadata, Spark metadata etc) Simon
Parts of it compete directly (the Spark engine!) other parts could sit very nicely in a solution alongside Databricks (SQL OnDemand, Provisioned SQL Pools etc). For us, the Databricks spark engine is a lot more mature & fully featured - so if you're after a deep, premium Spark experience then Databricks would suit. If you want an easy-to-use, single platform that can do some spark and some traditional SQL, then Synapse will work.
I think generally everyone was! Now that they've settled on Synapse Analytics - Spark Pools, Serverless SQL Pools & Dedicated SQL Pools it won't change again for a while... hopefully... Simon
This is better than the microsoft training session i had
This is excellent, best explanation I have seen so far that it simplified and helps break things down. I can't count the number of times I have to explain this to customers.
This is the best overview of any product I have seen.
I`m extremely happy to finally find a channel with meaningful info on latest Azure Data stuff and little-to-no ''entertainment content' and no lootbox unboxing.
If people send me things, I'll happily unbox it... but otherwise yep, all tech nerdery here :)
Your explanation directly answers the question I had in mind. Nice explanation
Excellent explanation. Thank you very much for putting this together.
Thanks for sharing this, it's really good.
This was a great way to get introduced to the lay of the land, thank you so much!
Really appreciate your overview of Synapse analytics including the costing. Thank you very much.
you all probably dont care but does someone know a trick to get back into an instagram account?
I somehow forgot my account password. I appreciate any assistance you can offer me
@Benedict Terry instablaster :)
Thank you for this video, it was really very informative and gives good understanding on this new Azure Synapse Analytics experience especially for people like me who are new to it and are coming from earlier SQL Data warehouse with lot of confusion between what was earlier and what is now
It's great how you can explain all that in such as concise fashion, thanks!
Thanks for the overview of this and for the subtitle! really appreciate!!
Thank you. You're explanations are truly spot on.
It makes so much more sense. Many thanks!
Great, great video. Exactly what I needed.
Very helpful and easy explanation. Keep doing the work. Really appreciate it
Great Video! Excellent explanation, you put everything in its place so well! Thank you.
Thanks for answering every question in my head :)
Excellent video & simple explanation
Thanks!
You forgot about Power BI integration in Synapse Studio.. It is also part of the overall Synapse umbrella.
True! However - although you can link a workspace, create models/reports from the Synapse Studio workspace, the actual data, licences, compute etc still lives within PowerBI. In this case I see it as more of a convenient integration than a new service. Unless there are areas I've not seen yet!
I definitely should have mentioned it, but it's not one of the key areas of confusion I was tackling! Thanks for pointing it out!
Simon
MS should be paying you if they aren't already. Great video mate, thank you. Do you have one stepping through a mini example of how to org your lake and basically run through an ELT end to end ?
@@mzhukovs whoops - missed this one! I can add a rough guide to lake structures to the video list!
Awesome presentation. Thank you very much. One (probably dumb) question only, if the provisioned SQL pools are just computation power (and paid by the hour!), where would the so called Multidimensional Model go (the good old DWH cubes with facts and dimensions etc) in the "modern" big data approach? Because the Data Lake does not support this relational model does it? The Provisioned SQL Pools sound really expensive to be in use 24/7, doesn't it? Thanks again for the great content, count another subscriber.
Really helpful...Keep sharing your knowledge..
Great tutorial and information... Just wish audio was better
Hah - was one of our early videos. Most vids have way better audio now :)
Fantastic, you're a great teacher!
Very well explained, subscribed!
Thanks for your explanation, it will be better if you can make some demos
I now understand. Thank you
Hi.
Can we use synapse analytics to perform ad-hoc query on the tabular model deployed on azure analysis services without moving the data from analysis server to data lake. Basically the idea is to query data on analysis server. Is the ADOMD only way out bcz that would require too much of scripting.
Waiting for you response.
You are man!!! Thanks for the video
Great tutorials was wondering if you have the code examples /notes for the Azure Synapse playlist on Github or somewhere else? Thanks
Hi excellent, can you do another video on this since it has been two years
I'll be doing a revamped "Intro to" set later this year! I think enough has changed that it's warranted :)
Well explained mate
Thank you
Hi, is it possible to just store the data in an Azure SQL Database, instead of the SQLDWH? I don't think our datasets would benefit from the SQLDWH (MPP) part, but the available tools (that Workspaces offers) would help us.
Yep, you certainly can - you don't have the ability to read directly from a lake with SQLDB, but you can certainly do some lake prep then copy into the SQLDB using data factory. It's also possible to prep the data in the lake and just use the Serverless SQL endpoint directly over the lake - depends which database features you need!
Simon
loved your video, tnx :)
You forgot to go back to Datallegro! :)
Thanks
Thanks , for the on demand SQL - is that also an 'MPP' SQL Server? If not , can i still use it for scenarios in which there may be less than TB's of data for analysis?
Hey! So from what we can tell, it's neither MPP or SMP, it's somewhere in the middle. Given it's charged by how much data is read (in some increment of mb although you tend to see prices in TB), it's REALLY good and cheap for small amounts of data, especially for ad-hoc analytics queries. I'd imagine the costs would rack up pretty quick if you tried to treat it like a high-concurrency transactional engine - single record selects etc.
But if you have a small-ish dataset, let's say your whole data mart is ~300Gb, and you're not hammering it constantly, SQL OD might be way cheaper than Provisioned Pools!
@@AdvancingAnalytics Thanks so much . Really helps clarify things. While i believe the stack of services for Azure is really powerful, sometimes a little clarity from MS (which you have done) would be really helpful!
How to build a tabular model for power bi in synapse, or we have to use Azure analysis services integrated with synapse. and how power bi use Synapse with data modeling?
Hey Bassem - you can't build the tabular model directly within Synapse, but you can use Synapse as a source for the data model, and build reports within the workspace. The data model you need to build using the usual tools (Power BI Desktop for most).
I ran through the setup in this video: ua-cam.com/video/ldN6D2lhNyA/v-deo.html
In terms of Azure Analysis Services - I've not looked into hooking it up to use Synapse as an endpoint, but you'd be find with the SQL OD / Provisioned Pool endpoints, as with any other SQL DB.
Simon
@@AdvancingAnalytics if do not use Azure analysis services, where is data saved while using power bi, do we have the same performance of data in memory (tabular)? I mean do I need to add AAS in my architect for performance in dashboards or no need? considering not to build models inside power bi.
@@bassemamer4893 you have a choice - you can either store the data as a power bi tabular model, which has size limitations / cost implications for premium. Performance-wise, the tabular model is the same engine as Azure Analysis Services
Alternatively, you can use a direct query against either SQL On-Demand or Provisioned Pools, depending on data volume/query frequency.
If you have a lot of data & high throughput, then there's the hybrid option of aggregations & composite models in Power BI which selectively uses Direct Query only when users drill down past a certain grain. Lots of different patterns, depends on data/budget!
Is the Recursive cte supported in Synapse Analytics DW?
Nope, not supported. Unless things have changed with the workspace preview version, but I've seen nothing to suggest that!
Simon
Great video - but one slight correction - R is not currently directly supported in synapse workspace. Apparently that visual from Microsoft was not correct. stackoverflow.com/questions/65617969/is-azure-synapse-analyitcs-supporting-r-language
Yeah, this video was quite some time ago, while still in the early public preview phases - we weren't sure whether R would be included or not by GA.
Guess we know the answer now :)
when we provision SQL Pool DW then we don't select the Server on which is hosted. Unlike in SQLDW earlier where we provisioned in on a given SQL server. Is my understanding correct? Thanks
Yep! Essentially the Synapse Workspace itself is the logical SQL Server, then any databases you create underneath are associated to it (Provisioned SQL Pools, SQL On-Demand metadata, Spark metadata etc)
Simon
Also Tables can be created in the SQLDW not in the SQL on Demand. ( external tables can be created in both however )
Is this a competitor to databricks platfrom?
Parts of it compete directly (the Spark engine!) other parts could sit very nicely in a solution alongside Databricks (SQL OnDemand, Provisioned SQL Pools etc).
For us, the Databricks spark engine is a lot more mature & fully featured - so if you're after a deep, premium Spark experience then Databricks would suit. If you want an easy-to-use, single platform that can do some spark and some traditional SQL, then Synapse will work.
God thank you. I was so frustrated of their renaming.
I think generally everyone was! Now that they've settled on Synapse Analytics - Spark Pools, Serverless SQL Pools & Dedicated SQL Pools it won't change again for a while... hopefully...
Simon
@@AdvancingAnalytics I hope you're right. Btw amazing work, keep it up.
Hi, can you work on your room acoustics? while your content is great I find your voice to be resounding a bit, just my thought but great content.
great video. ordinary audio
It's a good series of videos but I can't continue. Your accent didn't work for me.
over dramatic ...