DataVerse Academy
DataVerse Academy
  • 51
  • 118 385
Microsoft Fabric Spark Notebook - Learn PySpark and SparkSQL in 2hr(Beginners Course) #microsoft
Microsoft Fabric End to End Solution - PySpark and SparkSQL Full Course For Beginners
Note:- Download the whole folder
Dataset and Code Link- 1drv.ms/f/s!Ak57yG1_Qg0iey_2DKTSnObu6Ts?e=QyMSNr
----- Video Outline -----
0:01 Introduction
0:15 Loading File From You Local System to Lakehouse
1:40 Reading CSV File Using PySpark in Microsoft Fabric Notebook
4:02 Select and SelectExpr Function in PySpark
7:30 WithCoumn, WithColumns and WithColumnRenamed Function in PySpark
13:02 Where and Filter Function in PySpark
15:38 GroupBy and Aggregation Function in PySpark
19:45 Joining Two Dataframes Using PySpark
25:34 Union and UnionByName in PySpark
32:06 Drop, DropDuplicates and Distinct Function in PySpark
40:17 Write/Load data to a delta Table and Parquet File Using PySpark
45:16 Overwrite, Append , Ignore and Error Write Mode in PySpark
53:40 Create Delta Table and Load Data through Dataframe in PySpark
01:00:34 Create Delta Table with Partition and Load Data into it Using Dataframe in PySpark
01:10:57 Difference Between Managed and External Table
01:19:34 Create delta table using SparkSQL
01:23:15 Create delta table with Partition using SparkSQL and Insert Data from a View
01:28:48 DML Operations over a delta table using SparkSQL
01:34:17 Temporary and Global Reporary View in PySpark
01:38:32 Data Transformation Over a delta table using SparkSQL
01:50:01 Reading Delta table using PySpark and SparkSQL
Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html
Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html
If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos-
1. Create Microsoft Fabric Data warehouse: ua-cam.com/video/eFSrun_IhtA/v-deo.html
2. load data into Microsoft Fabric data warehouse using Data Pipeline: ua-cam.com/video/Lk_Tt6qrDH0/v-deo.html
3. Create table in Microsoft Fabric data warehouse: ua-cam.com/video/WkVbKcZjgHU/v-deo.html
4. load data into Microsoft Fabric data warehouse using Copy into Command: ua-cam.com/video/WkVbKcZjgHU/v-deo.html
5. Transform and load data into Microsoft Fabric data warehouse using T-SQL Procedure - ua-cam.com/video/Ew86sP4PZWw/v-deo.html
Переглядів: 4 707

Відео

Microsoft Fabric End to End Ecommerce Project - Building Medallion Architecture in Lakehouse #ai
Переглядів 22 тис.8 місяців тому
Microsoft Fabric End to End Solution - Building Medallion Architecture in Lakehouse #microsoft #microsoftfabric #azure #lakehouse #dataengineering #dataengineering #modelling #facts #dimesions #technology #ai #chatgpt #india Note: Download the whole folder Dataset and Code Link- 1drv.ms/f/s!Ak57yG1_Qg0iey_2DKTSnObu6Ts?e=QyMSNr Project chapters 0:01 Project Overview - Microsoft fabric lakehouse ...
Microsoft Fabric Lakehouse: What is semantic models and how to connect to it through PowerBI
Переглядів 10 тис.8 місяців тому
Semantic Model in Microsoft Fabric Lakehouse What is SQL end point in lakehouse What is default semantic model in lakehouse Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1...
Microsoft Fabric Notebook: Difference Between External and Managed Table #Microsoftfabrc #azure
Переглядів 1,6 тис.8 місяців тому
Microsoft Fabric PowerBI | Lakehouse: How to create External Table, Managed Table and Parquet Table Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1. Create Microsoft Fabri...
Microsoft Fabric Notebook: What is the difference between Save and SaveAsTable #Microsoft #azure
Переглядів 9008 місяців тому
Microsoft Fabric Lakehouse: How to create External Table, Managed Table and Parquet Table Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1. Create Microsoft Fabric Data war...
Microsoft Fabric Lakehouse: How to Connect to ADLS Gen2 Using Shortcut in Lakehouse
Переглядів 2 тис.8 місяців тому
Microsoft Fabric Shortcut: How to Connect to ADLS Gen2 Using Shortcut in Lakehouse Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1. Create Microsoft Fabric Data warehouse:...
2 Microsoft Fabric DataflowGen2: Incremental load from Azure SQL DB to Warehouse using DataflowGen2
Переглядів 1,8 тис.10 місяців тому
Microsoft Fabric Tutorial Series- How to do Incremental Load Using from Azure SQL database to Warehouse using Dataflow Gen2 Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1...
11 Microsoft Fabric Data Pipeline: How to do Incremental Load Using Copy Activity in data pipeline
Переглядів 6 тис.10 місяців тому
Microsoft Fabric Tutorial Series- How to do Incremental Load Using Copy Activity in Microsoft Fabric data pipeline. Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos- 1. Create...
10 Microsoft Fabric Data Pipeline: Load multiple tables from azure SQL database to Warehouse
Переглядів 2,6 тис.10 місяців тому
Microsoft Fabric Tutorial Series- How to load multiple tables from azure SQL database to Warehouse using single copy activity. Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Please watch below videos...
03 Data Modelling: OLTP vs OLAP (Data Warehouse Architecture) System in detail #datawarehouse
Переглядів 320Рік тому
Difference between OLTP vs OLAP (Data Warehouse) System | Online Transaction Processing | Online Analytical Processing | Data Warehouse Architecture | OLAP Cube Data Modelling Tutorial: ER Diagram in DBMS | Entity Relationship Diagram | ER | How to draw an ER Diagram in DBMS | How to make an ER Diagram Description: Welcome to the "Mastering Data Modeling" playlist! In this series, we dive deep ...
02 Data Modelling: What is Conceptual, Logical and Physical Data Model in Detail?
Переглядів 1,3 тис.Рік тому
Data Modelling Tutorial: What is Conceptual, Logical and Physical Data Model in Detail? | Data Mart |Data Modelling |Data Warehouse |OLAP and OLTP System |ER Diagram in DBMS | Entity Relationship Diagram | ER | How to draw an ER Diagram in DBMS | How to make an ER Diagram Description: Welcome to the "Mastering Data Modeling" playlist! In this series, we dive deep into the world of data modeling...
01 Data Modelling Tutorial: ER (Entity Relationship) Diagram in detail
Переглядів 780Рік тому
Data Modelling Tutorial: ER Diagram in DBMS | Entity Relationship Diagram | ER | How to draw an ER Diagram in DBMS | How to make an ER Diagram Description: Welcome to the "Mastering Data Modeling" playlist! In this series, we dive deep into the world of data modeling, exploring everything from the fundamentals to advanced techniques. Whether you're a data analyst, data scientist, database admin...
Microsoft Fabric KQL:How to read data from kafka Realtime stream and load data into the KQL Database
Переглядів 1,2 тис.Рік тому
PowerBI Microsoft Fabric Tutorial:How to read data from kafka Realtime stream and load data into the KQL Database. | Microsoft Fabric RealTime Analytics | Microsoft Fabric KQL Database | Microsoft Fabric KQL Query set | Microsoft Fabric Event Streaming Microsoft fabric tutorial Series: How to read data from an event stream and load data into the KQL Database. Microsoft fabric data pipeline scri...
1 Microsoft Fabric Dataflow Gen2: How to read data from SharePoint Site and load data into Lakehouse
Переглядів 4,7 тис.Рік тому
dataflow gen2 | Microsoft Fabric Data Pipeline - How to read data from SharePoint Site using Microsoft Fabric Dataflow Gen2 and load data into the Lakehouse Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Wareho...
09 Microsoft Fabric Data Pipeline: How to use procedure activity in Microsoft Fabric Data Pipeline
Переглядів 1,3 тис.Рік тому
Microsoft Fabric Tutorial Series- How to use procedure activity in Microsoft Fabric Data Pipeline to drop table in Microsoft Fabric Data warehouse. Microsoft fabric data pipeline script activity: ua-cam.com/video/gxoAWCtP07I/v-deo.html Microsoft Fabric data Lakehouse end to end project - ua-cam.com/video/gKdlsHm7QgU/v-deo.html If you want to know more about Microsoft Fabric data Warehouse, Plea...
005 Microsoft Fabric KQL: How to read data from an event stream and load data into the lakehouse
Переглядів 1,5 тис.Рік тому
005 Microsoft Fabric KQL: How to read data from an event stream and load data into the lakehouse
004 Microsoft Fabric KQL: How to read data from an event stream and load data into the KQL Database
Переглядів 673Рік тому
004 Microsoft Fabric KQL: How to read data from an event stream and load data into the KQL Database
003 Microsoft Fabric KQL Database: What is KQL Query Set and Create PowerBI Report from KQL Query
Переглядів 803Рік тому
003 Microsoft Fabric KQL Database: What is KQL Query Set and Create PowerBI Report from KQL Query
Microsoft Fabric KQL Database:How to Create KQL database and upload local file into the KQL Database
Переглядів 609Рік тому
Microsoft Fabric KQL Database:How to Create KQL database and upload local file into the KQL Database
001 Microsoft Fabric KQL Database: Realtime Analytics, Event Streaming and KQL Database Overview
Переглядів 798Рік тому
001 Microsoft Fabric KQL Database: Realtime Analytics, Event Streaming and KQL Database Overview
08 Microsoft Fabric Data Pipeline: How to Use Lookup and Foreach Activity in Data Pipeline
Переглядів 3,7 тис.Рік тому
08 Microsoft Fabric Data Pipeline: How to Use Lookup and Foreach Activity in Data Pipeline
07 Microsoft Fabric Data Pipeline: How to use Dataflow gen2 in Microsoft Fabric Data Pipeline
Переглядів 1,1 тис.Рік тому
07 Microsoft Fabric Data Pipeline: How to use Dataflow gen2 in Microsoft Fabric Data Pipeline
002 Microsoft Fabric Tutorial: How to login to Microsoft fabric account using Personal Email
Переглядів 2 тис.Рік тому
002 Microsoft Fabric Tutorial: How to login to Microsoft fabric account using Personal Email
009 Microsoft Fabric Data warehouse: How to connect to warehouse through PowerBI Desktop #microsoft
Переглядів 1,3 тис.Рік тому
009 Microsoft Fabric Data warehouse: How to connect to warehouse through PowerBI Desktop #microsoft
008 Microsoft Fabric Lakehouse: How to Connect to Lakehouse through PowerBI Desktop #microsoftfabric
Переглядів 1,3 тис.Рік тому
008 Microsoft Fabric Lakehouse: How to Connect to Lakehouse through PowerBI Desktop #microsoftfabric
06 Microsoft Fabric Data Pipeline: How to use Teams activity in data pipeline #microsoftfabric
Переглядів 592Рік тому
06 Microsoft Fabric Data Pipeline: How to use Teams activity in data pipeline #microsoftfabric
05 Microsoft Fabric Data Pipeline: How to use Outlook activity in data pipeline #microsoftfabric
Переглядів 899Рік тому
05 Microsoft Fabric Data Pipeline: How to use Outlook activity in data pipeline #microsoftfabric
04 Microsoft Fabric Data Pipeline: How to use Get metadata activity of data pipeline #microsoft
Переглядів 1,5 тис.Рік тому
04 Microsoft Fabric Data Pipeline: How to use Get metadata activity of data pipeline #microsoft
03 Microsoft Fabric Data Pipeline: Create schema and table using Script activity in data warehouse
Переглядів 2,3 тис.Рік тому
03 Microsoft Fabric Data Pipeline: Create schema and table using Script activity in data warehouse
03 Microsoft Fabric Dataflow gen2: Load/Get Data from Rest API into Lakehouse Using Dataflow gen2
Переглядів 1,6 тис.Рік тому
03 Microsoft Fabric Dataflow gen2: Load/Get Data from Rest API into Lakehouse Using Dataflow gen2

КОМЕНТАРІ

  • @poonammalpani561
    @poonammalpani561 4 дні тому

    Next button is disable

  • @murthyherofashion
    @murthyherofashion 20 днів тому

    🎉amazing

  • @pragatisharma6036
    @pragatisharma6036 Місяць тому

    if my file get new data it will also work for the updated data or should i need to creat Agin the data flow

    • @DataVerse_Academy
      @DataVerse_Academy Місяць тому

      You can implement the incremental load inside the dataflow gen2 as well.

  • @GiancarloBlaze
    @GiancarloBlaze Місяць тому

    Very informative! thank you

  • @kiranj8769
    @kiranj8769 Місяць тому

    Excellent, great video..🎉

  • @nickfaleev9233
    @nickfaleev9233 Місяць тому

    I just finished watching this video on using PySpark in Microsoft Fabric, and it was incredibly helpful! The explanations were clear, and the step-by-step walkthrough made complex concepts easy to understand. This is definitely a must-watch for anyone looking to get started with PySpark in Microsoft Fabric. Great job, and thank you for sharing this valuable content! I highly recommend subscribing to this channel. The content is consistently high-quality, with tutorials that are not only informative but also easy to follow. Whether you're a beginner or an experienced professional, this channel offers valuable insights and tips that can help you stay ahead in the data world. Don't miss out on the great content they’re putting out!

  • @gopalammanikantarao593
    @gopalammanikantarao593 Місяць тому

    Nice Video, It's helped me to understand M.Fabric flow.

  • @Otto_at_work
    @Otto_at_work Місяць тому

    Very useful. Thank you.

  • @surerakase5017
    @surerakase5017 2 місяці тому

    Can we pass parameters in dataflow gen2

  • @meashhh1134
    @meashhh1134 2 місяці тому

    Very very good explanation Thank you

  • @balajikrishnamoorthy7352
    @balajikrishnamoorthy7352 2 місяці тому

    will this work if we delete or update a data in the source table ?

    • @DataVerse_Academy
      @DataVerse_Academy 2 місяці тому

      It will work in case of update , not in delete. As it’s incremental approach not CDC.

  • @conexaomotorizada
    @conexaomotorizada 2 місяці тому

    Hi, how can i edit a table from a semantic model in power bi's power query ?

  • @namangarg7023
    @namangarg7023 2 місяці тому

    Can u pls explain how u get connection oath for source over here like we did in ADF

  • @moeeljawad5361
    @moeeljawad5361 2 місяці тому

    Thanks for your video, very intuitive and impressive. I have a question, if your Table_List table is saved as a delta table from a python script, and i would like to have the Max_value updated.... I believe i will not be able to keep the stored procedure, right? if not should i be replacing it with a notebook activity? Thanks

    • @DataVerse_Academy
      @DataVerse_Academy 2 місяці тому

      Yes you are right you will not be able to use stored procedure to update a delta table. You need to create a notebook for that.

  • @Giraffe-j3h
    @Giraffe-j3h 2 місяці тому

    how to create new columns and dim date table in this model

  • @vinaypratapsingh5815
    @vinaypratapsingh5815 2 місяці тому

    When i am clicking on New Semantic Model , I am not able to see all those tables to select a table or all tables . Because of that i am not able to create Semantic Model. Could you please help me here ? Thanks

    • @DataVerse_Academy
      @DataVerse_Academy 2 місяці тому

      Whats the error you are getting ?

    • @vinaypratapsingh5815
      @vinaypratapsingh5815 2 місяці тому

      @@DataVerse_Academy Thanks for your response. I am not getting any error, but I am not able to select any table to create my semantic model . Under Select all , it's not giving me tables name to select the table name

    • @DataVerse_Academy
      @DataVerse_Academy 2 місяці тому

      Please try below once Settings- admin portal- > tenant settings - > information protection -> allow users to apply sensitivity labels for content - enable this, Then you will be able to create semantic model through lakehouse

    • @julies5085
      @julies5085 17 днів тому

      @@vinaypratapsingh5815 Hi, I am facing the same issue, only 3 tables shown for selection , others not shown to be used for semantic model creation. Were you able to solve this problem?

  • @priyanthakarunathilake8030
    @priyanthakarunathilake8030 2 місяці тому

    Really helpful. Thanks.

  • @ceciliaayala3923
    @ceciliaayala3923 2 місяці тому

    Great explanation, thank you!

  • @sridharnallagatla7442
    @sridharnallagatla7442 2 місяці тому

    For microsoft fabric how the market there is any calls

    • @DataVerse_Academy
      @DataVerse_Academy 2 місяці тому

      Not now, but you will a lot of movement towards fabric in upcoming 1-2 year.

  • @tv.TheDogFather
    @tv.TheDogFather 2 місяці тому

    Thanks for the video... Gold_Product is still not included in the Code zip file. Can you please include it? Not as Important, but at the same time, can you include the Run_Load notebook?

  • @BrundaAS-e8r
    @BrundaAS-e8r 2 місяці тому

    Not able to access the azure blob storage link you have shared

  • @pragatisharma6036
    @pragatisharma6036 2 місяці тому

    product script is missing in data code file please upload it

  • @andreaskoblischke8186
    @andreaskoblischke8186 3 місяці тому

    Nice one. Usefulefor me. Thx

  • @lucasamorim5823
    @lucasamorim5823 3 місяці тому

    Thank you, help me a lot with this short video.

  • @seetha110378
    @seetha110378 3 місяці тому

    how to read data from a REST API that has authentication tokens (Refresh tokens) into a Fabric lakehouse?

  • @priyankaparida8423
    @priyankaparida8423 3 місяці тому

    Hello Sir, After line no. 23 it is directly showing line no.77 .the middle part is skipped so not getting the code in between that. can you help with it.

  • @dhheanom10d
    @dhheanom10d 3 місяці тому

    Thank u so much!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  • @ADhuidv
    @ADhuidv 3 місяці тому

    Hello sir, Thank you so much providing these productive videos. Today, I faced a challenge, and the solution I couldn't find elsewhere. That is How to Extract data from SAP Hana Cloud to Microsoft Fabric (cloud to cloud connectivity). Could you please help me here?

  • @heyrobined
    @heyrobined 4 місяці тому

    So I did the same with on prem sql server to the warehouse , and checked every connections are preview working but when I am validating the activity I got this error "Copying data from on-premises connection with staging storage in workspace is not supported. Please use external staging storage instead." what could be the solution?

  • @vamshisamineniz5905
    @vamshisamineniz5905 4 місяці тому

    Is there a way to connect to ADLS directly without creating Shortcut in Lakehouse

  • @ADhuidv
    @ADhuidv 4 місяці тому

    Sir, How can we build the JDBC/Pyodbc connection between Fabric Data warehouse and Fabric Notebook. I have been finding it since a long time, but un-successful

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      But why do you need it, what is the use case which you are trying to implement?

    • @ADhuidv
      @ADhuidv 4 місяці тому

      1.Initially, we are getting data from multiple sources and sinking them in one warehouse (Raw Data). 2. Now we want to extract data from this warehouse (Raw Data) to another warehouse (Transformed Data) through a Notebook wherein we will be performing our transformation logic. Hence, I want to build the connection between warehouse and Notebook only using JDBC or Pyodbc

  • @Shreekanthsharma-t6x
    @Shreekanthsharma-t6x 4 місяці тому

    Hi , I have some complex "Scalar user defined functions" defined in MYSQL and I have to migrate them to fabric, but as of now fabric doesn't support creation of "Scalar user defined functions" in warehouse. In this scenario please let me know alternative options I can use. Thanks

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      you can build that logic inside the procedure. I know you will not able to return a value using a function, but you can build whatever the logic which you are trying to build. If you can give me context, then i will provide you the code as well

  • @ADhuidv
    @ADhuidv 4 місяці тому

    Sir, Can we extract the data directly from warehouse to the notebook, then transform it, and then finally save it to the same warehouse??

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      You can transform the data inside a notebook from warehouse, but you can’t write the data into the warehouse.

  • @Shreekanthsharma-t6x
    @Shreekanthsharma-t6x 4 місяці тому

    Hi Good Morning!, I have to convert the existing SQL server stored procedure into fabric environment, In my stored procedures there are CURSOR commands but fabric doesn't support CURSOR commands, in this case how do I proceed, is there any alternative.

  • @Bharathkumar-l3n
    @Bharathkumar-l3n 4 місяці тому

    Hi sir, can we use metadata activity instead of look up activity, in order to perform same operation and get same result? Is metadata activity can do the same work as look up activity do?

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      No, you will not be able to write query inside the meta data activity.

  • @EduInquisitive
    @EduInquisitive 4 місяці тому

    thanks for the great explanation but here i didnt get one thing and that is where the metadata for managed table is getting created. I cant see any files created in Files folder while for external we can see them in the path provided.

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      for managed tables, metadata and data is inside the tables folder itselft

  • @Shreekanthsharma-t6x
    @Shreekanthsharma-t6x 4 місяці тому

    this is great video. thanks

  • @Shreekanthsharma-t6x
    @Shreekanthsharma-t6x 4 місяці тому

    I have a SQL server stored procedure which updates, deletes and merges data into a table , how do I convert the stored procedure to pyspark job, is it possible to update a table in fabric using pyspark?, please make a video on this topic

    • @DataVerse_Academy
      @DataVerse_Academy 4 місяці тому

      It’s very easy to do the same thing in pyspark, we can do all the stuff which you mentioned. I am a on break for couple of months. I am going to start creating video very soon.

    • @Shreekanthsharma-t6x
      @Shreekanthsharma-t6x 4 місяці тому

      @@DataVerse_Academy please do create a video when you are back from break. Thanks

  • @AnisurRahman-wm2ys
    @AnisurRahman-wm2ys 5 місяців тому

    Excellent !!!!! Do you have this type of video for SCD2 ?

    • @tv.TheDogFather
      @tv.TheDogFather 2 місяці тому

      I think for the DIM Merges just wrap the merge inside an Insert Into and change the Update of the Merge Accordingly.

  • @yveshermann
    @yveshermann 5 місяців тому

    this guy is a champion!! Thanks so much :):)

  • @IEYdel
    @IEYdel 5 місяців тому

    Super helpful! Do you have a video that shows the silver layer with an example of joining related data from heterogenous data sources with data cleansing and deduplication? :D Still you are my hero Vishnu! Thank you for this video!

  • @gagansingh3481
    @gagansingh3481 5 місяців тому

    Sir could you please make a video on Azure log analytics for Semantic Model and azure mirroring in fabric

  • @rajudasari8482
    @rajudasari8482 5 місяців тому

    which table is speed in retrieving the data

  • @sanishthomas2858
    @sanishthomas2858 5 місяців тому

    Nice. quick question, the Presentation slide shown for Architecture is Power Point or any other software?

  • @ROHITKUMARGUJAR-k8t
    @ROHITKUMARGUJAR-k8t 5 місяців тому

    Very good explanation.

  • @John.Wick.221
    @John.Wick.221 5 місяців тому

    Where can I get more such data source

  • @samuel_t_chou
    @samuel_t_chou 5 місяців тому

    Thank you, random Indian UA-camr. Very clear and useful.

  • @orasha4846
    @orasha4846 5 місяців тому

    Can you read multiple files if they have the same structure?

    • @DataVerse_Academy
      @DataVerse_Academy 5 місяців тому

      Yes we can do that. It just like we read the data from a folder in PowerBI.

    • @kel78v2
      @kel78v2 3 місяці тому

      Gen2 works fine with one file but throws a data type error from transformation folder when the dataflow combines multiple files

  • @JatinHingorani-jm3hc
    @JatinHingorani-jm3hc 5 місяців тому

    i followed this process but getting error loading data in import mode "Unable to open the physical file. Operating system error 5: “5(Access is denied.)”" any idea why access denied error i have pro license and Member access of workspace

  • @longphamminh5804
    @longphamminh5804 5 місяців тому

    why did you create two folder "current" and "archive" in Files

    • @DataVerse_Academy
      @DataVerse_Academy 5 місяців тому

      To archive the processed file from current to archive folder.

    • @longphamminh5804
      @longphamminh5804 5 місяців тому

      Thank you for answer

    • @longphamminh5804
      @longphamminh5804 5 місяців тому

      @@DataVerse_Academy I have one more question: When to use spark.read.table() and spark.sql