Microsoft Fabric Notebooks - Showcase with advanced features
Вставка
- Опубліковано 9 жов 2023
- Welcome back to another episode of Fabric Espresso DE&DS series. In this episode, we introduce and explore the nuances of Fabric Notebooks, a web-based interactive surface that's rapidly becoming the go-to tool for data scientists and engineers for writing big data analytic jobs using Apache Spark.
Key points presented by Jene:
What are Fabric Notebooks? Microsoft Fabric notebook is a primary code item for developing Apache Spark jobs and machine learning experiments. It's a web-based interactive surface used by data scientists and data engineers to write code benefiting from rich visualizations and Markdown text.
Distinctive Features and Focus Area: Unlike its counterparts, Fabric Notebooks are tailored for developers working in data scenarios, providing a seamless experience for tasks like data ingestion, data preparation, data transformation, and data analytics.
Comparison to Synapse and Jupyter Notebooks: Compared to Synapse Notebooks, Fabric Notebooks are built on a SaaS platform, offering various SaaS-like features including low-code experience and enhanced collaborative features. They facilitate easier interaction with PowerBI report/dashboard.
Unlike Jupyter Notebooks, Fabric Notebooks are integrated tightly with the lakehouse and other Fabric key features and functionalities and are rich in spark specific features like high concurrency mode to share a spark session, spark job inline monitoring, spark job diagnostics, and built-in Microsoft Spark Utilities.
Advanced Features: The notebooks also support advanced features like IPywidgets and reference run which significantly enhance functionality, although at the time of this interview, there are some compatibility issues with IPywidgets which are being addressed.
🎙 Meet the Speakers:
👤 Guest from Microsoft Fabric Product Group: Jene Zhang, Senior Program Manager of Fabric Notebooks
Jene is a Senior Program Manager at Microsoft with a passion for developing advanced developer tools and AI technology. Currently, she is spearheading the creation of a specialized Spark notebook aimed at enabling data engineers and scientists to excel in their domains.
Linkedin: / edelweissno1
Twitter: / edelweissno1
👤 Host: Estera Kot, Senior Product Manager at Microsoft and a member of the Fabric Product Group. She holds the role of Product Owner for Apache Spark-based runtimes in Microsoft Fabric and Synapse Analytics. Estera is a Data & AI Architect and is passionate about computer science.
LinkedIn: / esterakot
Twitter: / estera_kot
👍 Like this video? Don't forget to hit the 'Like' button and share it with your network!
🔔 Stay Updated: For more insights into Microsoft Fabric Data Engineering and Data Science, and all things tech, make sure to subscribe to our channel and hit the notification bell so you never miss an episode!
#Microsoft #MicrosoftFabric #FabricNotebooks #DataScience #BigData #DataAnalytics #ApacheSpark #JupyterNotebook #SynapseNotebook #IPywidgets #TechInnovation #MicrosoftSparkUtilities
FYI: Please timestamp videos > 5 min, it would make it much easier to consume long-form content esp when there's so many cool things like this :)
Example:
01:05 Overview of Fabric Notebooks
02:08 Fabric Notebooks vs. Synapse Notebooks
04:04 Fabric Notebooks vs. Jupyter Notebooks
05:56 Feature: Lakehouse Explorer pane drag & drop
08:01 Feature: Notebook resources pane
09:00 Feature: Python module support
09:51 Feature: Notebook sharing with permissions control
10:36 Feature: Real-time co-editing
11:13 Feature: Commenting pane (coming: tagging in comments)
12:04 Feature: Immersive authoring (auto-save, code-assist, etc.)
14:28 Feature: Built-in code snippets
15:00 Feature: iPython kernel and widgets (interactive elements)
16:00 Feature: Magics (i.e. like macros; run child notebooks, %pip install, custom magic)
18:47 Feature: Table of contents
19:30 Feature: Visualization - display function
19:55 Feature: display fnxn - interactive chart view (coming soon: exploration feature)
21:03 Feature: display fnxn - summary attribute
22:23 Feature: Open source library visualization (i.e. matplotlib, seaborn, bokeh etc.)
23:28 Feature: Markdown support (drag & drop images from local PC to import)
25:11 Topics TBD in the next episode
Awesome
Interesting fact
Hi Jene, concerning "build-in display function" (19:39):
Currently the *table view does not allow selecting/copying values* from the displayed result. Will this be fixed with the mentioned improvements for GA (20:40)?
Thanks for the feedback! That a super valid scenario but unfortunately we won't have it by GA, because the "Select" state in the next version will have specific reactions (preview value, calculate summary, etc), but I'll record this request in our backlog, hope we can find some way to get it in next wave!
Do we need to manage compute resources for the workloads? Like creating clusters in Databricks or synapse
Please expand more on python modules when I try to use my modules after the drag and drop import I get an error if trying to get functions defined in the module.
The data wrangler was not demonstrated, very helpful for gaining insight into your data frame
Data Wrangler is here ua-cam.com/video/-g6KveKQXu4/v-deo.html
Can I use notebook for any python code or should it only be used with Spark?
how can we read a variable in Notebook that's set using SetActity in MS Fabric.
Can you share notebook please?
we'll publish a polished version in the "Use a sample" on the data engineering/data scientist workload homepage😊
Would be nice if dataframe will be uniqly named rather simple df, name of table or file as default will be nice, Also i could imagine configuring a simple naming pattern on lakehouse level, which could enfoce starnard.
Are you referring to the dataframe name that automatically generated from drag&drop?
@@JeneZhang Yes, that will speed up development, now users have to rename it. Specialy if they are going to use data wrangler, where they need meaningful names to identify df
Got it, that's a good suggestion! I'll see what I can do to make it smoother, thank you for the valuable feedback!😎@@pp56825