Announcing Delta Sharing with Demo | Matei Zaharia | Keynote Data + AI Summit NA 2021
Вставка
- Опубліковано 25 сер 2024
- "Data needs to flow beyond the borders of individual organizations," says Databricks CEO Ali Ghodsi. He announces Delta Sharing, the industry's first open protocol for secure data sharing, as open source under the Linux Foundation. Databricks Chief Technologist Matei Zaharia dives into the goals and the details of being a data provider or a data recipient.
Register for free to see the rest of the keynotes and exciting announcements live, plus over 200+ sessions. Learn from the creators and top contributors of technologies like PyTorch, TensorFlow, MLflow, Delta Lake, Apache Spark, Hugging Face, DBT and more.
databricks.com...
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / databricks
Instagram: / databricksinc Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. databricks.com...
Python sharing in two lines: one line to import delta_sharing, and a second line to get a pandas dataframe from the share! Sweet and simple.
Oh, and the Tableau and PowerBI integrations are great too - if you are into this whole new GUI thing! ;-)
"Reduce friction of consumption", right on!. Really excited to see Delta Lake mature out of the Databricks ecosystem and embrace openness.
I believe it should work cross cloud ( Azure Dataset --> AWS/GCP) . What are the performance , security considerations , limitations and challenges you observed? Are you looking at feature like caching and live table kinda replications in future for the reciepients in this such scenarios?
wow its amazing! Very surprised
22:46 Tableau, 24:05 Power BI
Do we need any specific cluster to do this? when I give create share this is not worked ..I choose Notebook as SQL
I am facing error in creating share . Running command -" create share default.annual_enterprise_survey_delta; ". It throwing me error -" no viable alternative at input 'create share'(line 1, pos 7)". I am using community edition of databricks with runtime version 8.2, spark version 3.1.1 and scala version 2.12. Please help me to resolve this error.
same in azure databricks
Even i am getting the same error. The documentation says that it is in preview and we need to signup for early release.
Error in SQL statement: DatabricksServiceHttpClientException: ENDPOINT_NOT_FOUND: Endpoint not found for /2.0/unity-catalog/shares
I'm getting this error , using DBR 10. Any Idea what to do?
what if the data in S3 is encrypted?
This works with S3 server-side encryption, even with customer-managed encryption keys!
Can we access the delta share object as data recipient from
SQL server? Or it can be accessed using azure databrick ? Please suggest
creator of spark is the creator of delta sharing...
Hi everybody, who can help me how I can configure the yaml. Someone can give me a complete example, or complete documentation for this?thanks so much.
When will Unity Catalogue come in Azure Databricks
While running Create Share command it throws me error. [Error in SQL statement] I am running the command on cluster Apachek saprk 3.1.1 and python version is 3.8.8. Please help me to sort out this error
I want to share the activation link (as in the 22:50) for my local delta sharing server using python so that I could use web connector of Tableau and open my delta tables in Tableau, is it possible using Python?
Do you know when the Delta sharing connector will be available in Power BI desktop? At least beta version? I do not see it at all in October 2021 version 😔.
it's available now -- check out powerbi.microsoft.com/en-us/blog/power-bi-november-2021-feature-summary/#post-17912-_Toc87473948
@databricks How to remove recipient ? How to delete the delta share name which is created ?
documentation has to improve - how do I start my own delta sharing server?
#rust ❤ #rustlang