- 141
- 273 794
Dagster
United States
Приєднався 27 тра 2019
Ship data pipelines with extraordinary velocity with Dagster.
Dagster helps data engineers tame complexity. Elevate your data pipelines with software-defined assets, first-class testing, and deep integration with the modern data stack.
Dagster is a cloud-native open-source orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.
Dagster helps data engineers tame complexity. Elevate your data pipelines with software-defined assets, first-class testing, and deep integration with the modern data stack.
Dagster is a cloud-native open-source orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.
Code Locations Explained
Code locations are one of the unique features in Dagster that teams use to isolate environments while still maintaining a single pane of glass on their entire Data Platform. They are perfect if you are working in a large organization with multiple teams or managing data pipelines for various clients. Code locations provide the flexibility and isolation needed to ensure smooth and efficient workflows.
Переглядів: 27
Відео
The Evolution of Data Catalogs and Why They Matter
Переглядів 1379 годин тому
Dagster's integrated approach to data cataloging improves developer experience and enhances data governance and discovery, creating a win-win for technical teams and business users alike. Alex Noonan, Developer Advocate at Dagster Labs, walks us through the history of data catalogs, the unfulfilled promise of stand-alone data catalogs, and the differentiated approach Dagster takes by integratin...
Unlocking the Power of Dagster and dbt
Переглядів 533День тому
Timeline: 0:00 Why Dagster and dbt work well together 1:16 Dagster UI benifits 1:55 How to setup your dbt project in Dagster 2:32 Learning Resources Unlock the Full Potential of Your dbt Projects with Dagster! 🚀 Discover why Dagster is the ultimate platform for deploying and managing your dbt projects. In this video, we dive into: How Dagster and dbt create a powerful synergy for your data plat...
Orchestrating Flexible Compute for ML with Dagster and Modal
Переглядів 61814 днів тому
Project Repository: github.com/dagster-io/dagster-modal-demo In this Deep Dive, we showcase how Dagster and Modal combine to orchestrate scalable workloads with an intuitive developer experience to build scalable, robust pipelines. We discuss how Dagster and Modal can be used together to accelerate machine learning workflows. Dagster, a data pipeline orchestration tool, manages pipeline state a...
Building a True Data Platform: Beyond the Modern Data Stack - a Dagster Deep Dive with Pedram Navid
Переглядів 822Місяць тому
The Modern Data Stack offers countless tools but often creates hard-to-manage pipelines. Pedram Navid, Head of Data Engineering and Developer Relations at Dagster Labs shows how to transform disjointed tools into a unified, observable data platform with default high-quality data. 00:00 Introduction 00:35 Agenda 00:52: The Unkept Promise of the Modern Data Stack 06:50: The Data Platform Engineer...
Dagster, SDF, & the Evolution of the Data Platform (A Dagster Deep Dive)
Переглядів 938Місяць тому
Explore how the combined strengths of Dagster’s orchestration and SDF’s transformation capabilities can enhance your developer experience, streamline your data pipelines, reduce costs, and enhance data quality and reliability. Key Takeaways: - Unified Workflow Management: Seamlessly integrate and manage your data workflows. - Enhanced Data Quality: Ensure consistent and reliable data through ad...
Building Reliable Data Platforms (A Dagster Deep Dive)
Переглядів 1,2 тис.2 місяці тому
Data quality issues are the silent killers of today’s data-driven initiatives. Poor data quality leads to incorrect insights, flawed decisions, operational inefficiencies, and ultimately, financial loss. In this Dagster Deep Dive, Colton Padden discusses strategies for integrating comprehensive data quality checks into data pipelines using #Dagster, providing practical insights and real-world e...
Introducing Dagster+ : Beyond traditional orchestration
Переглядів 1,8 тис.3 місяці тому
The Dagster Labs team introduces you to Dagster , a data orchestration platform that pushes the boundaries of traditional orchestration. With Dagster you can weave data quality, data cataloging, DataOps best practices, and advanced observability into your platform. 00:00 Intro 04:42 Data Catalog 09:57 Data Reliability 16:28 Branch Deployments with Change Tracking 19:40 Dagster Insights 26:06 Pa...
Dagster+ Launch: Welcome
Переглядів 4423 місяці тому
Welcome to the launch of Dagster , a data orchestration solution that aims to redefine the category. Pete Hunt, CEO of Dagster Labs shares the new features that your team can leverage to accelerate your data engineering work: - Data Cataloging - Data Reliability Checks (Quality, Freshness, Schema) - Branch Deployments with Change Tracking - Dagster Insights
Dagster+ overview
Переглядів 5 тис.5 місяців тому
Dagster is a powerful orchestration solution for building and running modern data platforms. It goes well beyond traditional orchestration tools, building on the rich metadata collected during runs to build high-order logic and observability into your data pipelines. Unlike workflow-oriented orchestrators (Airflow, Prefect) Dagster's framework is compatible with the other asset-oriented tools i...
Dagster+ implementation partner update
Переглядів 2035 місяців тому
Eric Chernoff, Head of Partnerships at Dagster Labs, provides an update on the growth of the professional services support network and the hyperscaler integrations for the Dagster ecosystem.
Dagster+ Branch Deployments with Change Tracking
Переглядів 3245 місяців тому
To speed up innovation, data engineering teams need to embrace software engineering best practices. Dagster provides a unique capability for data teams to rapidly iterate, deploy, and test changes in ephemeral staging environments. Jamie DeMaria, software engineer at Dagster Labs, walks us through how Branch Deployments can benefit your team, and how the new Change Tracking feature lets you zer...
Dagster+ - Join the launch event on April 17th
Переглядів 7856 місяців тому
Dagster - Join the launch event on April 17th
Dagster and the Data Mesh (A Dagster Deep Dive)
Переглядів 1,4 тис.6 місяців тому
Dagster and the Data Mesh (A Dagster Deep Dive)
Pipeline Sandboxing: Boosting developer productivity with LakeFS & Dagster
Переглядів 6056 місяців тому
Pipeline Sandboxing: Boosting developer productivity with LakeFS & Dagster
Thinking in Partitions (A Dagster Deep Dive)
Переглядів 1,9 тис.7 місяців тому
Thinking in Partitions (A Dagster Deep Dive)
Configuration & Resources (A Dagster Deep Dive)
Переглядів 2,5 тис.7 місяців тому
Configuration & Resources (A Dagster Deep Dive)
Flexible Scheduling with Automation in Data Engineering (A Dagster Deep Dive)
Переглядів 2,2 тис.8 місяців тому
Flexible Scheduling with Automation in Data Engineering (A Dagster Deep Dive)
Dagster Shorts: Thinking in Partitions
Переглядів 1,5 тис.9 місяців тому
Dagster Shorts: Thinking in Partitions
Data Quality as part of the Data Pipeline
Переглядів 2,4 тис.9 місяців тому
Data Quality as part of the Data Pipeline
Dagster's run UI & debugging features
Переглядів 62010 місяців тому
Dagster's run UI & debugging features
Why data engineers are moving their data pipelines off Airflow and onto Dagster
Переглядів 77310 місяців тому
Why data engineers are moving their data pipelines off Airflow and onto Dagster
Building a trusted and productive data platform with Software-defined Assets - a fireside chat.
Переглядів 76110 місяців тому
Building a trusted and productive data platform with Software-defined Assets - a fireside chat.