Great video! Interesting how Fivetran is taking steps to manage more of the data pipeline process. One of the biggest challenges with trying to operationalize dbt core is hosting the orchestration, so this should help a lot!
I think I'm missing some logic here (or my setup is wrong). In Fivetran I have multiple connectors defined, each has it's own dbt project and therefore repo on github. The Fivetran setup is one transformation (and one repo) for everything (ie not by connector). Is there a way to configure Fivetan to know which repo to connect to for the dbt project for this particular connector ?
Hi Kahan, thank you for uploading this video. I have a question about the default schema at 7:34 , I have already defined my schemas for base, stage and cdm in dbt_project.yml, will the default schema in Fivetran overwrite that in dbt_project.yml?
ok, got it, it is added at the front of the schema defined in dbt_project.yml. To remove it in the schema name, I have to build a custom generate_schemas.sql.
Hey Michael, do you know if FiveTran will enable triggers for 'when new data is loaded into your destination' instead of scheduled triggers? Looks like only scheduled is supported for dbt transformations at the moment.
Hi Jonathan - Good question. I'm not sure if that will become a specific feature. Based on the current design of the deployment.yml file, I would assume that it will remain on just a scheduled basis until something changes. It might be worth sending that feedback to Fivetran though!
Hi Jonathan, we're excited to share that this feature is now available in Beta. If you are a Fivetran user and you want access to Integrated Scheduling to trigger your dbt Core model runs from Fivetran, reach out to your account manager and we can get you added.
►► The Starter Guide for Modern Data (Free PDF) → www.kahandatasolutions.com/startermds
Great video! Interesting how Fivetran is taking steps to manage more of the data pipeline process.
One of the biggest challenges with trying to operationalize dbt core is hosting the orchestration, so this should help a lot!
Agreed and I think a lot of others have experienced the same thing. Really curious to see how dbt evolves in 2022 and beyond.
Great demo. Thanks for sharing
Thanks for watching!
Amazing Video. Still stands as today!
I think I'm missing some logic here (or my setup is wrong). In Fivetran I have multiple connectors defined, each has it's own dbt project and therefore repo on github. The Fivetran setup is one transformation (and one repo) for everything (ie not by connector). Is there a way to configure Fivetan to know which repo to connect to for the dbt project for this particular connector ?
Hi Kahan, thank you for uploading this video. I have a question about the default schema at 7:34 , I have already defined my schemas for base, stage and cdm in dbt_project.yml, will the default schema in Fivetran overwrite that in dbt_project.yml?
ok, got it, it is added at the front of the schema defined in dbt_project.yml. To remove it in the schema name, I have to build a custom generate_schemas.sql.
Can there be multiple git repos referenced by Fivetran?
Hey Michael, do you know if FiveTran will enable triggers for 'when new data is loaded into your destination' instead of scheduled triggers? Looks like only scheduled is supported for dbt transformations at the moment.
Hi Jonathan - Good question. I'm not sure if that will become a specific feature. Based on the current design of the deployment.yml file, I would assume that it will remain on just a scheduled basis until something changes. It might be worth sending that feedback to Fivetran though!
Hi Jonathan, we're excited to share that this feature is now available in Beta. If you are a Fivetran user and you want access to Integrated Scheduling to trigger your dbt Core model runs from Fivetran, reach out to your account manager and we can get you added.
@@Fivetran Can there be multiple git repos referenced by Fivetran? Say for different teams?