Hi, I have got error like : Not found: Dataset peak-catbird-440802-b3:dataform was not found in location US i have given all permissions as you mentioned and the dataset loction is us in bigquery.
Yes, with the Dataform CLI (or SDKs). Something like "dataform run" when you have the CLI installed. The gcloud CLI does not appear to support Dataform.
You can use Cloud Storage BigLake tables, object tables, or external tables, like you would in normal BigQuery. Or did you have a different use case in mind? cloud.google.com/bigquery/docs/external-data-sources
SQLX is essentially a templated version of SQL, which also includes some metadata (the details at the top). Most SQL scripts are valid SQLX scripts - it's sort of an extension of SQL.
Great explanation of the data form product, do you know if it is possible to integrate dataform with bitbucket? How can I force the dataform to run through composer airflow?
Yes, BitBucket integration is possible. The Dataform Airflow operator should work for using Dataform via Composer - typically, this will be via the DataformCreateWorkflowInvocationOperator. I would recommend our Analytics Engineering with Dataform playlist if you want to dive into the details of Dataform. ua-cam.com/video/-AwB-QcHIAU/v-deo.html
If you'd like more detail, check out our series on Analytics Engineering with Dataform:
ua-cam.com/play/PLthij7va0xuK1RezLqyLUHgYCFadHJ8Ol.html
Hi,
I have got error like : Not found: Dataset peak-catbird-440802-b3:dataform was not found in location US
i have given all permissions as you mentioned and the dataset loction is us in bigquery.
Hello, Can I Execute Dataform Workflows in CloudShell?
Yes, with the Dataform CLI (or SDKs). Something like "dataform run" when you have the CLI installed. The gcloud CLI does not appear to support Dataform.
Is it possible to read a file directly from Google Cloud Storage using dataform?
You can use Cloud Storage BigLake tables, object tables, or external tables, like you would in normal BigQuery. Or did you have a different use case in mind?
cloud.google.com/bigquery/docs/external-data-sources
Can you explain what is SQL X file
SQLX is essentially a templated version of SQL, which also includes some metadata (the details at the top). Most SQL scripts are valid SQLX scripts - it's sort of an extension of SQL.
Great explanation of the data form product, do you know if it is possible to integrate dataform with bitbucket? How can I force the dataform to run through composer airflow?
Yes, BitBucket integration is possible. The Dataform Airflow operator should work for using Dataform via Composer - typically, this will be via the DataformCreateWorkflowInvocationOperator.
I would recommend our Analytics Engineering with Dataform playlist if you want to dive into the details of Dataform.
ua-cam.com/video/-AwB-QcHIAU/v-deo.html