ETL | AWS Glue | AWS S3 | Data Cleansing | Transforming data with AWS Glue in ETL workflows
Вставка
- Опубліковано 18 січ 2024
- ===================================================================
1. SUBSCRIBE FOR MORE LEARNING :
/ @cloudquicklabs
===================================================================
2. CLOUD QUICK LABS - CHANNEL MEMBERSHIP FOR MORE BENEFITS :
/ @cloudquicklabs
===================================================================
3. BUY ME A COFFEE AS A TOKEN OF APPRECIATION :
www.buymeacoffee.com/cloudqui...
===================================================================
In this lab you will learn about AWS Glue, which is a serverless data integration service that makes it easier to discover, prepare, move, and integrate data from multiple sources for analytics, machine learning (ML), and application development. You can use a crawler to populate the AWS Glue Data Catalog with tables. This is the primary method used by most AWS Glue users. A crawler can crawl multiple data stores in a single run. Upon completion, the crawler creates or updates one or more tables in your Data Catalog. Extract, transform, and load (ETL) jobs that you define in AWS Glue use these Data Catalog tables as sources and targets. The ETL job reads from and writes to the data stores that are specified in the source and target Data Catalog tables.
example data : github.com/RekhuGopal/PythonH...
#awsglue #etl #datatransformation #s3 #cloudquicklabs #crawler #etljob
#dataengineering #data #datacleansing #datamodificada - Наука та технологія
Excellent video, please make more videos like that on ETL, admire your hard work, very smooth explanation…… Thanks
Thank you for watching my videos.
Indeed , I shall make videos on data engineering.
Glad that my videos helped you.
Excellent brother❤
Thank you for watching my videos.
Glad that it helped you.
Loved your series on AWS Glue.. nicely explained.. 😊 Please upload more scenarios with other transformations such as join and filter !
Subscribed already.. thank you !!
Thank you for watching my videos.
Glad that it helped you.
Indeed I shall make more videos around this topic ahead.
excellent
Thank you for watching my videos.
Glad that it helped you.
Thanks a lot....please make more videos with different transformation and also the use of cloudwatch and lambda with it....
Thank you for watching my videos.
Glad that it helper you.
I shall make more videos around this topic and Thank you for giving inputs.
It is showing failed at the end after running the job ...can you make a video on how to debug this issue
Thank you for watching my videos.
You can find ETL pipeline logs if you go pipeline runs or you can also find the same at Cloudwatch log groups.
How can I run multiple glue jobs.
Thank you for watching my videos.
Do you mean creating multiple job or multi threading within a job ?
Bro You create Glue job by using python script...... In real time..... Visual ETL is Never preferred......
Thank you for watching my videos.
Indeed , I would creating videos on creating job via python script. But this video is to focus on ETL topic.