Data Engineering Was Hard Until I Learned These 5 Secrets!
Вставка
- Опубліковано 21 лип 2024
- Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark: com.rpy.club/pdp/yYnEMzLOX?pl...
USE CODE: COMBO50 for 50% off
Learn more about courses here - datavidhya.com/courses
In this video, we will talk about the top 5 game-changing secrets that made data engineering much easier for me. Discover how mastering essential tools, optimizing workflows, and boosting productivity can transform your data engineering journey.
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / datawithdarshil
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
✨ Hashtags ✨
#dataengineer #coding #aws
1500 Likes and I will upload Fundamentals of Data Engineering Masterclass here
For free?? Darshil
1500 on the video? Because you are about there in one day or that comment? Y'all smash the like on both, we need our masterclass!
Can u make videos on these topics u have said so that we can understand properly.
@DarshilParmar done, we want the masterclass 🤝
0:23 1st mistake: understand the purpose of building the data pipeline, ask the why instead of straight dive into the technical details
2:02 2nd mastering sql is non negotiable
3:08 3rd automate everything
4:36 4th mistake: did not learn about debugging, maybe try Grafana / Prometheus, with an alerting mechanism
5:54 5th Learning is a continuous process
YOU VIDEOS ARE REALLY AMAZING!!
I never comment usually but your work is really inspiring.
Data always excites me and being a data research analyst now I wanted to shift to data engineering as I'm in love the work flow of it. Since I knew fundamentals in MYSQL and python for data analysis which includes fundamentals like loops, functions and conditional statements then forwarding to hands on numpy and pandas. Is it enough or is there something I should learn apart from this for data engineering ?
Best video editing
Thank you Darshil. How to learn about debugging?
Thank you for great contributions to the community and make me realise to contribute something to the community. So I started recently contributing to the community with SQL interview preparation, problem and solution series. Soon I will contribute many Thank you.
Really Helpful!!!
Thank you
Great video Darshil...If i carrier change to data engineer to learn is difficult to get job ?
Helpful
How to understand the flow of data for the specific project or work
It starts from asking questions to business people, what are they really expecting and do with the data, then from there you can start thinking about how to execute it
can you please share how you create these animations in your videos?
Thank you Brother!
You are welcome
Sir can u pls tell me that whether Information and Data Security subject or Essentials of Data Collection and Ethics subject will be the better option for data engineering career. This is an elective subject in my 3rd sem
Bro how can we create a code snippets like you showed in thumbnail? Which tool it is I saw many people use this to post their code on LinkedIn also, That one with 3 color dots on top left?
Just write code to image on google
@@DarshilParmar thank you so much bro!
Hey mate , great video. Thank you. Would you let me know what software do you use for video editing
I used to use FinalCutPro but now I have an editor who edits my videos
Hi bhaiya
Needed a laptop in college and programming i started learning data science and ml but dont do gaming so i didnt take any GPU laptop
I have 2 options
First lenovo ideapad pro 5 with 14" ultra 9 H processor 32 gb ram 1 tb ssd and 84 wh battery
Second is lenovo yoga slim 7i with ultra 7 processor 32 gb ram 1 tb ssd and 65 wh battery
Both price same
Which one should i go for?
Can you please upload some thorough projects along with the business mindset behind them and also how we should build further projects
I have lot of them on my UA-cam - check UA-cam analytics
Plz ,anyone can tell me which CSE specialization should i choose for data engineering ??
Could you please make a video on Debugging concepts, especially w.r.t. data engineering?
Will do
@DarshilParmar ... Is data Engineering is a suitable for someone who is love coding very much and don't like work with visualization stuff like dashboard, who like to focus on distributed systems, scalablility performanece and need to heavy logic thinking to do
Hi Darshil, I have some experience on Azure adf. This combo will help me to go as a azure data engineer or only for AWS engineers.
Have you used Azure cloud in this combo ?
Yes, we have project in Spark course on Azure
@@DarshilParmar what about pipeline implementation by using Azure ?
Yes, it's building data pipeline only!
I am struggling to become a Data Engineer. In few months I will write my Bachelor thesis and it should be about data engineering. I hope with this thesis i will be able to enter the job market. Any recommendations for a thesis to enter the market?
I'd say learn high-demand tools, try to get any job in data, doesn't have to be DE only and make your way from there
Darshil now that you have more than 1.5k likes
Could you please upload the data engineering fundamentals masterclass
I need to build a PPT, record, and edit video, it takes time! will upload once it is done
Bro in second part of youtube analysis video s3 trigger is not working properly and the data(parquet files) is not getting uploaded and no graph is visible
Check IAM permission
@@DarshilParmar i checked can it be related to the thing that data types have to be same when creating etl jobs, interface has changed completly,kindly help
Which tool are you using to create architecture diagrams?
Google Slides
Waiting for your Fundamentals of Data Engineering Masterclass 💙
Coming soon
❤
Is dsa compulsary to crack an product based company , if yes then how much , any suggestions from where to get this skill set
I have a video on this ua-cam.com/video/gAk9m_d27yk/v-deo.htmlsi=fJOrV6Gzxa8Ecyov
I had created an account on 24 June,
From today total 8 days are remaining I.e till 25 july for my free trial period.
But it gets deactivated and asking my credit card details, WILL THEY CHARGE ME ??
I completed my b. Com computers and later i have competed my masters in marketing domain. Is am a eligible for a data engineer
I just Googled that how to like a video 1500 times with single account(Unfortunately No option), Please upload Master class
Will do it soon
Bhaiya how it is data engineering professional certificate by ibm on coursera
I suggest you to do GCP one
@@DarshilParmar and for data analytics and data science how is ibm data analytics and data science course?
Sir, I'm a third year cse student. I want to be a data engineer and could you please suggest resources for that. And, do freshers get job in this field?
Check out my roadmap
Hi bhaiya
Needed a laptop in college
I chose 2 options hp pavilion plus 16gb ram fixed 1 tb ram fixed i7 13700h rtx3050 6gb 1.01 lakh
Ideapad pro 5 16gb ram fixed 1 tb ssd ultra 7 155H rtx 3050 6gb 1.13 lakh
Which one should i prefer and if i want to go more low which can i take
Don't go HP
@@DarshilParmar bhaiya actually i am very confused on which laptop to take and dont want to overspend
I dont do machine learning now but may do so i dont know if i should go for a dedicated graphics card and which processor to go i7 or ultra 7 and ram
Can you please suggest me which configuration should i go for and what price range
I am getting lenovo ideapad same laptop with 13700h online for 94k but parents are saying not to buy online so i cant decide what to do but i need a laptop now
@@lectures3727 If you play games then graphic card is good else just go for other option
Basic config - 16GB ram, 500-1TB SSD is good, any latest processor.
If you can afford maybe try Microsoft surface or macbook
@@DarshilParmar no i dont play games
Can i go for any latest processor
I am learning data science machine learning now
Will that laptop be enough if i take ultra 7 or i7
I am mostly getting p type processor
Can a person from outside india . Can take data vidya combo course
Yes you can - datavidhya.com/combo-course-info
@@DarshilParmar and the subscription is for life time
@@hemanthvarupula9873 Yes
can we get more discount on your combo course for limited period like it is still expensive for students who are just starting off in the field , hope you understand and you'll do consider it
I have 50% offer going on, if you want you can start with one course first and then go one by one
@@DarshilParmarI was trying to buy your sql course but there is no discount even the coupon code is also not valid
Is there any other way around ?
@@avinashvyas1597
Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark: com.rpy.club/pdp/yYnEMzLOX?plan=6607b619c69cf00b7b934479
USE CODE: COMBO50 for 50% off
Purchase Individual Courses with 50% off:
1. Python for Data Engineering - pay.datavidhya.com/cop/UHfHDhbKVS?code=PYT50
2. SQL for Data Engineering - pay.datavidhya.com/cop/x4UpBcaKsi?code=SQLT50
3. Data Warehouse for Data Engineering with Snowflake - pay.datavidhya.com/cop/BpfGFSahpQ?code=DWT50
4. Enroll in the Apache Spark Course Here - datavidhya.com/courses/apache
USE CODE: EARLYSPARK for 50% off
@@DarshilParmar Thank you Darshil
Also you’ll be mentoring me now is that correct ?
@@avinashvyas1597 There are no 1 to 1 session but we have dedicated discord for this so if you have questions I will help you out
did you had a knowledge on data engineering when you started your internship
I had knowledge about ML and Data Science but then I learned DE on internship
Kya thi cho bhaiband?
Please provide discount in your course
. please provide an email id or whatsapp to connect with you
50% offer is going on, please check description