Thank you for watching! Did you see your questions? Have anymore? Drop your thoughts below! Preplaced Mentoring platform: visit.preplaced.in/4ia Use code: JASH500 for additional discounts
I can relate to zero ETL because I am using Azure Data Factory(pipelines and dataflows) and you should know all techniques of tool to convert SQL into dataflow. Wonderful video. Kudos 😇😇
I recently started a new job as a senior data engineer at a new company. As a fresher in this field, I am excited to have discovered your channel and am looking forward to finding valuable content and resources that will help me improve and grow in my career. I hope to take advantage of any opportunities that come my way as I begin this new chapter in my professional journey.
I was happy to see SS of my question here, and thanks a lot for mentoring throughout. Hope to see more insightful stuff from you on data engineering. Thanks for being my inspiration.
Im an informtica developer with 6 yaers experince... Im in the middle way of using aws... Should i get into complete data engineering tech or be at informtica and iics
Hi jash from where i can preapre for data enginner system design interview?? all the material or tutorial is avialble only for software enginner role only, please help, i mostly reject in system design round
Thanks for answering my question . I just got really happy 😊 after seeing my question in the video no one has ever given such a value to my doubts. I hope you will collect this compliment in your folder as your mother told you to do and I know you will get many from others.keep them. you are doing an extrodinary work. I want to ask one more question How to develop strong IT fundamentals?do you recommend any course or youtube channel?
Great to be able to answer your question in the video! For IT fundamentals, I don't know any courses or videos but this one looks like a good starting point. ua-cam.com/video/625zZYlZHoA/v-deo.html You ca find out more on UA-cam too. No need to go for a paid course for this
That video is great. So, I am an Oracle DBA. I have some hands on MongoDB also. I work on Linux. I have BCA Degree. How much work will be required for me to switch to the Data Engineering?? or Is it possible to switch from DBA with BCA Degree?
You are going with python but if the given choice would you consider JavaScript because node.js is also famous among developers for backend. We can also become full stack with JavaScript....asking from a freshers perspective..😅
Sir can u make a detailed video on how a end to end pipeline looks in google, I am a fresher coming from different background i have followed ur road map now what kind of projects i can add in my resume and where could i find such projects
Hi Soham, thanks for the suggestion. I will try to make a video on this 😊 and for sample data pipelines, checkout my video on system design (2nd part) and website called projectpro.
As you mentioned that DE should have knowledge of Linux commands, how do I learn linux like should I just learn the commands or use it with the projects after learning like if I had to do that I have to install all my platform(DB, Python, cloud, Spark,etc) in Linux because I have Windows OS. Also before learning Linux should I install Virtual box for Linux or go for dual boot ubantu. Please suggest if you have something better because I want to sail smoothly. Also which DB you use I picked Postgresql for me Thank you for these videos!
To learn Linux commands, checkout a course I have mentioned in roadmap video and for where to practice them, create a VM in cloud like GCP. They come in free tier so you won't have to pay for them as long as you use a small one. This VM can be Linux based. For DB, postgres is great. Other alternative is MySQL. But postgres is more widely used.
Hi Jash Radia, I am From Nepal 🇳🇵. A 19 y/o boy having a curiosity to learn Data Science. Can you make a video on the courses that are available online for data science step-by-step. If you can !! It means a lot to us ❤
Pyspark is my preference. Scala is faster since that is spark native but with spark 3.0, pyspark is closing the gap too. Apart from that Python has better and more data connectors, better data libraries and also wider use case even outside of Spark.
Im a fresher joined as application developer but anyway im doing oracle dba job so I thought of moving towards data engineering field in the future days. So is it managable transition from dba to data engineer?
Hey Jash! Thank you for all that you do for us budding data engineers. I recently dropped out of College. If I learn following your roadmap and do the projects, how much of a difference does a College Degree make in getting that first job? Also, please share anything you think can help a College Dropout along the journey of data engineering. I would appreciate your guidance. Thank You Again!
College degree makes a difference to get your first job. Getting it will be harder. But start with internships, prove your worth, get a job. After 1-2 years of work experience. It won't matter
Hello bhaiya hope you reply to this. So I have joined my company in 2021 as a fresher now its about to be 2 yrs and i have been stuck in QA since then i was from IT background so I was quite depressed about getting in QA but I tried it but i know that it's not for me. I have tried talking to managers and all but internal switch seems difficult. I am also an indecisive person I am confused between front end dev and data engineering as my career for the long run i like both of them so can you please guide me what should I do abt the future of both in comparison which would be better and how to get job as data engg as my qa exp wouldn't have been helpful In switching. Thank you
You have to upskill yourself for both. Either dev or data engineering. But before that, you must make a decision on your own. Do you like creating something useful out of data more or developing new applications? Accordingly go to any one field. After that, focus on upskilling in that field for at least 4-6 months before you start applying for it.
@JashRadia since you have worked in Zs associates. They have roles like Decision Analytics Associate. Can someone from this role can move into the roles of data engineer in Zs Associates I have been selected as a DAA and i am from a non cs background. 🙏💕
Hi Vibhor, I did answer your question in the video while shooting but then had to remove some questions while editing to not make the video too long. I would say the first step for you would be to talk with your PD coach and tell them your goal. They will provide you a PoC for someone who is in data engineering that can guide you. After this, you can talk to your manager. The process can take about 6 months. Another option is, stay as a DAA officially but if your manager agrees, then work on DE related projects in a more technical role so that you can get that exposure. In your resume also you can mention DE if you're working on such projects
You're welcome! Just learn basics of map reduce and why spark is better than that. No need to learn Hive. No SQL is a separate topic. It is good to have as a bonus but not always a core skill for DE. Focus on core skills first
Hi Jash, Good afternoon, I am backend api developer with 7 + exp working on sap technology. For swtiching to DE roll, pyspark + Azure Databricks, certification on Databricks + training will help me to go into Data engineer?
Hi Ganesh, yes learning spark and databricks can be helpful. Spark databricks certificate will also be good. However, do practice python, SQL and cloud skills too before interviews.
Please make roadmap for data science. There are not good videos about data science roadmap. I like your roadmap for data engineer, but I want to become data scientist, can you make roadmap for it?
I am a civil enginner currently learning data analyst skills for entry level jobs. And i wanted to know which masters degree should i take in data engineering. Like, can you suggest me any good courses
Hello bro, I am BCA graduate .. and wants to pursue my MCA through IGNOU so i could work and study simultaneously .. So my question is ... are open university degrees accepted in product based companies if i want to be a data engineer ?
Yes! Companies don't care about your degrees. They care about skills. However if you have relevant experience, then it becomes easy for you. So start with any company that can give you DE job then switch to big product based companies.
Hii I am working as a developer and preparing for switch into data engineer domain completed sql,python Need to study but my problem is how to manage both job and upskilling. As a developer also I don’t know much to write code also I struggle lot to solve issues so is there much coding required for data engineer or data analyst role pls tell
Hi Jash, I have 6 year of experience in java as a developer. Recently I joined as a data engineer. I wanted to know how this switch help me to grow my career in terms of salary and learning. And I do love coding. Is this switch good for me ?
Just pay for Gemini Ultra and let him guide you when you look at real requirements in ETL etc. Let Gemini give you the outlines and ideas about what’s useful for data engineering tasks. Stay focused and forget about the old FANGG rulebooks.
Hi jash thanks for this video this video clear most of the doubt i have one doubt I have expertise as a QA engineer and want to change careers to the field of data engineering. However, most employers have a three-month notice period, and the majority of employers want rapid hiring. How can I overcome with this issue? Your advice will be valuable.
There will be some companies who are okay with 3 months notice period if you get an offer even though it is not attractive, I would suggest you to accept it. Then in 3 months you can easily apply and get more offers in your NP
Thank you for watching! Did you see your questions? Have anymore? Drop your thoughts below!
Preplaced Mentoring platform: visit.preplaced.in/4ia Use code: JASH500 for additional discounts
I can relate to zero ETL because I am using Azure Data Factory(pipelines and dataflows) and you should know all techniques of tool to convert SQL into dataflow. Wonderful video. Kudos 😇😇
True and thank you! 😊
Thanks Jash for answering these questions...!
Most welcome!
Learned a lot from here, wonderful insight. Thanks!
Glad to know! 😁
I recently started a new job as a senior data engineer at a new company. As a fresher in this field, I am excited to have discovered your channel and am looking forward to finding valuable content and resources that will help me improve and grow in my career.
I hope to take advantage of any opportunities that come my way as I begin this new chapter in my professional journey.
All the best.. keep going 🎉🎉
@@nandesh783 Thank you ✨
Wish you all the very best! 😊
@@JashRadia Thank you Jash😊
Super insightful Jash 🔥🔥
Thanks! 😊
amazing video brother
Thank you! 😊
Thanks for answering my question.
No problem!
I was happy to see SS of my question here, and thanks a lot for mentoring throughout. Hope to see more insightful stuff from you on data engineering. Thanks for being my inspiration.
Thanks Chintan 😊
Hi Jash, I am a liverpool fan. Great content, helps me a lot in my career. #YNWA
Hahaha beside data you are a laughter engineer too😂 great content keep it up🤗
Thanks bro! 😂
What would you advise a person from networking background to get into data engineering with zero programming knowledge?
I would say start with IT fundamentals. Learning OS concepts and then go to programming by learning a simple language like Python!
Im an informtica developer with 6 yaers experince... Im in the middle way of using aws... Should i get into complete data engineering tech or be at informtica and iics
Hi jash from where i can preapre for data enginner system design interview?? all the material or tutorial is avialble only for software enginner role only, please help, i mostly reject in system design round
Thanks for answering my question . I just got really happy 😊 after seeing my question in the video no one has ever given such a value to my doubts.
I hope you will collect this compliment in your folder as your mother told you to do and I know you will get many from others.keep them. you are doing an extrodinary work.
I want to ask one more question
How to develop strong IT fundamentals?do you recommend any course or youtube channel?
Great to be able to answer your question in the video! For IT fundamentals, I don't know any courses or videos but this one looks like a good starting point.
ua-cam.com/video/625zZYlZHoA/v-deo.html
You ca find out more on UA-cam too. No need to go for a paid course for this
@@JashRadia thank you so much 😊. Iam Going to watch that video .thanks for making time to give reply to every question
@@vanka.suneel it's a pleasure!
That video is great.
So, I am an Oracle DBA. I have some hands on MongoDB also.
I work on Linux. I have BCA Degree.
How much work will be required for me to switch to the Data Engineering??
or Is it possible to switch from DBA with BCA Degree?
Thanks and it can take about 4 months with the skills you mentioned you already have. And yes it is possible
You are going with python but if the given choice would you consider JavaScript because node.js is also famous among developers for backend. We can also become full stack with JavaScript....asking from a freshers perspective..😅
Js is great! But more for SWEs and not so much for DEs. For DEs Python and Java or Scala is mostly preferred because of data intensive usecases.
I also think rust might become important in the future. Libraries like polars are absolutely amazing!
Sir can u make a detailed video on how a end to end pipeline looks in google, I am a fresher coming from different background i have followed ur road map now what kind of projects i can add in my resume and where could i find such projects
Hi Soham, thanks for the suggestion. I will try to make a video on this 😊 and for sample data pipelines, checkout my video on system design (2nd part) and website called projectpro.
As you mentioned that DE should have knowledge of Linux commands, how do I learn linux like should I just learn the commands or use it with the projects after learning like if I had to do that I have to install all my platform(DB, Python, cloud, Spark,etc) in Linux because I have Windows OS.
Also before learning Linux should I install Virtual box for Linux or go for dual boot ubantu. Please suggest if you have something better because I want to sail smoothly.
Also which DB you use I picked Postgresql for me
Thank you for these videos!
To learn Linux commands, checkout a course I have mentioned in roadmap video and for where to practice them, create a VM in cloud like GCP. They come in free tier so you won't have to pay for them as long as you use a small one. This VM can be Linux based.
For DB, postgres is great. Other alternative is MySQL. But postgres is more widely used.
@@JashRadia Thank you very much! Also, have a great weekend ahead :D
@@AltafKhan-ka1zer 🍻
Hi Jash Radia, I am From Nepal 🇳🇵. A 19 y/o boy having a curiosity to learn Data Science. Can you make a video on the courses that are available online for data science step-by-step. If you can !! It means a lot to us ❤
Hi ,
can i ask you to talk about the technologies used ?
It good to move in data engineering after 8 years of experience as PHP developer
Yes! 2023 is as good a year as any
No
Which is more preferred
Azure data engineer or data engineer
Azure data engineer is just a subset of data engineer.
Please answer this question... which one should I choose, spark with scala or pyspark
Pyspark is my preference. Scala is faster since that is spark native but with spark 3.0, pyspark is closing the gap too.
Apart from that Python has better and more data connectors, better data libraries and also wider use case even outside of Spark.
Im a fresher joined as application developer but anyway im doing oracle dba job so I thought of moving towards data engineering field in the future days. So is it managable transition from dba to data engineer?
Hi Sir, I'm working as a informatica (ETL) Developer. Looking to work on Big Data.
Not sure what strategy to learn the Big Data
Checkout my roadmap on the channel. If you have any specific question, then let me know.
@@JashRadia Okay Sir
Hey Jash!
Thank you for all that you do for us budding data engineers.
I recently dropped out of College.
If I learn following your roadmap and do the projects, how much of a difference does a College Degree make in getting that first job?
Also, please share anything you think can help a College Dropout along the journey of data engineering.
I would appreciate your guidance.
Thank You Again!
College degree makes a difference to get your first job. Getting it will be harder. But start with internships, prove your worth, get a job.
After 1-2 years of work experience. It won't matter
Bro, if you really dropped out of college, I wanna have a chat with you !!
Hello bhaiya hope you reply to this. So I have joined my company in 2021 as a fresher now its about to be 2 yrs and i have been stuck in QA since then i was from IT background so I was quite depressed about getting in QA but I tried it but i know that it's not for me. I have tried talking to managers and all but internal switch seems difficult. I am also an indecisive person I am confused between front end dev and data engineering as my career for the long run i like both of them so can you please guide me what should I do abt the future of both in comparison which would be better and how to get job as data engg as my qa exp wouldn't have been helpful In switching.
Thank you
You have to upskill yourself for both. Either dev or data engineering. But before that, you must make a decision on your own. Do you like creating something useful out of data more or developing new applications? Accordingly go to any one field. After that, focus on upskilling in that field for at least 4-6 months before you start applying for it.
@JashRadia since you have worked in Zs associates. They have roles like Decision Analytics Associate. Can someone from this role can move into the roles of data engineer in Zs Associates
I have been selected as a DAA and i am from a non cs background.
🙏💕
Hi Vibhor, I did answer your question in the video while shooting but then had to remove some questions while editing to not make the video too long.
I would say the first step for you would be to talk with your PD coach and tell them your goal. They will provide you a PoC for someone who is in data engineering that can guide you. After this, you can talk to your manager. The process can take about 6 months.
Another option is, stay as a DAA officially but if your manager agrees, then work on DE related projects in a more technical role so that you can get that exposure. In your resume also you can mention DE if you're working on such projects
After securing certificate what will be the next step to sharpen your skills.
Then apply the skills you gained from certification. Create a complex data processing project that is highly automated and config driven!
Thanks for answering the Question.
I have one more doubt is this really needed to learn hive, no- sql, map reduce concepts before learning pyspark?
You're welcome! Just learn basics of map reduce and why spark is better than that. No need to learn Hive.
No SQL is a separate topic. It is good to have as a bonus but not always a core skill for DE. Focus on core skills first
Hi Jash, Good afternoon, I am backend api developer with 7 + exp working on sap technology. For swtiching to DE roll, pyspark + Azure Databricks, certification on Databricks + training will help me to go into Data engineer?
Hi Ganesh, yes learning spark and databricks can be helpful. Spark databricks certificate will also be good. However, do practice python, SQL and cloud skills too before interviews.
Please make roadmap for data science. There are not good videos about data science roadmap.
I like your roadmap for data engineer, but I want to become data scientist, can you make roadmap for it?
I will see if I can collaborate with someone to make this. I have a few days scientist friends
I am a civil enginner currently learning data analyst skills for entry level jobs. And i wanted to know which masters degree should i take in data engineering. Like, can you suggest me any good courses
Check this out: bit.ly/3VYSCpf
@@JashRadia How are IIT madras, IIT banglore, BITS Pilani courses?
@@taranggourshettiwar672 I am not sure. Haven't taken them.
@@JashRadia Ok, Thankyou for your help😄
@@taranggourshettiwar672 no problem!
Hello bro, I am BCA graduate .. and wants to pursue my MCA through IGNOU so i could work and study simultaneously .. So my question is ... are open university degrees accepted in product based companies if i want to be a data engineer ?
Yes! Companies don't care about your degrees. They care about skills. However if you have relevant experience, then it becomes easy for you. So start with any company that can give you DE job then switch to big product based companies.
Hii
I am working as a developer and preparing for switch into data engineer domain completed sql,python
Need to study but my problem is how to manage both job and upskilling. As a developer also I don’t know much to write code also I struggle lot to solve issues so is there much coding required for data engineer or data analyst role pls tell
Data engineering role requires more coding than analyst role. You can gain DSA skills while studying for 1-2 hours a day.
Dude is their any difference between an azure data engineer and data engineer?
Azure data engineer means the person is only working in azure. That's the difference
road map and resources &blogs for dataengineer
I already have a roadmap on the channel. There is nothing more to add to it right now.
resume and interview preparation
Hi Jash, I have 6 year of experience in java as a developer. Recently I joined as a data engineer. I wanted to know how this switch help me to grow my career in terms of salary and learning. And I do love coding. Is this switch good for me ?
How was your experience in data engineering? And since you loved coding , are you enjoying data engineering or not?
Can i use java instead of python?
Yes, completely fine as I answered in my last question.
Dependency on other teams..really sucks😑
Ikr! 🤦♂️
Just pay for Gemini Ultra and let him guide you when you look at real requirements in ETL etc. Let Gemini give you the outlines and ideas about what’s useful for data engineering tasks. Stay focused and forget about the old FANGG rulebooks.
Yes, this is what I recommend when someone is interested in building a project. It's very easy to do wjth LLMs now
Hi jash thanks for this video this video clear most of the doubt
i have one doubt I have expertise as a QA engineer and want to change careers to the field of data engineering. However, most employers have a three-month notice period, and the majority of employers want rapid hiring. How can I overcome with this issue?
Your advice will be valuable.
There will be some companies who are okay with 3 months notice period if you get an offer even though it is not attractive, I would suggest you to accept it. Then in 3 months you can easily apply and get more offers in your NP
YNWA sir
What an irony and we developers are dependant on Data engineer for data 😂
Hello Jash. Please date me.
Sorry, I don't do dates. They are very confusing. Sometimes come in DDMMYYYY and sometimes MMDDYYYY 😂
@@JashRadia Guy chose technical answer 😂
@@JashRadia NaileD it 😆😆💪