How to Validate Millions of Record in ETL testing?
Вставка
- Опубліковано 12 вер 2024
- How to Validate Millions of Record in ETL testing?
We have two different scenarios that I have explained where we have to validate the data -
1- When Source and Target both are in same server or database
2- When Source or Target may present in different Database or File or Azure NoSQL
#etltesting
#interviewquestions
Blog - www.testerhoon.com
Only thing i want to tell u ..i will wait for your videos.....and its worth of my time....thanks a lot for ur big help....
These are very imp questions....
Please dont stop.....people like me have life in this....
thank u so much for ur motivational words... it really help me to make new vdo... 🙏🙏😇
@@testerhoon28 mam please may I add subtitles in English
How to Validate Millions of Record in ETL testing
this is the only vido over YT which contains relevant explanation with respect to caption.
Hi Sissy.. Thank you very much.. For this video..As a new joiner for ETL testing.. Very useful for me...
thank you 🙏😊
lot of helpful video
request to pls ans for following
Request to Please provide answer for below scenarios frequently getting these queries @interview
*) how to validate flat file in landing area.
1.How to validate data and quality between landing and stage area
if source is flat file and stag is db with out converting flat file to any other format.
2.If json file has millions of record and this is source file
and target is oracle table then what is your approach to validate and how to validate
3.If oracle database is source and Teradata is Target and millions of record
then how to compare without using any automation tool and not with sample data, need to validate full load data
Yes, in this vdo i have explained that to validate the cross db data we have to use automation... bcoz it wont be possible to run sql query to validate 2 different server data..
Super mam.. I am wrkng as a qa engineer.. Its really helpful for me.. Plz do more videos..
thank you 🙏🙂
Same question asked in my recent interview but can't ,so can you elaborate 2 nd scenario in more detail with python
Thank you Mam... Really appreciate your efforts to share some real time scenario
okay 🙏
How to coordinate with dev Team in case there is not mapping document ?
Best video for etl testing in UA-cam
thank you so much 🙂🙏
@@testerhoon28 not able to open your web page
@@Falcon-dt6li try once - www.testerhoon.com
Hi Mam, it was really helpful, can you please make a video on ETL Testing using Automation using python or Java.
Grate one you are explained amazing
Wow... Was waiting for this
Thank you so much for vdo
Please make complete sql from beginning to advance level course
okay 👍
Ma'am please make videos on ETL testing using cloud - Azure, AWS, GCP.
Sure will make 👍
Thank you so much... explain very nicely thank you mam
🙏🙏✌😇
can you pls help how can we compare production data with test data,if not using any automated tool
Okay 👍
After getting defect we follow following procedure:
1.check is reproducible
2.email to developer
3.then respond defect resolution type
4.then we raise defect into hpal
Is this right process ?
I’m getting frustrated learning SQL. Can you tell me how to master SQL and which topics of SQL needed for a etl tester.
No need to get frustrated .. just focus on joins first, later learn subquery and some date, sub str functions... plssee my sql interview vdo and learn only those topics.. 👍
Thank u so much for good explanation
🙏
@@testerhoon28
Nice video thanks sis....
how to explain to interviewer this -- what is your testing approach if source have 1000 recods and out of this 900 loaded ??
Approach -
First validate all the 900 records are same in source or target.
Second, find the missing records and reason for not loaded in target...
Thank you sister 🙏
Hi, Can you tell me how to perform ETL testing on unstructured & semi-structured data
okay👍
mam can you please take practicle example of cross DB valiadtion
ok will make 👍
Thanks a lot for KT!!
thank u 🙏
Thanks a lot excellent explanation can you go over python basic for testers.
okay 👍
Suppose if you have 20GB text file data for the validation. How you are validated this data without using automation tool.
To test we have to do the sampling of data and test it
Thank you @@testerhoon28
Thanks a lot mam, can you please end to end automation video?
okay 🙏🙂
Thank you for your effort. Can you please explain in English or you can add captions under the videos.
okay
Thank you! Please share the sheet link python/c# sample code for cross db validation
okay will soon post the selenium code that used to validate the report... 🙏
please mam .. make video on how to do automation using python script like how to check null, count .it will be helpfull for us
okay 👍
Can we handle million of record if database is same ?
Sql server to sql server
Using SQL server
yes
and which is the commonly used automation tool for etl
You could use any automation tool, in my organization we have build c# script and python script to validate the data..
Can you make video on E2E flat file to database validation.Ifs very helpful for us.
i already made that vdo... pls chk my old vdos 🙏
Hi mam,tableau tool pe ek vedio banado.
okay 👍
Does except query is possible between SQL server and bigdata?
no
Can only count, duplicate and null validation done between SQL server and bigdata Hive?
please make brief video on different db validation
pls check out all my vdos.. 🙏
Can automation possible between Oracle and big data?
yes
Thank you
Can you explain about flat file to database .
I already cover that in my below vdo link -
ua-cam.com/video/Khds_7JJWZM/v-deo.html
In the second scenario can we use Informatica tool and validate the data?
yes u can...
Asked for me same question
How to get srs document or build from server?
How we can validate data from different Databases manually? (without python)
Manually we have to do the sampling of data and validate.... it wont be possible to test millions of data manually... 🙏
Mam please make video on end to end project on real time project
pls check out all my vdos.. 🙏
This is not enough for us we need more details video on cross db testing as well as we need test closure report
okay 👍
Can you please share your code How that works. That really helpful please madam share how that code is prepared
i have already shown that in my 2nd recent vdo.. pls chkout 🙏
can u provide the script mam?? please
for millions of records for cross db in excel what query we have to write for source and target ...it will take except ?
No, we could not use sql query in cross db... to overcome this issue, we have to use automation
@@testerhoon28 I am requesting you please make video on automation with end to end etl testing it will really help
Mam we really need if u can provide us the script
even if I provide you the script... you would not be able to execute... it needs some environment to run 🙏
Mam i dont waant to run ur script. I want to know the logic , will create my script.
I know we need environment to run all this.
@@testerhoon28
how to check this in mongo db
Kaha hai Automation madam..
wo is vdo me nahi hai... that vdo is yet to make.. 🙏
Mam can u give us to script which u prepared to validate the data..
so that we can replicate the thinngs@@testerhoon28
Mam pls csharp ka code explain karo na hoga to..
wo code nahi show kar sakti 😰
I have written code for automating count ,null,duplicate validation in python using pandas ...but the difference is that both the table present in same DB..
mam how to do Source target mapping in excel, can u help?
pls see me data validation vdo... their i have explained it... 🙏🙏
Plz explain 1 end to end project on etl testing
I had already explained pls go thru my end to end vdo... 🙏
can you please explain in english ? that will be really helpful please
okay
English or not English? I can't decide😮
Hi..thanks for sharing practical things like this can you share one more video to count check and minus check Null checks
If any error record comes then how to debug..
thank u for liking my content... i already made a vdo on that... pls chk my all end to end vdos..,🙏
@@testerhoon28
Can you please explain one or 2 business logics like direct mapping and derived mapping logic ../
How firm a script by those logic
Show practically plz....
Pls mak it in English r else try to give English caption
Sure 👍
Hi Mam, Please explain in English that would be great help😊
yes i have started making vdos in english only... thatwas an old vdo 🙂
Mam pls automation code show karo na
code mai show ni kar sakti... 🤐
Please do videos only in English
okay.. will try to add subtitle..
@@testerhoon28 no subtitles, please do videos in English only so that people can understand easily and improve their communication as well
mam, can u connect and help me if u have time i can pay u 😇. whenever u r free...its just excel sheet data source target mapping I,
pls give an indian time if u r free
it will not take more than an hour i guess pls
Hello, i have already made vdo on source to target data validation in excel... pls see that vdo.. 🙏
@@testerhoon28 mam ,
Mere palle nahi pad raha ... 😅
Pls
@@testerhoon28 plzzzzzzzzzzzzzzzzzzzzzzz
@@Dope007 sorry am not providing 1-1 support..
Hello mam I mail you few things to understand but didnt get any reply could you please let me know how can I contact you?
pls email me... iwill respond ..
@@testerhoon28.. Hi I mail kindly reply Ns thxs for the response
Hello I have few question related to etl I mailed you plz reply on that