I have 2 procedures in the Postgres database that take 2 hours each to complete, but I want to run them parallelly without affecting any kind of db performance so that they both can finish in 2 hours and they are not dependent on each other. I have 4 cores allocated to that server. if I run multiple procedures through concurrent.futures, will it be able to complete it in 2 hours? Which will be helpful multi threading or processing?
Hi Sir, I have a question for you based on your comparison of multithreading and multi processing. So, As GIL applies during multithreading, we can never use more than 1 core no matter how many threads we create and run? Am I right?
Sir, Why output is not showing in jupyter notebook ?? When I use mp.Process() my code runs but I am unable to see my output. I want to know why this problem occurs in jupyter notebook. It would be absolutely great if you could help me over that. 🙏
Hello, i have one case can you help me with that , i get near about 10k XML files(in each xml data is so much , many tags present ) per hour and I want to read all XML files, get data from them and also perform some mathematical calculations on some XML tag values and insert data into database and also make csv file for each xml files , can you please help that , which approach i take for this scenario , becuase of i am getting huge amount of files in per minute with large amount of data in every xml
Hi Thanks for your comment.. it’s difficult to suggest anything based on the limited information. I am providing my suggestion with a disclaimer that I don’t know about your development environment and system, you need to evaluate my suggestion before implementing From the look of it ..you can use parallel processing. If you’re using python you can use multiprocessing for the same since you are also doing mathematical calculations. Hope this will help Also I will recommend to use stack overflow for these kind of questions.. Thanks 🙏
I have 2 procedures in the Postgres database that take 2 hours each to complete, but I want to run them parallelly without affecting any kind of db performance so that they both can finish in 2 hours and they are not dependent on each other. I have 4 cores allocated to that server. if I run multiple procedures through concurrent.futures, will it be able to complete it in 2 hours?
Which will be helpful multi threading or processing?
Hi Sir, I have a question for you based on your comparison of multithreading and multi processing.
So, As GIL applies during multithreading, we can never use more than 1 core no matter how many threads we create and run?
Am I right?
Thanks for your comment
At any point of time - yes.. however for IO bound applications you can still use multi threading
Sir, Why output is not showing in jupyter notebook ?? When I use mp.Process() my code runs but I am unable to see my output. I want to know why this problem occurs in jupyter notebook. It would be absolutely great if you could help me over that. 🙏
Thanks for your comment..see at time 4:40 onwards in the above video.. I have explained the same..hope it helps thanks 🙏🙏🙏
Hello, i have one case can you help me with that , i get near about 10k XML files(in each xml data is so much , many tags present ) per hour and I want to read all XML files, get data from them and also perform some mathematical calculations on some XML tag values and insert data into database and also make csv file for each xml files , can you please help that , which approach i take for this scenario , becuase of i am getting huge amount of files in per minute with large amount of data in every xml
Hi
Thanks for your comment.. it’s difficult to suggest anything based on the limited information.
I am providing my suggestion with a disclaimer that I don’t know about your development environment and system, you need to evaluate my suggestion before implementing
From the look of it ..you can use parallel processing. If you’re using python you can use multiprocessing for the same since you are also doing mathematical calculations.
Hope this will help
Also I will recommend to use stack overflow for these kind of questions..
Thanks 🙏
@@Cognitive-Programmer Thanks