I've used Azure Load Testing recently, it was a pretty straightforward, turnkey type experience. Combined with a browser extension to help construct a JMeter test based on my browser interactions, it was a very fast way to get some quality information about how the application stands up to load. A bit of feedback, I found that the web UI started to get a bit slow when the number of request in the JMeter script got into the dozens. For context, I configured a test that performance tested a user hitting multiple pages in my site, and simulating the load for every endpoint that was called (excluding cached requests for subsequent pages loaded). The test made approximately 100 requests. I'm guessing that the data for every request might have been returned to the UI even though the graphs probably couldn't process/display all of the data. Aside from some UI optimisation it's a really good service that simplifies the process of getting started with performance tests. I see the value and strength of this service in enabling discrete, targeted tests run as part of a CI/CD pipeline, plus the flexibility of enabling exploratory testing by engineers that do not necessarily specialise in JMeter or performance testing.
One question though, what's the maximum virtual user concurrency for the service? Where I work, we have some load tests that have 600k concurrent virtual users, where each virtual user's session lasts about 30-60 minutes. Would Azure Load testing be able to manage that? P.S. I've read that each engine instance is rated for 250 virtual users at most. So with the slider maxing out at 45 engine instances the max virtual users you could hope to simulate is theoretically 11,250. Is that true or is there some way around this number? E.g. if there is wait time in the JMeter script that emulates thinking/reading time for users, could we conceivably get more virtual users per engine instance?
Hi Tyson, thank you for the feedback! As you noted, Azure Load Testing currently supports up to 45 test engines in parallel, which ranges between 11,000-15,000 concurrent virtual users in each test, depending on the resource needs of the JMX script. (Note that this typically translates to a much higher traffic throughput - or requests per second (RPS) - since each virtual user can generate many requests.) We are working to increase this limit to 50,000 concurrent virtual users, and we’ll continue to raise this scale based on demand. The test results dashboard will also soon report load engine health, which will help users fine-tune JMX concurrency settings. If you need to simulate traffic at even higher loads, one workaround is to execute multiple load tests at the same time - feel free to engage with the product team at msft.it/6050wsc2Q for more guidance on this.
Hi Thanks for the video on Azure load test. You have uploaded the JMX file and completed the test. I have developed a framework with jmeter using bat scripting which has multiple VUs test like 100, 300, 500 1000VUs and lots of CSV test data. 1. Can I use multiple JMX files and run them one after the other in Azure load test? 2. How can we use the files like CSV or any other files like picture for multipart form data in Azure load test? 3. Can we see the HTML reports for the individual response time like in Jmeter HTML report? I have pulled all these metrics on Azure dashboard and can see all these metrics. When compared to Azure dashboard, what is more use of Azure load test? As both metrics are almost same.
It's cool but what about components of my App Service? For example if it uses OpenAI, how to load test it properly? Is there some "mocked" OpenAI models available for load testing? I can't test app with real connection to OpenAI it is too expensive
Hi, Very interesting Video, TNX! Questions, how do I use the Data files with JMX script Azure Load? Another word should it be uploaded in to special directory? Also how about specific controls ( *jar files) how Azure Load will handle it ? What about setting execution of loads on schedules? TNX!
Just started looking at Azure Load Testing after moving our application into an Azure environment. Great video, thanks!
I've used Azure Load Testing recently, it was a pretty straightforward, turnkey type experience. Combined with a browser extension to help construct a JMeter test based on my browser interactions, it was a very fast way to get some quality information about how the application stands up to load.
A bit of feedback, I found that the web UI started to get a bit slow when the number of request in the JMeter script got into the dozens. For context, I configured a test that performance tested a user hitting multiple pages in my site, and simulating the load for every endpoint that was called (excluding cached requests for subsequent pages loaded).
The test made approximately 100 requests. I'm guessing that the data for every request might have been returned to the UI even though the graphs probably couldn't process/display all of the data. Aside from some UI optimisation it's a really good service that simplifies the process of getting started with performance tests.
I see the value and strength of this service in enabling discrete, targeted tests run as part of a CI/CD pipeline, plus the flexibility of enabling exploratory testing by engineers that do not necessarily specialise in JMeter or performance testing.
One question though, what's the maximum virtual user concurrency for the service? Where I work, we have some load tests that have 600k concurrent virtual users, where each virtual user's session lasts about 30-60 minutes. Would Azure Load testing be able to manage that?
P.S. I've read that each engine instance is rated for 250 virtual users at most. So with the slider maxing out at 45 engine instances the max virtual users you could hope to simulate is theoretically 11,250. Is that true or is there some way around this number? E.g. if there is wait time in the JMeter script that emulates thinking/reading time for users, could we conceivably get more virtual users per engine instance?
Hi Tyson, thank you for the feedback! As you noted, Azure Load Testing currently supports up to 45 test engines in parallel, which ranges between 11,000-15,000 concurrent virtual users in each test, depending on the resource needs of the JMX script. (Note that this typically translates to a much higher traffic throughput - or requests per second (RPS) - since each virtual user can generate many requests.) We are working to increase this limit to 50,000 concurrent virtual users, and we’ll continue to raise this scale based on demand. The test results dashboard will also soon report load engine health, which will help users fine-tune JMX concurrency settings. If you need to simulate traffic at even higher loads, one workaround is to execute multiple load tests at the same time - feel free to engage with the product team at msft.it/6050wsc2Q for more guidance on this.
Exciting feature and awesome demo
Awesome! create a quick load test with just a few clicks ;)
Awesome video, thanks! Very informative 👏🏻
Hi Thanks for the video on Azure load test. You have uploaded the JMX file and completed the test. I have developed a framework with jmeter using bat scripting which has multiple VUs test like 100, 300, 500 1000VUs and lots of CSV test data.
1. Can I use multiple JMX files and run them one after the other in Azure load test?
2. How can we use the files like CSV or any other files like picture for multipart form data in Azure load test?
3. Can we see the HTML reports for the individual response time like in Jmeter HTML report?
I have pulled all these metrics on Azure dashboard and can see all these metrics. When compared to Azure dashboard, what is more use of Azure load test? As both metrics are almost same.
Skip to 3:16. You are welcome. Great demo!
It's cool but what about components of my App Service? For example if it uses OpenAI, how to load test it properly? Is there some "mocked" OpenAI models available for load testing? I can't test app with real connection to OpenAI it is too expensive
Hi, Very interesting Video, TNX! Questions, how do I use the Data files with JMX script Azure Load? Another word should it be uploaded in to special directory? Also how about specific controls ( *jar files) how Azure Load will handle it ? What about setting execution of loads on schedules? TNX!
Thanks for the good demo. Is just jMeter supported or is there any plan for other test frameworks? Currently it have to competes with k6.
Looks really good, will you be looking to support Gatling in the future ?
The is really fantastic. I am really excited to work on it going ahead should I get a chance. I simply love doing Performance Testing ☺️