Performance Testing: Tool
— Tool should suport the protocols that your application supports at frontend (http, https etc.) or at the backend (AJAX etc).
— Record & replay Scripts:
— How does it record?
— How does it Replay?
— Modularizing the script?
— Enhancing the Scripts : adding think times and pace times.
— Can you replay scripts on multiple browsers?
— What language does the tool support?
— What kind of detailing/breakdown you expect from the reporting (reports, graphs etc.)
— Licensing Model?
Performance Testing Tool Architecture
Automated performance test tools typically have the following components.
- Recording of end-user activity.
- Modification of the recorded scripts to associate internal/external data
Test management module
- Creation and execution of load test sessions or scenarios to represent different
mixes of end-user activity.
- Generates the load and enables creating various injection profiles.
- Ability to analyze the data collected from each test execution. This data is
typically a mixture of autogenerated reports and graphical or tabular reporting. T
Performance Testing Environment
It’s ideal to have a replica of production environment as performance testing environment. But its not generally possible in real world and setting up a performance testing environment often ends up being the most challenging task.
There are lot of factors behind this, the most common ones of which are:
- Number of servers : Due to complexity and cost factor, it is not possible to have exact number of servers in a performance testing environment. If number of servers cannot be replicated at each tier, it is best to match the specification of live servers. This way, capacity of one server can be determined which can be used as a reliable baseline for horizontal scaling.
- Bandwidth and connectivity of network infrastructure: Target servers generally are not deployed in same location as their live counterparts but they can still share the same network infrastructure.
- Number of application tiers: Performance tests are more realistic if the server tier configuration for the performance testing environment is as close as possible to the live environment. For example if there is a proxy server lies between user and application server, it should be the same for the performance testing environment.
- Application DB Size: There should not be a lot of discrepancy between the size of the DB for production and the performance testing environment. It doesn’t make make sense to run performance tests against a 1GB DB if the production has 50 GB Database. Apart from size, content of DB also impact the response time for queries, hence that should also ideally match the production counterpart.
In a nutshell, A subset of the live environment with fewer servers but specification and tier-deployment matches to that of the live environment is the best bargain for performance test environment if an exact replica is not possible.
Performance testing can be done on live environment if the impact on real user traffic is taken care of. For example, some companies do performance testing in out-of-normal hours.
Injection profiles define the way users are introduced into the application. Different tools provide different types of injection profiles.
- Big Bang: All virtual users start at the same time. Should be used carefully because if too many users are introduced suddenly, it might bring down the system completely even before the actual performance tests run.
- Ramp Up: This profile starts with some users initially and then users are added at specified intervals until a target number of users is reached.
- Ramp up with Step: Idea is to reach a specified number of target users but injection is paused at certain intervals. For e.g. target number of users may be 1000 and injection might be paused at 250, 500 and 750 users to study application performance at these numbers.
- Ramp Up (with step), Ramp Down (with step): Ramp up to a certain number of users (with or without step) and ramp down to certain number of users (with or without step).
Things to remember:
- Data Rollback: When you run performance tests, the DB should be rolled back to initial state, which is an initial known state. This is done so that it is easy to compare the results.
- No other testing should be performed while doing