Performance tests are executed to determine how a system behaves in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.
To alleviate the risk of degrading server performance for customers’ applications which run on shared ITD infrastructure, all web applications (ITD written and vendor) must be performance tested prior to production use. ITD manages a testing environment which enables our staff to test web applications to ensure they perform efficiently. A performance test is executed prior to an application’s initial production deployment and before a modified application is re-deployed to production. Cosmetic changes are exempt from the performance test requirement.
The rates for Performance Testing of applications are variable and will be based on the number of estimated users for the application. The costs of performance testing are used to cover a portion of the license and support costs for the testing tool.
Key Performance Indicators or KPIs are quantifiable measures used by an organization to evaluate its success or the success of a particular activity. For Performance Testing, KPIs are used to ensure efficient server performance.
If minimum KPIs are not met, a dedicated environment may be required which will result in additional costs to the agency.
|Application||Key Performance Indicator||Target||Description||Comment|
|Any Application||HTTP Response Code||2XX and 3XX||Acceptable||
Any response codes beginning with a 2 and 3 are acceptable. For example, 200 and 300 are acceptable.
Any response codes beginning with a 4 or 5 are unacceptable. For example, 401 - Unauthorized, 403 – Security, 404 - Missing images/includes, 407 - Proxy authentication, 500 - Server down, 502 - Proxy errors, 503 - Service Unavailable
|Any Application||End User Response Time||<= 1.5 sec/action step||Response times > 1.5 seconds/action step need to be investigated.||Known to have above average response times: File uploads and downloads, Cognos and Crystal report generation, and GIS.|
|Database||Key Performance Indicator||Target||Description||Comment|
|SQL Server||Buffer Cache Hit Ratio||>= 90%||Indicates poor performing queries. This measures the health of the data cache or shared pool of the database.||At < 90%, the application may be performing large table scans, index scans or the memory allocated to the buffer/shared pool is too small|
|SQL Server||Cache Hit Ratio||>= 90%||Indicates lack of dynamic SQL or bind variables. This measures that SQL Statements are properly prepared||At < 90%, there may be a dynamic SQL issue:
|SQL Server||Average Disk Queue Length||<= 2||This is the number of disk requests waiting to be serviced||If this number is high, there are probably full table or clustered index scans.|
|SQL Server||Avg. Wait Time||< 500 ms||Causing too many locks for too long a period of time||If the average waits are high, then it’s possible that the code needs to be tweaked|
|SQL Server||Full Scans/Sec||= baseline||There are table scans implemented rather than index lookups. Compare this to baseline||High number of table scans will cause bottlenecks|
|SQL Server||% Processor Time (CPU)||<= 80%||Percentage of time executing non-idle threads on the server||If % processor time is > 80, the application code may be looping excessively or may be performing table/index scans causing locks within the database|
|Oracle||Workload Repository (CPU time)||<= 20%||Database (DB) Time/Elapsed (minutes)||If CPU time is > 20% of the clock time, then this should be investigated and other Oracle indicators will be investigated.|
|Application Server Platform||Key Performance Indicator||Target||Description||Comment|
|Linux||CPU Utilization||<= 40%||The percentage of CPU time used by processes||Values > 40% for extended periods of time should be investigated|
|Linux||Available Memory||<= 80%||Amount of physical RAM used by processes||Values > 80% should be investigated
Threshold: (Total amount of RAM Used – Total amount of RAM Buffered – Total amount of RAM cached) / Total amount of RAM
|Linux||Swap Space||= 0%||The server should not be swapping before/during/after the test (Similar to the ‘page file’ in windows) Available Swap Space/Total Swap Size||Values > 0% should be investigated.|
|Windows||CPU Utilization||<= 70%||The percentage of time the threads of this process spend using the processors||Values > 70% for extended periods of time should be investigated|
|Windows||Memory\Available Mbytes||>= 20%||Amount of physical RAM available||Values < 20% should be investigated|
|Application Server||Key Performance Indicator||Target||Description||Comment|
|IIS||CPU Utilization||<= 70%||The percentage of time the threads of this process spend using the processors||Values > 70% for extended periods of time should be investigated|
|IIS||Handle Count||<= 5000||Number of threads active in this process||Thread count often increases when the load is too high|
|IIS||Requests Rejected||= 0||Number of rejected requests. Requests are rejected when one of the queue limits is exceeded||Values > 0 should be investigated|
|IIS||Application Restarts||= 0||The number of application restarts||Values > 0 should be investigated|