Wednesday, July 7, 2010

Selenium - Functional, Performance testing tool.

Selenium - Functional, Performance testing tool.

Selenium is powerful open source tool for testing (Functional) web applications; it is similar to HP QTP (Quick Test Professional).

It provide full control on the web pages, by allowing access to view source, DOM elements and complete navigation(clicks, selecting links etc) through different API’s.

Selenium IDE is a firefox add-on that can record the user actions in the form of script, which can reply again in the browser. It can also convert the recorded script in different programming languages.

We can integrate Selenium Remote Control(RC)  with different programming languages (.Net, Python, Perl, Ruby, Java) and execute the selenium scripts.

Some companies link BrowserMob, PushToTest use Selenium GRID to conduct performance testing. It consume more resources and not a good idea for conducting performance testing.

Link to All Selenium Projects

It is always worth learning these open source tools, not required to pay license fee.


Tuesday, July 6, 2010

Web Site Test Tools

Web Site Test Tools

This link contain 440 tools listed in 12 categories.


Performance Testing - TCP Connection Failures

Performance Testing - TCP Connection Failures.

I came across this article on WebPerformanceInc, which explain about establishing TCP connection and different reasons for connection failures…felt interesting.

Load Tester is a web site load testing tool, and as such we deal primarily with the most popular Internet communications protocol: the Hypertext Transfer Protocol, or HTTP, which controls the request and transmission of web pages between browser clients and web servers.  HTTP is based on a lower-level protocol known as the Transmission Control Protocol, or TCP.    For the most part, TCP works in the background, but its proper function is critical to your website, and problems at the TCP level can show up in many different ways during a load test.  These errors can sometimes be difficult to troubleshoot, requiring a packet sniffer such as Wireshark or tcpdump to analyze, while others are simpler.

TCP uses the concept of “ports” to identify and organize connections.  For every TCP connection, there are two ports – the source port, and the destination port.  For our purposes, the most important ports are port 80 and port 443, which are the two most common ports utilized by web servers – 80 for normal HTTP traffic, and 443 for SSL-encrypted traffic.  A typical TCP connection from a client to a webserver will involve a random source port such as 44567, and a destination port on the server of port 80.  Each web server can accept many hundreds of connections on port 80, but each connection must come from a different source port on each client.

To create these connections between ports, TCP relies on a three-way handshake.  The requesting client first sends a packet with the TCP SYN flag set, indicating that it wants to open a connection.  If the server has a process listening on the destination port, it will respond with a packet that has both the SYN flag set and the ACK flag set, which acknowledges the client’s SYN and indicates that a connection can be created on that port.  The client then sends a packet with the ACK flag set back to the server, and the connection is established.  The current connections can be viewed using the netstat tool on both Windows and Linux.

What does it look like when a TCP connection attempt fails?  The TCP packet with the SYN flag is sent from the client, which in our case is a load engine.  If the server sees such a packet, but does not have a process listening on the target port, it will typically respond with a TCP packet that has the ACK and RST flags set – a TCP reset.  This tells the client that connections are not available on this port.
Load Tester showing a connection refused (ACK RST)
Load Tester showing a connection refused (ACK RST)
This screenshot shows the result of a load engine failing to connect to the server.  In this case, you can see that I attempted to connect to TCP port 442, which doesn’t have a web server running on it (or any other service, for that matter).  Note that the response was received quickly, in about 1 second, indicating that the remote server saw the ill-fated packet and responded.  The most important thing to know about this error is that it is one of the most reliable errors that you’ll see – either the Load Tester controller or the load engine really is having trouble connecting to the site.  The most common reason for this is that either the site is down, or there is a firewall that is blocking the load engine but not the controller.
So … what happens when the remote server does not respond?
Load Tester showing a connection timeout (dropped packet)
Load Tester showing a connection timeout (dropped packet)
This screenshot shows the same attempted connection, only this time, no response was received from the target server – not even the TCP reset that indicates connections are not available on the target port.  Note how long it takes for Load Tester to report an error – 21 seconds, in this case.  I induced this error by configuring the Linux iptables firewall to drop all incoming packets on TCP port 442, so the server’s TCP stack never saw the incoming SYN packet and thus did not respond to it – from the server’s perspective, the packet never arrived.  A similar error will occur if the server cannot be reached for some reason; for example if you attempt to connect to the wrong hostname, the server is offline, or your traffic is being misrouted between the client or load engine and the server.  If you see these kinds of errors, then the first thing you should do is make sure that the server is up, and that any HTTP proxy servers necessary to reach the server are configured correctly.

Of course, TCP connections can also fail after a connection has been established.  Here’s an example:
Load Tester showing a server connection termination
Load Tester showing a server connection termination
This error message is much less clear.  Did the server close the connection on purpose?  If so, why?  If not, what happened? Did the process handling the server connection crash or return bad data?  In this case, it’s useful to know what Load Tester considers to be a successful connection.  Load Tester expects there to be HTTP headers, followed by data.  In this case, we did not finish receiving the HTTP headers, and so Load Tester considers the connection incomplete.  Load Tester failed to receive the headers in this case because I induced this error by attempting to elicit an HTTP response from the Secure Shell (ssh) service listening on TCP port 22, which terminated the connection after receiving what it saw as invalid data – Load Tester’s HTTP request.

In a real test, there’s a pretty large number of things that can cause this error, from server process crashes or errors, to overly aggressive firewalls, to reverse proxy failures, to misdirected traffic on a load balancer.  In such a case, a traffic analyzer such as Wireshark or tcpdump can be very helpful in determining what is happening.  Note that you may need to observe traffic in more locations that in front of the load engine or the controller though, as traffic can be altered by firewalls and load balancers.


Velocity - Web Performance Conference 2010

Velocity - Web Performance Conference 2010

Oreilly’s Velocity Conference exclusively for website performance and testing.

Metrics 101
View more presentations from Alistair Croll.

There are around 20 videos relating to this conference on YouTube


Sunday, July 4, 2010

Evaluation of computing

Evaluation of computing

30 minutes video, that explain the evaluation of computing from primitive stage, where there are no CRT monitors, user need to use paper as input and output medium ...interesting to watch. 

Best Practices For Speeding Web Site

 Best Practices for Speeding Up Your Web Site

Google Link

Yahoo Link


Performance Testing - Why site could be slow, even with low CPU/RAM/disk utilization.

Why site could be slow, even with low CPU/RAM/disk utilization.

Some times site appeared to slow down significantly, despite the fact that their CPU, RAM, and disk utilization did not rise in utilization significantly. While those three metrics are often good indicators of why systems can “slow down”, there are many other causes of performance problems. Today, we’re going to discuss one common root cause for slow websites that often gets overlooked: connection management.

Until very recently, most web browsers would only issue a maximum of two connections per host, as per the recommendation by the original HTTP/1.1 specification. This meant that if 1000 users all hit your home page at the same time, you could expect ~2000 open connections to your server. Let’s suppose that each connection consumes, on average, 0.01% of the server’s CPU and no significant RAM or disk activity.

That would mean that 2000 connections should be consuming 20% of the CPU, leaving a full 80% ready to handle additional load – or that the server should be able to handle another 4X load (4000 more users). However, this type of analysis fails to account for many other variables, most importantly the web server’s connection management settings.
Just about every web server available today (Apache, IIS, nginx, lighthttpd, etc) has one or more settings that control how connections are handle. This includes connection pooling, maximum allowed connections, Keep-Alive timeout values, etc. They all work basically the same way:
  • When a request (connection) comes in to the server, the server will look at the maximum active connections setting (ie: MaxClients in Apache) and decide if it can handle the request.
  • If it can, the request is processed and the number of active connections is incremented by one.
  • If it can’t, the request is placed in to a queue, where it will wait in line until it finally can be processed.
  • If that queue is too long (also a configuration setting in the server), the request will be rejected outright, usually with a 503 response code.
It’s this queue that can make your site to appear to be slow, despite low server utilization. Say the server allows up to 256 concurrent requests and each request takes 1 second to complete. That means if 1000 users visited the site at the same time, causing 2000 requests, then the first 128 (256/2) users would get a 1 second response time, the second 128 users would get a 2 second response time, and the last user would get an EIGHT SECOND response time.

The simple solution is to raise the concurrent request limit. However, be careful here: if you raise it too high it’s possible your server won’t have enough CPU or RAM to handle all the requests, resulting in all users be affected (rather than just some of them, like in the last example).
Also remember that not all requests are equal: a request to a dynamic search result will be much more expensive than one to a static CSS file. This is why larger sites optimize their hosting to place static files on special web servers with different configurations, usually with host names like, while leaving their more complex content to be handled by a larger quantity of servers with a fewer number of concurrent requests on each server.

So next time you’re wondering why your site is slow, take a look at more than just CPU and RAM. Find out how the server is processing the content and see if perhaps your web server is the bottleneck.
Source: browsermob


Performance Testing - TTFB (Time to First Byte)

Performance Testing - TTFB, TTLB

When you open any web page, there are series of requests from the client(usually a web browser) and response from the server.

TTFB (Time to First Byte) - Amount of time it took for the client (usually a web browser) to receive the first byte of server response.

TTLB (Time to Last Byte) -  Amount of time it took for the client (usually a web browser) to receive the last byte of server response i.e total time taken to download an object

Both metrics are used in performance testing for analyzing the bottleneck.

I will explain the metrics by taking an example.
Both the requests appear to be hosted on two different servers.
Both the requests tool almost 1 second to complete, but "pic.png" having file size four times grater than the "login.jsp". What is going on? In-order to understand the complete story, we need to know TTFB.
With this additional information, we can understand what is happening.

In case of "login.jsp" after receiving the first byte, it took 09ms to download the remaining content.

Where as "pic.png" received the first byte very quickly but took another 900ms to download the remaining content.

"login.jsp" bottleneck likely to be server side processing, due to heavy CPU usage. This is common for dynamic pages which need to process before sending the response to the client. If the process involve database or other expensive operation, that could be cause of slow performance.
Situation is different for "pic.png", delay is likely due to slow network or poor configuration for the server hosting the image. As the image is not a dynamic content, it will not be consuming more CPU.

So how do you resolve these different situations?

In the case of objects with long TTFB times, like index.jsp, the solution often requires a software-level optimization. It could involve adding a database index, introducing some object-level caching, or a configuration change (such as database connection pooling). Be careful to fall in to the trap of throwing more hardware at the problem to solve these types of issues. While it might work in the short term, these issues almost always are due to sub-optimal software and throwing extra hardware at the problem will be like putting a band-aid on a bullet hole.

In the case of objects with relatively short TTFB times but overall long TTLB times, the solution is usually very different. While there may be a software solution, such as configuring Apache’s connections to be better optimized for the server it runs on, most of the time the root cause is due to network/hardware-related issues. Check with the ISP that hosts the server to confirm the max bandwidth throughput allowed. If the object response is slow during peak times but fast during off-peak times, it may need extra web servers (ie: hardware).

Alternatively, you might want to look at a Content Delivery Network (CDN) like CDNetworks to help host the objects in a physically closer location. For a low-cost CDN, check out Amazon’s CloudFront service, which can let you host images and other static objects in nine separate locations around the world. This is a great, low-cost solution for people who want to serve static content to many different geographies but don’t have the budget or desire to open mutliple data centers.
Source: browsermob