QTP vs Selenium 1.0 vs Selenium 2.0 (Webdriver)

QTP vs Selenium 1.0 vs Selenium 2.0 (Webdriver)

Selenium 1.0 vs Selenium 2.0

1. Licence type
QTP - Commercial software (Concurrent and per user licence)
Selenium - Open source.
Webdriver - Open source.

2. Software size
QTP - Around 1.5GB
Selenium - Set of Libraries, around 20MB (Need to include other supporting software)
Webdriver - Set of Libraries, around 20MB (Need to include other supporting software)

3. Software Support
QTP - From HP
Selenium - Saucelabs.com, Element34 , Commercial Support
Webdriver - Same as Selenium

4. Identifying objects.
QTP -  Object properties, Repository objects
Selenium - Html ID, xPath, CSS, DOM, Link text
Webdriver - Html ID, xPath, CSS, DOM, Partial Link text, Class Name

5. Spying object

QTP - GUI Spy
Selenium - There is no option, can record script in Selenium IDE, can spy objects using IE developer tool bar, FireBug and also using http://saucelabs.com/builder
Webdriver - Same as Selenium, there is no option of recording the scripts. With the introduction of built in developer tool bar from IE8 made thing easy for IE.

6. Scripting language
QTP - VB Script
Selenium - Any language HTTP library (Java, C#, Python, Ruby...)
Webdriver - Java, .Net, Python, Ruby, Node JS

7. Environment.
QTP - Windows
Selenium - Windows, OS X, Linux, Solaris.
Webdriver - Windows, Mac, Linux, iOS, Android

8. Browsers supported

QTP - IE, Firefox (Run scripts)
Selenium - IE, Firefox, Safari, Opera
WebDriver - IE, Firefox, Safari, Opera, Chrome, Android, iPhone

9. Architecture
QTP - Not known, It makes tight integration with IE browser using Windows API's. I think it doesn't require any session information, it will retrieve all the opened browsers info and connect to it using windows APIs. It relay on .Net 3.5 libraries.
Selenium - It can be written in any language that support HTTP library, corresponding language bindings are sent to Selenium RC designed on Jetty, RC launch the browser by injecting the "BrowserBot" JavaScript into the browser. Screen operation are performed by the "BrowserBot" by communicating with RC using XMLHttp request.

WebDriver - Each browser is operated in a different way by using best suitable browser language, so that all the functionality can be accomplished.

IE - IE COM automation interfaces are wrapped in c++ classes and connected to Webdriver API.

What is "Thread Boundry"? Java is multi-thread and IE COMs have single thread, this created lot of problems. It is solved by holding the IE instance in a separate thread and using the PostThreadMessage Win32 API to communicate across the thread boundary.

Every language that WebDriver support has mechanism to call C code directly, so the C language was introduced between Webdriver and c++.

Special mechanism is used to make c to communicate with OOPs c++.

Using WebDriver JsonWireProtocol it communicate with a local instance of an HTTP server and connects to C++ DLL using a native-code interop technology such as JNA, ctypes, pinvoke or DL.

JNA (Java Native Access) - Java projects that run on different platforms, especially on windows. JNA makes it easy to call native functions.

Firefox - Javascript in an XPCOM component, it is implemented as Firefox extension.

HTTPD is embedded  HTTP server from Mozilla.

Dispatcher takes the request and iterate over the known list of URLs. Once the match is found. JSON object representing the command to execute is constructed.

This is then passed as a JSON string to a custom XPCOM component written by WebDriver team called the CommandProcessor.

Chrome - Controls the browser using Chrome's automation proxy framework. Consequently, the ChromeDriver is only compatible with Chrome version 12.0.712.0 or newer.

Opera - It is a vendor supported WebDriver implementation developed by Opera Software.
The OperaDriver uses the Scope interface (same as for Dragonfly) to communicate directly with Opera from Java. Consequently, the OperaDriver is only compatible with Opera version 11.5 or newer.

Having friendly relationship with different browser teams made WebDriver to support multi-browsers efficiently using different technologies. Selecting best suitable language for a browser is key aspect for its success, as it is interacting with its natural language we can be sure that any functionality can be accomplished.

I started my carrier with Winrunner in 2003, later moved into QTP, amazing tool at that time. For supporting  multi-browsers and QTP licence issues as team grows, moved to Selenium1. After seeing the Selenium2(Webdriver) performance, it was amazing. Now I write my entire automation code using Selenium2 Page Objects, TestNG and custom framework.

Commercial software's like QTP need to rethink on their age old design.
Selenium is a serious contender for QTP; opensource, multi-browser, mult-platform, multi-language, short releases and common APIs for all the browsers made it upper-hand.

Atoms are used across all the browsers to reduce the duplicate code, any issue relating to querying the DOM should fix all the browsers issues.

10. Limitations (I mean automation is not possible)
QTP -  As per my understanding nil (I don't want to mention about the Number precision, Text length...)
Selenium - File upload/download, Model dialog ... How to over come these limitations select links?
Webdriver - Model Dialog still don't work.  Issue Resolved 

11. Identifying objects
QTP - Browser.Page.Object.Method for each step.
Selenium - Directly execute methods, no browser page reference required.
WebDriver - Directly execute methods, no browser page reference required.


12. Debugging code (Setting breakpoints in the code)
QTP - Yes
Selenium - No (Need to read logs and understand the error)
Webdriver - No (Write wrapper around webdriver listener to read logs, from 2.15 it is automatically generating logs)

13. Control Opened browsers (Attach to running browsers)
QTP - Yes (Opened after QTP program)
Selenium - No (Session information is lost) .
Webdriver - No (Fix in Progress)

14. Supporting Applications
QTP - Web applications, SAP, Activex, VB, Windows...(Support provided using Addins)
Selenium - Only Web application.
Webdriver - Only Web application.

15. As performance testing tool
QTP - No, you can run one QTP application in one CPU.
Selenium - It can open many many browsers using GRID, many companies line PushToTest, BrowserMob, Gomez use selenium technology for running load test.
Webdriver - Same as Selenium, now we have GRID2.

16. Current Version
QTP - 11
Selenium - 1.0
Webdriver - 2.15

17. Object not found timeout
QTP - It will wait till the timeout happen.
Selenium - If browser status is Done, it will through exception. In some cases it will wait for time out. I did like this feature.
Webdriver - Same as Selenium, It has implicit and explicit timeouts.

18. Execute JavaScript
QTP - No (Not required, you can get entire page info directly from QTP)
Selenium - Yes
Webdriver - Yes

19. Access page DOM
QTP - Yes
Selenium - Yes
Webdriver - Yes

20. Flex objects
QTP - Yes
Selenium - Yes  Selenium Flex
Webdriver - Yes  Selenium Flex

21. Scripting complexity
QTP - Simple, but implementing framework is complex.
Selenium -Simple, but implementing framework is complex. Need to know following items for full fledge implementation.
Webdriver - Simple, but implementing page objects is slightly complex.

22. Cost
QTP - 9K USD per user (Approx) + Annual maintenance charges.
Selenium - FREE (Any number of users...)
Webdriver - FREE (Any number of users...)

23. IDE
QTP - Own IDE
Selenium - Any IDE (I prefer Eclipse)
Webdriver - Any IDE (I prefer Eclipse)

24. Flavors
QTP - QTP with different Add-in's

Selenium - Selenium core, Selenium IDE, Selenium RC, Selenium GRID.
Webdriver - Webdriver, GRID2.

25. Extendability
QTP - No
Selenium - Yes, you can customize and implement for new browsers.
Webdriver - Same as Selenium.

26. Exception Handling
QTP - Need to handle manually. Using .Exist, On error resume next, recovery scenarios. Script would stop in the middle by throwing run-time error if not handled properly. Programmer should have complete understanding where to use exception handling, else he suppress real errors.
Selenium - Handled automatically when used with TestNG, it will automatically move to next test case and user can implement Try...Catch to handle specific exceptions.
Webdriver - Same as Selenium. It has some extra exceptions like NoElementFound where code can be handled in a better way.

27. Reporting and Assertions(Checkpoints)
QTP - Reporting and checkpoints are built in the same package.
Selenium - Need to depend on testing frameworks like TestNG or Junit.
Webdriver - Same as Selenium.

28. Recording the script
QTP - Built in recorder that can generate script on IE.
Selenium - Selenium IDE as Firefox add on.
Webdriver - No

29. Script Execution
QTP - Just hit Play button.
Selenium - Need to run Selenium Server and it opens a separate window apart from the browser.
Webdriver - Just hit play button.

30. Page synchronization
QTP - Use .Sync method  to check the page load for every page navigation. This is the statement which I don't like to put in my code creating redundancy. I have redefined each method to element .Sync in my program.
Selenium - Use Wait for page load. This is the statement which I don't like to put in my code creating redundancy. I have created new methods eliminate from my program.
Webdriver - It is handle automatically, no method is required. Awesome job from the Web driver team, you guys really understood the concept and directly place the synchronization at the webdriver code level for those methods where there is browser navigation. This made be curios to understand how it is implemented at code level, came across this code (Highlighted in Green) in c++.
IE driver instance creation

int wdNewDriverInstance(WebDriver** result)

{

*result = NULL;

TRY

{

terminateIe();

WebDriver *driver = new WebDriver();

driver->ie = new InternetExplorerDriver();

driver->ie->setVisible(true);

driver->implicitWaitTimeout = 0;

openIeInstance = driver->ie;

*result = driver;

return SUCCESS;

}

END_TRY

return ENOSUCHDRIVER;

} Open URL in the above instance, wait for page load int wdGet(WebDriver* driver, const wchar_t* url) {         if (!driver || !driver->ie) return ENOSUCHDRIVER;         try {                 driver->ie->get(url);                 driver->ie->waitForNavigateToFinish();                 return SUCCESS;         } END_TRY; } bool Browser::Wait() {         bool is_navigating = true;         //std::cout << "Navigate Events Completed." << std::endl;         this->is_navigation_started_ = false;         HWND dialog = this->GetActiveDialogWindowHandle();         if (dialog != NULL) {                 //std::cout "Found alert. Aborting wait." << std::endl;                 this->set_wait_required(false);                 return true;         }         // Navigate events completed. Waiting for browser.Busy != false...         is_navigating = this->is_navigation_started_;         VARIANT_BOOL is_busy(VARIANT_FALSE);         HRESULT hr = this->browser_->get_Busy(&is_busy);         if (is_navigating || FAILED(hr) || is_busy) {                 //std::cout << "Browser busy property is true.\r\n";                 return false;         }         // Waiting for browser.ReadyState == READYSTATE_COMPLETE...;         is_navigating = this->is_navigation_started_;         READYSTATE ready_state;         hr = this->browser_->get_ReadyState(&ready_state);         if (is_navigating || FAILED(hr) || ready_state != READYSTATE_COMPLETE) {                 //std::cout << "readyState is not 'Complete'.\r\n";                 return false;         }         // Waiting for document property != null...         is_navigating = this->is_navigation_started_;         CComQIPtr<IDispatch> document_dispatch;         hr = this->browser_->get_Document(&document_dispatch);         if (is_navigating && FAILED(hr) && !document_dispatch) {                 //std::cout << "Get Document failed.\r\n";                 return false;         }         // Waiting for document to complete...         CComPtr<IHTMLDocument2> doc;         hr = document_dispatch->QueryInterface(&doc);         if (SUCCEEDED(hr)) {                 is_navigating = this->IsDocumentNavigating(doc);         }         if (!is_navigating) {                 this->set_wait_required(false);         }         return !is_navigating; }

31. Handle AJAX calls
QTP - Manual
Selenium - Manual
Webdriber - Manual (There are certain classes to handle it, but didn't work for me)

32. XPath Support (Awesome technique to locate elements when there is no id or name)
QTP - Yes
Selenium - Yes (Very slow on IE)
Webdriver - Yes

33. Accessing windows objects (Folders, Excel, Notepad)
QTP - Easy (As it is developed in VB script native windows script. I hope this is one of the reason why qtp selected VB Script)
Selenium - Not so easy, need to depend on open APIs.
Webdriver - Same as selenium.

34. Code base
QTP - Huge, storing Actions occupy lot of space and more number of files. I prefer creating code in function libraries only (.vbs, .txt, .qfl)
Selenium - Small, text file with .java extension. Suitable with any configuration management tools.
Webdriver - Same as selenium.

35. Executing test on different systems
QTP - Need to setup QTP and copy the code execute it.
Selenium - Using ANT it is possible to compile into standalone jar file, so that it is easily executed from the command prompt.
Webdriver - Same as selenium.

36. APIs
QTP - Each object has methods and properties. QTP classified the objects into (WebEdit[Textbox], WebButton, WebList, WebRadio Button....). Objects are classified into Windows, Web, People soft, SAP, Visual Basic, Activex...., need to add extra licensed programs called Addin for supporting of the corresponding objects. Addin program can help QTP to identify the properties in specific environment. To identify an object on the screen, need to provide properties. Properties are classified into Mandate, Optional and ordinal properties.
To identify the an object on the screen, following statement is required.

Browser(RepositeryObjectName/Properties).Page(RepositeryObjectName/Properties).Object(RepositeryObjectName/Properties).method

Selenium - Directory based API, all methods exist in one class. It is a table based API that contain 3 columns

Selenium command, Locator(Object ID, Name, xPath...), list of parameters in array format.
Note: Browser, Page details are not required; its reference is automatically handled by Selenium session.

Webdriver - Complete OOPs.
Define Browser object (IE, Firefox, Chrome...)
Define Element object.
Run the method on the element object.

// Create an instance of WebDriver backed by Firefox
WebDriver driver = new FirefoxDriver();

// Now go to the Google home page
driver.get("http://www.google.com");

// Find the search box, search for something
WebElement searchBox = driver.findElement(By.name("q"));
searchBox.sendKeys("selenium");

// And now display the title of the page
System.out.println("Title: " + driver.getTitle());

37. Number of Methods
QTP - Each object has certain methods, most of the methods are unique like Set, Click, Select...
Selenium - Few methods.
Webdriver - You can complete most of the script using click and SendKeys.

38. Object Repository (Object information is stored separately during recording and can be edited if required)
QTP - Yes. Also provide option to define objects at code level called descriptive programming (DP).
Selenium - NO, only through code.
Webdriver - NO, only through code.

39. Page Objects Implementation
QTP - VB script don't supports complete OOPs, but can be achieved with Dual-Function Framework.
Selenium- I think no, it is not possible to move Selenium session from one class to another, but methods can be grouped as per the page by extending the Selenium interface.
Webdriver - Yes, support complete OOPs. (Planning to write a new post on this with an example).

40. Synthesizing events on the page
QTP - Synthesize events on DOM.
Selenium - Synthesize events on DOM.
Webdriver - Type looks very realistic as though real user is typing. I think they are sending the events to IE instance message queue. (Need to understand the technique used)


Over all comparison


QTP -
Browsers - (IE/Firefox)
OS - (Windows)
Scripting - (VB Script)
Licence - ($8000 per user)
Limitations(No solution) - Nil

Webdriver -
Browsers - (IE, Firefox, Safari, Opera, Chrome, Android, iPhone),
OS - (Windows, Mac, Linux, iOS, Android)
Scripting - (Java, .Net, Python, Ruby, Node JS)
Licence - Free
Limitations(No solution) - Model-Popup (Some can be handled). By making small changes in the UI it can be easily tackled, display model-popup content in Div by graying the background. Issue Resolved 

Now Your Choice  ?



---