Loadrunner - Merging Virtual user log files.

Loadrunner - Merging Virtual user log files.

Good loadrunner script should record all the important events during execution - success messages, check point messages, acknowledgment numbers; so that we can merge all the virtual users logs and generate a single merged log file as per the requirement.
This merged log file can be compared with the database entries, if required.

Especially recording should happen on failed transactions, so that we can show these transactions to developers.


Attaching sample log merging script, need to customize as per the requirements (which messages to pick).
Below script will search for the specified strings in the all the log files generated by the virtual users (all the files in a particular folder having any name) and consolidate it in a single file.

For advanced filtering of data, use special character as delimiter (pipe or # symbol) between group of words while generating log messages in the loadrunner, so that we export to an excel file and apply filters.

 'Merge the log files
 'Update line number 16 17 19 accordingly 
 'Enter directory path of the log folder
 sDirectory = "C:\Documents and Settings\bharathm\Desktop\log\"
 iRecords = 0
 Set oFSO = CreateObject("Scripting.FileSystemObject")
 If oFSO.FolderExists(sDirectory) Then
  Set oNotepad = oFSO.createtextfile(sDirectory & "MergedRecords.txt")
  Set objFolder = oFSO.GetFolder(sDirectory)
  Set colFiles = objFolder.Files
  For Each objFile in colFiles
    'Wscript.Echo objFile
    Set oFile = oFSO.OpenTextFile(objFile, 1)
     Do While Not oFile.AtEndOfStream
      sText = oFile.ReadLine
       If instr(1,sText,"Action.c(796)") > 0 Then
        If instr(1,sText,"Success") > 0 Then
          iLen = len(sText)
          sText1 = Mid(sText,15,iLen-15)
          'WScript.Echo sText1
          oNotepad.writeline(sText1)
          iRecords = iRecords + 1
        End If
       End If
     Loop
     oFile.Close
  Next
  oNotepad.writeline("#############################################")
  oNotepad.writeline("Total number of records merged: " & iRecords)
  oNotepad.Close
  WScript.Echo "Log merging Completed...verify the new file"
 Else
  WScript.Echo "No folder exist."
 End If

Loadrunner - Good scripting guidelines



----

Loadrunner - How to download and save file.

Loadrunner - How to download and save file.

Based on the test scenario, we may need to replicate downloading the file by number of users.
While downloading file, we come across multiple windows based pop-ups, unfortunately actions on these pop-ups are not recorded by load runner. To over come this limitation, we need to capture the file stream and save this data as actual file using c file system.


Following points to be taken into consideration while designing the code.
1. Each file is given unique name, otherwise it would create conflict with the OS. I take virtual user id and time stamp from the parameter lit for creating file names, so that it is unique always.
2. Create file names with proper file extension (.XLS or .PFD) similar to the file you are planning to download.
3. Always use "\\" for file path.
4. Make sure you have created folder in the load injector. (c:\LR Files)
5. Always use the function "lr_eval_string_ext_free" to free the allocated memory.

Attaching sample code

    // Variable declaration 
    long filePointer;
    char *DataPointer;
    unsigned long DataLength;
    char FilePath[50],FilePath1[50]; 

    // Creating unique file names and constructing the folder path
    strcat(FilePath1,lr_eval_string("{UserId}"));
    strcat(FilePath1,"_");
    strcat(FilePath1,lr_eval_string("{DateTime}"));
    strcat(FilePath1,".xls");
    strcat(FilePath,"c:\\load\\");
    strcat(FilePath,FilePath1);

    // Just displaying the path that you have constructed above (debugging purpose only) 
    lr_output_message("File path:%s",FilePath);
    lr_output_message("File path1:%s",FilePath1);
    

    //Start creating file (File Pointer) for writing in the desired path.
    filePointer = fopen(FilePath,"wb");
    
    //Increase parameter size to accommodate your file size. 
    web_set_max_html_param_len("9999");
    
    //This function is used to capture the entire data returned by the server only in the body.
    web_reg_save_param("FILE_DATA","LB=","RB=","Search=Body",LAST);

    Loadrunner function to initiate click event and start the download process

    //This function is used to create data buffer and assign a pointer to it, so that this data can be used in file processing.  Also calculate length for the above captured data.
    // "0, 0, -1" are reserved values by loadrunner. Check the help file for more details 
    lr_eval_string_ext("{FILE_DATA}", strlen("{FILE_DATA}"), &DataPointer, &DataLength, 0, 0, -1);
    
    //This function write data into above specified file.
    fwrite(DataPointer, DataLength, 1, filePointer );



   //This function frees the above created data buffer, best practice as per loadrunner documentation.  
   lr_eval_string_ext_free(DataPointer);
    
    //This function close the file pointer
   fclose(filePointer);

-----

QTP - Send test results as SMS to a mobile phone.

QTP - Send test results as SMS to a mobile phone.

When framework or test is getting executed (scheduled daily unattended) on production environments where test results are critical, it would be better to see the test results on your mobile immediately after the test completion, so that quick action can be taken if there are any anomalies.

There are different APIs available, I have used API provided by clickatell
You can register and test your code on this website.

Attaching sample code.

Set WshShell = WScript.CreateObject("WScript.Shell")
Set oXMLHTTP = CreateObject("Microsoft.XmlHttp")
sUrl = "http://api.clickatell.com/http/sendmsg"
sAPI_ID = "xxxx"
sPassword = "xxxx"
sUsername = "xxxx"
sMobileNo = "xxxx"
sText = "Message from clickatell.com" ' Assign the SMS text message to this variable
sPostData = "api_id=" & sAPI_ID
sPostData = sPostData & "&user=" & sUsername
sPostData = sPostData & "&password=" & sPassword
sPostData = sPostData & "&to=" & sMobileNo
sPostData = sPostData & "&text=" & sText 
oXMLHTTP.Open "POST", sUrl, false
oXMLHTTP.SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
oXMLHTTP.Send sPostData
sResult = oXMLHTTP.responseText
Set oXMLHTTP = nothing
Wscript.Echo sResult
Set WshShell = nothing

After sending the message, you would receive following text as conformation message.

---

QTP - Where and how to use "On Error Resume Next" statement.

QTP - Where and how to use "On Error Resume Next" statement.

"On Error Resume Next" causes execution to continue with the statement immediately following the statement that caused the run-time error, or with the statement immediately following the most recent call out of the procedure containing the "On Error Resume Next" statement. This allows execution to continue despite a run-time error. You can then build the error-handling routine inline within the procedure.

An "On Error Resume Next" statement becomes inactive when another procedure is called, so you should execute an "On Error Resume Next" statement in each called routine if you want inline error handling within that routine. When a procedure is exited, the error-handling capability reverts to whatever error-handling was in place before entering the exited procedure.

Use "On Error GoTo 0" to disable error handling if you have previously enabled it using "On Error Resume Next".

It is not a good practice to use "On Error Resume Next" for entire test or project, this will suppress actual errors during execution and complicate debugging process. Just using "On Error Resume Next" will not help us, need to write proper error-handling routine.

In QTP, .Sync is the only method for which "On Error Resume Next" to be used, it is not so useful for other methods, you can use .Exist or check specific object property instead of "On Error Resume Next". Your code would get complicated if there are more error-handling routines. Enable error handling by using "On Error Resume Next" for necessary statements and disable error handling by using "On Error GoTo 0".

Err object is part of VB Script, created and updated automatically. It contain information about the last error occurred in the script. Script engine will overwrite the error information when it encounters next error, information is not preserved. Always clear the object information once the error check is completed else same error information will cause issue in the next error check.

Different methods associated with Err object
Err.number - Contain value that contain error code, Zero no error occurred.
Err.description - Descriptive message associated with the error.
Err.source - Set name of the object or application where error generated.
Err.helpcontext - Return value containing context id of the appropriate topic in the help file.
Err.helpfile - Path of help file.
Err.rise - Generate run-time error.



Attaching sample code from my Dual-Function framework

On Error Resume Next
oObject.Sync

'Error-handling routine 
If err.number = 0 Then
Else
        err.clear
        gsExecutionStatusTestCase = conF
        gsStatusMessageTestCase = gsStatusMessageTestCase & "Page " & oObject.GetTOProperty("title") & " don't exist OR Using Incorrect Keyword " & " # "
End If
On Error GoTo 0

----

Loadrunner - Ajax(Click & Script) - EvalJavaScript, EvalJavaScriptResultParam

Loadrunner - Ajax(Click & Script) - EvalJavaScript, EvalJavaScriptResultParam

Select AJAX(Click & script) protocol if your application contain Ajax functionality.

AJAX (Asynchronous java and XML) is preferred solution for creating fluid applications with advance user interactivity. Traditionally browser use GET and PUT method to refresh entire page for every interaction with the server, this would create load on server and network. By using AJAX, browser and server exchange small amounts of data in (XML or JASON format) asynchronously in the background through XMLHttpRequest object with out reloading the whole page for every user action. Javascript code use the data received from the server and update HTML in browser DOM (Document object model) which contain all the elements of the browser session. 

By using very limited object properties, HP could implement QTP technology into this protocol.

Loadrunner will create corresponding Web_xxxxx function, when user perform actions on STANDARD web objects. It can't recognize third party objects, then we need to use Java script to emulate user actions.
Understand third party object methods, properties by looking at the page source code or interacting with the developer who created the page and create corresponding javascript code.

Attaching sample code for FCK editor and ULTRA web grid.

        web_browser("Java_Script",
            ACTION, 
            "EvalJavaScript= var oEditor = FCKeditorAPI.GetInstance('ctl00_cpheSupplier_txtNdaMessage');var oDOM = oEditor.EditorDocument;oDOM.body.innerText = 'Bharath';",
            LAST); 

        web_browser("Java_Script",
            ACTION, 
            "EvalJavaScript=igtbl_getRowById('ctl00xcpheRFxxUltraWebGrid1_r_16').getCell(7).setValue('999');updating = true;"
            "EvalJavaScript=igtbl_getRowById('ctl00xcpheRFxxUltraWebGrid1_r_17').activate();",
            LAST); 

Some time we may require to retrieve page DOM values. Attaching sample code that retrieve element properties into loadrunner.

    web_browser("JavaScriptCode", 
        DESCRIPTION, 
        ACTION, 
        "EvalJavaScript=var id = document.getElementById('message');EvalJavaScriptResultParam = id.style.display;", 
        "EvalJavaScriptResultParam=prmJavaScriptReturn",
        LAST);
    lr_output_message("%s",lr_eval_string("{prmJavaScriptReturn}"));  

----

Loadrunner - Ajax(Click & Script) - How to handle user events not recognized by LR.

Loadrunner - Ajax(Click & Script) - How to handle user events not recognized by LR.

Select AJAX(Click & script) protocol if your application contain Ajax functionality.

AJAX (Asynchronous java and XML) is preferred solution for creating fluid application with advance user interactivity. Traditionally browser use GET and PUT method to refresh entire page for every interaction with the server, this would create load on server and network. By using AJAX, browser and server exchange small amounts of data in (XML or JASON format) asynchronously in the background through XMLHttpRequest object with out reloading the whole page for every user action. Javascript code use the data received from the server and update HTML in browser DOM (Document object model) which contain all the elements of the browser session. 

By using very limited object properties, HP could implement QTP technology into this protocol.

Loadrunner will create corresponding Web_xxxxx function, when user perform actions on STANDARD web objects. It can't recognize third party objects, then we need to use Java script to emulate user actions.
Understand third party object methods, properties by looking at the page source code or interacting with the developer who created the page and create corresponding javascript code.

Attaching sample code for FCK editor and ULTRA web grid.

        web_browser("Java_Script",
            ACTION, 
            "EvalJavaScript= var oEditor = FCKeditorAPI.GetInstance('ctl00_cpheSupplier_txtNdaMessage');var oDOM = oEditor.EditorDocument;oDOM.body.innerText = 'Bharath';",
            LAST); 

        web_browser("Java_Script",
            ACTION, 
            "EvalJavaScript=igtbl_getRowById('ctl00xcpheRFxxUltraWebGrid1_r_16').getCell(7).setValue('999');updating = true;"
            "EvalJavaScript=igtbl_getRowById('ctl00xcpheRFxxUltraWebGrid1_r_17').activate();",
            LAST); 

Some time we may require to retrieve page DOM values. Attaching sample code that retrieve element properties into loadrunner.

    web_browser("JavaScriptCode", 
        DESCRIPTION, 
        ACTION, 
        "EvalJavaScript=var id = document.getElementById('message');EvalJavaScriptResultParam = id.style.display;", 
        "EvalJavaScriptResultParam=prmJavaScriptReturn",
        LAST);
    lr_output_message("%s",lr_eval_string("{prmJavaScriptReturn}"));  

----

Loadrunner - Good scripting guidelines.

Loadrunner - Good scripting guidelines.


1. Design your initial script in such a way that it works from the parameter list or fixed value (enter data into the script directly, especially used during debugging when user want to check with specific id's) and store the information into an variable for further use, I will explain it below.

Attaching sample code.


In the fixed value code, store data into a variable and convert into a parameter.

    /// Fixed value
    //sprintf(sUserName,"test@test.com"); 
    //lr_save_string(sUserName,"prmUserName");

    // OR 

In the parameter list code, store the data into variable.
    /// From the parameter list
    sprintf(sUserName,lr_eval_string("{prmUserName}")); <\code>

Enable or disable the code which ever is appropriate.

2. Response size, Log settings,  lr_output_message and lr_vuser_status_message

a) Measure response size of each request, this will give you idea about light and heavy pages.
b) Enable log when required dynamically through code, this will reduce the network activity between controller and load injector.
c) Use lr_output_message to store check point pass/fail status, login information, time stamps and dynamically generated acknowledgment numbers. Once test is completed, we can merge all the virtual user logs into a single file and compare values with the database or present failed check points along with the login information to the development team, so that it would be easy to tell which logins have failed while submitting the transactions.
Note: Need to write code in vbs to merge all the virtual user logs and generate report as required.
d) Use lr_vuser_status_message to display real time virtual user status in the virtual users window. Generally we see virtual users in "Running..' status during execution, we don't know in which page virtual user x is in the application. If your script is running for more than 30 minutes, real time virtual user information will help us to understand the health of your test and application.
Attaching screen shot displaying each virtual user login details and page activity information.
Place this function in important junctures, so that we can understand the real virtual user activity.

By looking at the above screen shot we can easily understand in which page our virtual users exist.

Attaching sample code for the above mentioned functions.

lDownloadSize = web_get_int_property( HTTP_INFO_DOWNLOAD_SIZE );
lr_set_debug_message( LR_MSG_CLASS_BRIEF_LOG , LR_MSG_ON );
lr_output_message("%s | In Update Page...Bytes: %i",sUser,lDownloadSize);
lr_vuser_status_message("%s | In Update Page...Bytes: %i",sUser,lDownloadSize);
lr_set_debug_message( LR_MSG_CLASS_BRIEF_LOG , LR_MSG_OFF);

3. About lr_end_transaction

Create transactions to capture response time.
Consider following points while assigning LR_PASS to lr_end_transaction
a) Check transaction status.
b) Place checkpoints on conformation messages...
c) Size of response.
d) Transaction response time (Configured in SLA).

Attaching sample code

if (iStatus == LR_PASS || atoi(lr_eval_string("{UpdateSuccessful_count}")) == 1 || lDownloadSize > 5000) 
{
    lr_end_transaction("07_Update_Bid", LR_PASS);
}
else
{
    lr_end_transaction("07_Update_Bid", LR_FAIL);
}

Note: Assign numbers in ascending order to the transaction names, so that transactions will be arranged properly in the controller and analysis report.

4. Global think time

Think time is essential for emulating real scenarios, without it virtual users will be running at speed that is not true. It is very tedious to change the think time across the script.

Attaching sample code that will automate the process, change at one place will have effect across the script.
tips and tricks for configuration, scripting
and execution

int iThink_Time = 10;

lr_think_time(iThink_Time);

Create above variable as global variable by defining outside the vuser_int(), so that it can be accessed in all the actions


Loadrunner - Merging Virtual user log files.

Performance Testing - Validate server response.


HP loadrunner best practices, tips and tricks for configuration, scripting and execution

Hp load runner best practices

View more documents from Bharath Marrivada.

---

QTP - Always use registered methods for the STABILITY of framework or test.

QTP - Always use registered methods for the STABILITY of framework or test.

When method is executed on a test object there are lot of instances where object missing, properties changed, input data not matching with the run time object (length, list box values not matching). This will stop your test by displaying run-time error . In order to shield your test from these errors, we need to override existing methods by registering user defined functions.


I will explain this concept by taking WebEdit Set method.

When this method is fired
1.Check Object exist
2. Highlight the object if required (my framework requirement, just to demo the QTP flow during execution)
3. Check object is in enable state.
4. Check whether the object max length is less than or equal to user value.
5. If 1,3 and 4 conditions are satisfied, set method is executed. I use global variables (gsExecutionStatusTestCase, gsStatusMessageTestCase)  to send failed status messages to the framework. By using status and message, framework will update control and test case files. For more information about the framework look into Dual function framework blog.

In my framework I had redefined all the test object methods, so that script would never fail due to above mentioned issues. I strongly recommend not to use "On Error Resume Next" for the same.

RegisterUserFunc "WebEdit", "Set", "WebEditSetUserDefined" 


Public Function WebEditSetUserDefined (oObject, sValue)
        If oObject.Exist(10) Then
                If gsObjectHighlight = "ON" Then
                        oObject.Highlight
                End If
                If oObject.GetROProperty("disabled") = 0 Then
                        If  oObject.GetROProperty("max length") <  Len(sValue)  Then
                                gsExecutionStatusTestCase = conF
                                gsStatusMessageTestCase = gsStatusMessageTestCase & "Object WebEdit: " & oObject.GetTOProperty("html id") & " Input greater than field length " &  " # "
                        Else
                                oObject.Set sValue
                        End If
                Else
                        gsExecutionStatusTestCase = conF
                        gsStatusMessageTestCase = gsStatusMessageTestCase & "Object WebEdit: " & oObject.GetTOProperty("html id") & " in disable state " &  " # "
                End If
        Else
                gsExecutionStatusTestCase = conF
                gsStatusMessageTestCase = gsStatusMessageTestCase & "Object WebEdit: " & oObject.GetTOProperty("html id") & " don't exist on the page " & " # "
        End If
End Function

---

QTP - Schedule framework or test using AOM script.

QTP - Schedule framework or test using AOM script.

Good framework need to run the test automatically and send the results through email to all the stake holders.

Below AOM script perform following actions.
1. Find the place of execution using system IP, so that people can understand where exactly test got executed. If your test is executing from different locations, implementing this feature is a good option.
2. Empty few folders before starting the test, as per my dual function framework requirement.
2. Open the QTP with the corresponding test and pass the framework parameters as environment variables.
3. Once test got completed, copy the test folder to another new folder (folder name created using current time stamp), so that results are preserved from being overwritten by the next scheduled test.

Note: Script need to be customized as per the requirements and integrated with the windows scheduler.

'Folders path
sParentFolder = "C:\Automation FrameWork\AutomationTestDocuments\ProductionLoginTesting"
sControlFilePath = "AutomationTestDocuments\ProductionLoginTesting\ProductionLoginTestingControlFile.xls"

'******************************* Don't Edit the code beyond this line *******


sToCopyFolder = "C:\Automation FrameWork\ExecutedScripts\"

'Find system IP address

strQuery = "SELECT * FROM Win32_NetworkAdapterConfiguration WHERE MACAddress > ''"

Set objWMIService = GetObject( "winmgmts://./root/CIMV2" )
Set colItems      = objWMIService.ExecQuery( strQuery, "WQL", 48 )

For Each objItem In colItems
    If IsArray( objItem.IPAddress ) Then
        If UBound( objItem.IPAddress ) = 0 Then
            strIP = objItem.IPAddress(0)
        Else
            strIP = Join( objItem.IPAddress, "," )
        End If
    End If
Next

Set objWMIService = Nothing
Set colItems      = Nothing


'Compare the IP address with the known location IP address 
Select Case strIP
        Case "xxx.22.0.xxxx" : sPlace  = "HYD" ' QTP HYD System IP Address
        Case "xxxx.168.1.xxx" : sPlace = "NJ"  ' QTP  NJ system
'        Case "xx.xxx.xx.xxx" : sPlace = "MUM"

        Case Else : sPlace = "IP_" & strIP
End Select


' Delete all the files from the folders
set oFso =createObject("scripting.filesystemobject")
folder = sParentFolder & "\ExecutionLog\*.*"
On Error Resume Next
oFso.DeleteFile(folder)
On Error Goto 0
set oFso = Nothing

set oFso=createObject("scripting.filesystemobject")
folder = sParentFolder &  "\ScreenShots\*.*"
On Error Resume Next
oFso.DeleteFile(folder)
On Error Goto 0
set oFso = Nothing


'Call AOM script to activate QTP and run test by passing environment variables as parameters
Dim qtApp
Set qtApp = CreateObject("QuickTest.Application")
If qtApp.Launched Then ' If QuickTest is  open
  qtApp.Quit 
End If
qtApp.Launch
qtApp.Open "C:\Automation FrameWork\Frame_Work GeP"
qtApp.Visible = True
qtApp.WindowState = "Maximized"
qtApp.Test.Environment.Value("envsDataFileSelect") = sControlFilePath
qtApp.Test.Environment.Value("envsPlace") = sPlace
qtApp.Test.Run
qtApp.Test.Close
qtApp.Quit 
Set qtApp = nothing
Set qtResultsObj = nothing


sDateTIme = Now()

iDate = Datepart("d",sDateTime)
iLen = Len(iDate)
If iLen = 1 Then
        iDate = "0" & iDate
End If

sMonth=  mid(MonthName(Datepart("m",sDateTime)),1,3)

iYear = Datepart("yyyy",sDateTime)

iHour = Datepart("h",sDateTime)
iLen = Len(iHour)
If iLen = 1 Then
        iHour = "0" & iHour
End If

iMinute = Datepart("n",sDateTime)
iLen = Len(iMinute)
If iLen = 1 Then
        iMinute = "0" & iMinute
End If

iSec = Datepart("s",sDateTime)
iLen = Len(iSec)
If iLen = 1 Then
        iSec = "0" & iSec
End If


'sFileName =  iDate & "_" & sMonth & "_" & iYear & "_" & iHour & "_" & iMinute & "_" & iSec & "_" & sPlace
sFileName =  sMonth & "_" & iDate & "_" & iYear & "_" & iHour & "_" & iMinute & "_" & iSec & "_" & sPlace
sNewFolder = sToCopyFolder & sFileName

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.CreateFolder(sNewFolder )
Set objFolder = Nothing
Set objFSO = Nothing 

Execute "Const  FOF_CREATEPROGRESSDLG = &H0&"
Set objShell = CreateObject("Shell.Application")
Set objFolder = objShell.NameSpace(sNewFolder) 
objFolder.CopyHere sParentFolder , FOF_CREATEPROGRESSDLG
Set objFolder = Nothing
Set objShell = Nothing


sRenameFolderPath = sNewFolder & "\ProductionLoginTesting"
sRenameFolderPathTestCase = sNewFolder & "\ProductionLoginTesting\AutomationTestCases"

strComputer = "."

Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
abc="ASSOCIATORS OF {Win32_Directory.Name='" & sRenameFolderPath & "'} Where  ResultClass = CIM_DataFile"
Set colFiles = objWMIService.ExecQuery  (abc)
For Each objFile In colFiles
    strEnd = Right(objFile.Name, 4)
    If Ucase(strEnd) = ".XLS" then
    iLen = Len(objFile.Name)
    'MsgBox iLen
    sFolderPath = mid(objFile.Name,1,ilen-4)
    'Msgbox sFolderPath 
    strNewName = sFolderPath & sFileName & ".xls"
        'MsgBox strNewName 
        errResult = objFile.Rename(strNewName)

    End If
Next

Set objWMIService = GetObject("winmgmts:\\" & strComputer & "\root\cimv2")
abc="ASSOCIATORS OF {Win32_Directory.Name='" & sRenameFolderPathTestCase & "'} Where  ResultClass = CIM_DataFile"
Set colFiles = objWMIService.ExecQuery  (abc)
For Each objFile In colFiles
    strEnd = Right(objFile.Name, 4)
    If Ucase(strEnd) = ".XLS" then
    iLen = Len(objFile.Name)
    'MsgBox iLen
    sFolderPath = mid(objFile.Name,1,ilen-4)
    'Msgbox sFolderPath 
    strNewName = sFolderPath & sFileName & ".xls"
        'MsgBox strNewName 
        errResult = objFile.Rename(strNewName)

    End If
Next

Set objWMIService = Nothing

---

QTP - Add library files using AOM script

QTP - Add library files using AOM script

If you have more than 10 library files, it is very difficult to add these files manually from the Test settings and also there is possibility of missing few files.

You can run the following script, so that system will perform your work automatically.

Note: Change folder path and number of library folders(my script use two folders) as per your design.

'QTP Test Path
sQTPpath = "C:\Automation FrameWork\Frame_Work GeP"

'Test Core library
sCoreLib = "C:\Automation FrameWork\Library\CoreLibraries\"

'Test Main library
sGeneralLib = "C:\Automation FrameWork\Library\"


' Don't update any of the below items ******************************************

Set App = CreateObject("QuickTest.Application")
If App.Launched Then ' If QuickTest is  open
        App.Quit 
End If
App.Launch
App.Visible = True
App.WindowState = "Maximized"
App.Open sQTPpath

'Set the Libraries collection object
Set qtLibraries = App.Test.Settings.Resources.Libraries

qtLibraries.RemoveAll

Set fso = CreateObject("Scripting.FileSystemObject")


If sCoreLib = null Then
    Wscript.Echo "No Folder parameter was passed"
    Wscript.Quit
End If

Set folder = fso.GetFolder(sCoreLib)
Set files = folder.Files

For each folderIdx In files
    iLen = Len(folderIdx.Name)
    If Ucase(mid(folderIdx.Name,iLen-3,4)) = ".VBS" Then
            qtLibraries.Add sCoreLib & folderIdx.Name, qtLibraries.Count+1
    End If
Next

If sGeneralLib = null Then
    Wscript.Echo "No Folder parameter was passed"
    Wscript.Quit
End If

Set folder = fso.GetFolder(sGeneralLib)
Set files = folder.Files

For each folderIdx In files
    iLen = Len(folderIdx.Name)
    If Ucase(mid(folderIdx.Name,iLen-3,4)) = ".VBS" Then
            qtLibraries.Add sGeneralLib & folderIdx.Name, qtLibraries.Count+1
    End If
Next



App.Test.Save
Wscript.Echo "Script successfully added " & qtLibraries.Count & " Libraries"

Set app = Nothing
Set qtlibraries = Nothing
Set fso = Nothing
Set folder = Nothing
Set files = Nothing

---

QTP - Hybrid FrameWork - Part I

QTP - Hybrid Automation Frame Work


Initially I have presented this article at SlideShare


What is Automation Framework?

Framework is a wrapper around complex internal architecture which makes end user to interact with the system easily. It also define guidelines and set standards for all phases of ADLC (Automation Development Life Cycle).

This “Dual Function” (Hybrid) framework is a combination of three frameworks (Functional Decomposition, Data Driven, Keyword Driven) and (re-usability, advanced error logic, custom QTP methods, dual function) techniques.


Key features of Dual function Framework

1. Re-usability, low maintenance design (Dual Function).
2. Support different application environments (Development, Quality Control, production) and custom client settings.
3. Externally Configurable - Execute all the test cases, execute only failed test cases, execute test cases by test case id, execute test cases by using flags...can be configured in different ways.
4. Self Configurable and can run unattended with out results being overwritten.
5. Better ROI (Return Of Investment).
6. Real time status monitoring…display detail status during the test execution and automatically close the pop-up status in x sec.
7. Test status reporting in three ways…detailed, high level and summary log file.
8. System automatically send email notifications with test results once the test is complete.
9. Screen Capture on failed test step, notified in the test case itself, assign unique name and stored in the separate folder for easy accessing.
10. Automation test cases resemble manual test cases (Just replace with keywords and test data)
11. Easy way of creating test cases with test case generators (Select keywords from drop down).
12. Easy to maintain and develop scripts (DP, Custom Methods and Dual Function).
13. Test execution time stamp for each step, so that we can easily trace when the test was executed.
14. Same script can be executed on QTP 9.2, 9.5 and 10.0. Backward, forward compatibility.
15. If the test is running in different location, system automatically identify the place of execution using System IP. If the IP is unknown not in the defined list or changed, it would publish the IP address in the results.
16. If object is missing or object properties got changed on the application, system will notify object is missing and proceed with the next test case. This will not stop the test execution.
17. Calculate each page response time automatically.
18. Framework is designed in such a way that it can be initiated using AOM script or directly run using QTP GUI especially useful during debugging or script creation.
19. It would automatically stop automatically if specific number of test cases got failed in sequence, this will help us know there is serious problem in the application.
20. Multi language support.
21 . Sending test results as SMS.
22. It would automatically add new library files.
23. As the entire code reside in library files, total size of the framework is very small. This will help us to easily copy, backup and upload into version control tools.
24. Automatic results archiving facility, old results will not erased.


Advantages of Dual Function Framework and ROI (Return Of Investment)

1. Reduce testing time.
2. Improve testing productivity.
3. Improve product quality
4. Reduce QA costs.
5. Consistent test results.
6. Calculate page response time of all the pages.
7. Schedule test runs.



Frame Work Folder Structure

Framework contain different files associated with different functionality, all the files to be placed in the corresponding folders; so that it is easy to maintain.

Note: Framework can't identify the folders if folder names get renamed, except sub folders created under "AutomationTestDocuments".

AutomationTestDocuments folder contain sub folders relating to each test. If we have two tests, there will be two sub folders. Create new folders as per the test requirement. There is no restriction on the folder name.

Each test (ProductionLoginTesting from the above screen shot) contain 2 Files (Startup, Control) and 3 sub-folders (AutomationTestCases - contain test cases; ExecuationLog - system automatically create two new files once test is complete, based on the configuration, Test summary.txt and Response time.xls; ScreenShots - contain application screen shots, if there are any errors. It would automatically assign unique name with time stamp for each screen shot and notified in the test step itself.

Config folder contain xls files containing custom client settings.

ExecutedScripts folder contain copy of the test folder after execution. If we run the same test again and again (schedule on daily basis) results are over written. In order to preserver the test results, framework will copy the test folder into newly created  folder with time stamp, once the test got completed.

Frame_WorkGep folder contain QTP code of one action "KickOff" with single line of code "Call StartUP". Entire project code code reside in the VBS file.
Following are the reasons for choosing this kind of design.
1. VBS file are very light in terms of size.
2. Framework is entirely based on DP (Descriptive programming).
3. We can easily implement any configuration management tool.

Library folder contain application libraries which contain code relating to keyword functions and contain sub folder called "CoreLibraries" that contain files corresponding to actual framework only.

Dual Function Framework Architecture 

Note: High-level design intended to show in a simple manner. Many other scripts are called between intermediate stages.

QTP AOM Script

1. vbs file created using AOM.

2. Attached to windows scheduler.

3. Identify the place of execution using system IP. If script is executed on new location, it would consider IP address as place of execution.  

4. As there are many tests under "AutomationTestdocuments", script need to be updated with the test folder name and path that we are planning to execute.

5. Script will open the QTP and associate the test specified in the step 4.

6. Two parameters are passed as environment variables (location and test path).

7. Copy the results in "ExecutedScripts" folder to protect results from overwriting, if initiated through AOM script only.

8. Test can also be executed in standalone mode without AOM script, especially useful during test creation and debugging. 

KickOff Action

This is the only QTP action used in the framework, it contain "Call funKickOff" to initiate KickOff script.

KickOff Script

Contain global object declaration, initialization and controlling test execution mode. This script will initiate StartUp script.

StartUp script

This script will configure the framework as per the settings provided in the startup spreadsheet.

Control Script

Start up script will initiate control script. Based on the startup spreadsheet settings, script will read each test case from the control spread sheet and call the driver script with test case id, test case name. This script will also generate real time status of the test in the form of pop-up message and update the test status.

Driver script

Based on the test case id and test case name, it will map the test case lower bound and upper bond, then it will read each line of the test case and call corresponding key word functions. 

How lower bound is computed ?

Based on the test case id received from the control script, it will start verifying the line having the keyword "TestCaseID", if ID under P1 (Column F) is matching, then it is treated as lower bound.

How upper bound is computed ?

Once script identify the lower bound, it will start searching the immediate "STOP" key word, this will be considered as upper bound.

Then it will start reading all the keywords between lower and upper bound by calling corresponding key word functions and updating each step.

 

Application library

These functions will actually manipulate(enter data or perform certain action on the control) the application as per the design and send the status back to the driver script.

To understand different variables that I have created in the framework, look into the function available under this link.

Note: Above scripts contain complex validation and error logic techniques, just gave you high level overview. Following top down approach and each script is independent entity, this will help us to maintain framework with ease and change the functionality without effecting other scripts.

Start-up spreadsheet

Row 1 - Default URL - If you are using the same test in different environments (Development, Quality control, production), just change the URL, scripts would run without any issues. This will help us to run the tests in different environments with very little change.

Row 2 - DB Name - If you are accessing the database through QTP, you can specify the name after creating DSN.

Row 3 - Test Execute Flag - Drop Down values All,Yes,No,Empty - Execute Flag,Check,Test Case Ids

These settings are applied by the control script on the control spreadsheet.

All - Execute all the test cases

Yes(Y) - Execute test case with Y flag

No(N) -   Execute test case with N flag

Empty - Execute Flag - Execute test case with empty flag

Check - When selected, it would just generate test report with consolidate pass/fail status (if test is run multiple time, initially all test cases, next failed test cases) without actual execution the test. 

Test Case Ids - Execute only specific test case ids.

Row 4 - Test Execution Status Flag -  Drop Down values All,Pass,Fail,Bad Inputs,Empty - Execution Flag,Check,Test Case Ids

These settings are applied by the control script on the control spreadsheet.

If we want to control the test based on Pass/Fail status use this control.

All - Execute all the test cases

Pass - Execute test case with Pass status

Fail -   Execute test case with Fail status 

Bad Inputs -   Execute test case with Bad Inputs status 

Empty - Execute Flag - Execute test case with empty flag

Check - When selected, it would just generate test report with consolidate pass/fail status (if test is run multiple time, initially all test cases, next failed test cases) without actual execution the test.

Test Case Ids - Execute specific test case ids.

Row 3 and 4 can be used in different combination while executing the test, so that desired test cases are executed. Most of the combination's are considered.  

Row 5 - Execute Specific Test Case Ids - Mention test case ids that you want to execute, "Test Case Ids" flag need to be selected in Row 3 or 4. This will override all the settings.

Row 6 - Tester Name - Name of the tester who is executing the script, same name appear in control and log files after test execution.

Row 7 - Release no/ Module name.

Row 8 - Test Cycle - Mention test cycle.

Row 9 - Exceptions exist counter - Test would automatically stop once the failed test case count reach the counter value. It doesn't make sense to keep on executing the test with 50 or more failed test cases, some thing gone wrong badly.

Row 10 - Test status pop-up display - Drop down values On,Off,Valid Test Case

On - Always display real time test status pop-up.

Off - Don't display real time test status pop-up.

Valid Test Case - Display while executing valid test case, don't display for invalid test case, determined from the combination of Row 3 and 4 flag settings.

Row 11 - Test status pop-up display (sec) - How may seconds real time test pop-up need to be displayed, automatically close after specified seconds.

Row 17 - Object Highlighting - On/Off

On - It will highlight all the objects during test execution, reduce test execution speed.

Off - No highlighting

Row 18 - Silent Mode - On/Off

On -Framework generate many messages during execution, when test is getting executed unattended, we can suppress these message by selection On.

Off - User need to click OK for each framework message, used during debugging.

Row 19 - Calculate average page response time - On/Off

On - It will generate an excel file with each page response time and store it in the "ExecuationLog" folder. Extra coding required, need to modify excel COM model function as per the requirements.

Off - It will not generate any page response file.

Row 26 - Beep on any click - On/Off

On - Generate beep sound when system select any control. 

Off - No beep.

Reason for implementing this feature - One of the developer told me that you are not calculating page response time correctly there is flaw in page synchronization code. In order to prove that I am correct, I have implemented this feature. When system select any action on the control it will generate a beep, once page sync is complete, it will generate different beep. After showing the same to the developer, he agreed that my page response time code is correct.

Control Spreadsheet

Headers highlighted in YELLOW - User need to fill the columns.

Headers highlighted in WHITE - System automatically fill the columns during execution.

One control spread sheet for entire test.

Column 1 - Test Case Id - Each test case is assigned unique id in ascending order only.

Column 2 - Test Case Description - Optional, small description about the test case.

Column 3 - Execute Flag(Y/N/Empty) - Control script will retrieve the settings from the start up spread sheet Row 3 and check the flag. If both flags match, system will execute the test case else it will ignore it. This system will help us to quickly configure which test cases to be executed.

Column 4 - Test case file name - Need to specify the corresponding test case file name.  Each test can have multiple test cases if required, so that functionality can be distributed and easy to maintain.

Test case - Group of test steps(procedure) with specific test data to validate a requirement.

Test case file - Group of test cases placed in a file.

Column 5 - Response Time - Consolidated response time of all the test steps under specific test case.

Column 6 - Execution Status - Once test case id is executed, system will update the status with Pass/Fail/Bad Inputs (test data provided to the test steps are incorrect).

We can also use this flag to control test execution. Row 4 start up spreadsheet.

Column 7 - Execution Place or System IP / Status Message - Display execution place/ Error and check point messages, if there are any inconsistencies.

Column 8 - Time stamp - Test execution time stamp in Local time.

Column 9 - Test tester name - As specified in the start up spreadsheet.

Column 10 - Release no/ module name - As specified in the start up spreadsheet.

Test case Spreadsheet

Before Execution

 

Headers highlighted in YELLOW - User need to fill the columns.

Headers highlighted in WHITE - System automatically fill the columns during execution.

Test case spreadsheet consist of test steps similar to manual test cases.

Column A - Keyword - Perform specific action on the screen like enter data, retrieve data, check point. These keywords are selected from the drop down, not required to type the keywords. Once test case creator select the keyword, based on the design, column F to Q are highlighted and corresponding cells are inserted with comments, these are input parameters P1, P2, P3....P20. "LogInSignon" (Row 118) contain 4 parameters, P1 - User name, P2 - Password, P3 - Credentials are correct, P4 - Language.

After Execution

System start filling columns B, C, D and F during test execution.

Mapping of manual and automation test cases to check the coverage

 

We can create new column "Automation test case ID" in the manual test cases spreadsheet, so that there will be one to one mapping without missing any functionality.

Test summary log file

Generated and placed in the 'ExecutionLog" folder one test is completed.

It contain following information.
1. Place of execution.
2. Test start and end time with time zone.
3. Total executes test cases (Pass/fail/bad inputs) and ignored test cases (based on start up spreadsheet settings).
4. Star up spreadsheet details.
5. Information relating to sending email notifications.
6. Average page response time (depend on the start up spreadsheet settings)
7. Available PC physical memory before and after execution with time stamp.
8. Test termination message By system completed/ By user/ By exception exist counter.
9. Whether email sent successfully.
10. PC information.

It is a summary of Test results + Framework settings + PC info.




Real-Time status POP-UP message

 

It provide live health of your test, this can be configured in the start up spread sheet.

Above image is self explanatory.

If your test is running for 2 hours and don't have this feature, you will understand nothing if you look at your running test after one hour. It would be better to implement this feature as per your requirements, not so exhaustively as mentioned above. 

Note: Implementing these features will not have any performance bottleneck in terms of execution speed or memory usage.

 Test case generator (Keyword sheet to create test cases)

Simple Sheet

 

Test case generator spreadsheet contain two sheets

1. KeyList - Where you design keywords, same as above screen shot

2. Keyword - Where you create automation test cases by selecting the keywords from the drop down.

Macros are written to connect both sheets

How to create KeyWords?

Lets create keyword for the login screen, where you have User name, Password and Login button.

Flow - Enter user name, Password and select login button.

Keyword name - LoginSignOn 

Input Parameters - User name, Password

Assign two parameters P1, P2 under column E and F.

Create cell comments for P1 and P2 as user name and password.

O- Optional parameter

M - Mandate parameter (System will generate "BAD INPUTS" if users don't enter values and execute the script.

Points to be considered while creating Keywords

1. Standard CamelCase format.

2. Keyword granularity (never have same keyword doing same or overlapping functionality).

3. When creating complex keywords choose following format (product name + page Name + functionality)

4. Don't worry about the length of keyword, you are not typing it, just selecting from the drop down. It should be self explanatory.

Advantages of this design.

1. It is not required to enter test object properties in the test case, already implemented in the code.

2. We can created a document which contain screen shot of the page and corresponding keywords. By looking at it any one can create test cases.

3. Generally in keyword driven framework, we can enter one value or fire one method per line; in this design you can enter maximum of 20 values in a single line. 

4. It is not required to memorize all the inputs associated with a keyword, Just select the Keywork, it would automatically highlight the parameters and assign cell comments to each parameter, so that you will understand easily what P1, P2...parameters for.

5. Each key word is self explanatory designed using camel case, any person can understand it.

Complex Sheet with lengthy keywords

After creating the core framework, next job would be creating keywords as per the functionality and executing the test.





Continued... QTP - Hybrid FrameWork - Part II









---