Results and Reporting
Use these commands and functions to customize the results created by SenseTalk scripts in Eggplant Functional. (Also see The ScriptLogging in Global Properties.)
Behavior: When you run an assert command, Eggplant Functional evaluates the specified statement as true or false, and logs a success, failure, or warning message, or an exception depending on the settings of the Assert Command Properties. See the Quick Reference: Property Lists page for a description of the Assert Command Properties.
set the AssertionBehavior to "warning" //Changes the default behavior of assertions to generate a warning instead of an exception
assert that x < 10 //Creates a LogWarning message if the value of x is less than 10 due to the AssertionBehavior global property set previously
assert that ConfirmationNumber is "128901823" with Exception "Validating the order confirmation number." //Customizes the assertion message
params Platform, Version, DebuggingMode
assert paramCount() is greater than 2 with error //Asserts that at least 3 parameter values were passed into the script, and if not, logs an error
Log the assertionErrorCount&&"assertions have failed." //Logs the total number of assertion errors that have occurred during the execution
Assert that("iOS","Android","Windows","MacOS") contains SUTPlatform //Asserts that the value stored in SUTPlatform is one of 4 acceptable values and if not, throws an exception to stop the execution
put ReadText("ResultsUpperLeft","ResultsLowerRight") into UnfilteredResults //Uses OCR to read a number from the SUT and stores it in variable UnfilteredResults
Tap "Filter" //Performs an action on the SUT that causes a filter to be turned onWaitFor 10, "FilterOn"
put ReadText("ResultsUpperLeft","ResultsLowerRight") into FilteredResults
assert FilteredResults is less than UnfilteredResults with Warning //Asserts that the number read by OCR after applying a filter on the SUT is smaller than before applying the filter
Behavior: The BeginTestCase command is used to open a test case, and the EndTestCase command is used to close one. Each test case is given a name, put in quotes after the command as shown in the examples below.
Login //Runs a script named Login. Note that calling the script with RunWithNewResults instead would automatically create a separate test case for Login
WaitFor 20, "Results"
set myResult to the result
put myResult.testcase & comma & myResult.duration & comma & myResult.starttime & CRLF after file ResourcePath ("myTimings.csv") //Writes the duration of the test case for the "Results" image appearing out to a csv file
BeginTestCase "A" //Marks the start of the reporting period for case A
//Other commands here
EndTestCase "A" //Ends the reporting period for case A and logs the information that was gathered
BeginTestCase "B" //Starts recording information for case B
// other commands here
BeginTestCase "C" //Starts recording information for case C
// other commands here
EndTestCase "B" //Ends case B and reports information gathered since case B was opened
//Other commands here
//If the run ends with test case C still open, it will be ended and reported automatically
A test case is used to block out any section of code within a script (or across scripts during a run) for purposes of gathering information about that portion of the test. A script run may include multiple test cases, which may be sequential portions of a script or may overlap with one another as desired. Information about open test cases for the current script run can be obtained at any time using the OpenTestCases global property.
The information gathered for each test case includes:
- The duration of the test case (in seconds)
- The number of Errors logged while the test case is open
- The number of Successes logged while the test case is open
- The number of Warnings logged while the test case is open
- The number of Exceptions logged while the test case is open
- The name of the test case
- The date and time at which the test case started
Behavior: Captures a snapshot of the entire SUT screen, or a rectangle indicated in the property list. When CaptureScreen is executed as part of a script (versus as a selection or in the Ad Hoc Do Box), the screenshot defaults to saving into the results folder for the script. You can customize this command with a list of any number of the CaptureScreen Properties. See the Quick Reference: Property Lists page for a description of the CaptureScreen Properties.
Parameters: An optional property list.
CaptureScreen(Name: "ImageFileName", Rectangle: ((67, 33), imagelocation("OtherCorner")), increment:true)
//Use a script like this to determine a page has changed from the previous state, when there is no appropriate element to tell you that you are seeing the new state.
Repeat at least once until not imagefound(image:RxResult, waitFor:0) //A special repeat loop that ensures at least one iteration, ignoring the "not imagefound()" condition on the first iteration only. On subsequent iterations, the imagefound function will search for the screenshot captured by the CaptureScreen command
put ImageLocation("RxNumberUpperLeft") into RxLocation //Stores the hot spot location of image RxNumberUpperLeft in variable RxLocation
CaptureScreen(name:"PrescriptionNumber", Rectangle:(RxLocation, RxLocation + (140,100))) //Captures a screenshot of the unique number displayed to the right of the RxLocation
put the result into RxResult //Stores the full file path of the CaptureScreen screenshot
Click "ClosePrescription" //Performs an action that will cause the screen to change
if the repeatIndex = 5 then Throw "Image not Found","Closing the current prescription did not work." //Throws an exception if the repeat loop repeats 5 times
//Use a script like this to detect when you've scrolled to the bottom of the page
put RemoteScreenSize() into DUTScreenSize //Stores the resolution of the SUT in a variable called DUTScreenSize
put (.25*DUTScreenSize.x,.25*DUTScreenSize.y,.75*DUTScreenSize.x,.75*DUTScreenSize.y) into ClippingRectangle //Creates a rectangle that is a subset of the full SUT size. Using a subset helps avoid capturing dynamic elements on the screen, such as the clock or notifications area
CaptureScreen (Name: "state", Rectangle: ClippingRectangle) //Captures a screenshot of the SUT using the ClippingRectangle coordinates
put the result into refImage //Stores the file path of the saved CaptureScreen screenshot into a variable for easy referencing later
repeat at least once while not imageFound (image:refImage, searchRectangle:ClippingRectangle, waitFor:0) //Searches for the CaptureScreen image in the same location on the screen where it was originally captured and ensures that at least one scrolling action is executed against the SUT. Exits the repeat loop in the event that the CaptureScreen image is present inside the searchRectangle following a scrolling event
CaptureScreen (Name: "state", Rectangle: ClippingRectangle) //Creates a new CaptureScreen image to reflect the fact that a scroll action has occurred
put the result into refImage
SwipeUp //Repeats a scrolling action to scroll down the page
Wait 2.5 //A wait time is generally required to allow the UI to stop moving before continuing
//Use a script like this to determine whether an element on the screen, such as a video player, is changing
put imageLocation("VideoNameHeader") into VideoLoc //Stores the location of an reference image for the changing element
put (VideoLoc,VideoLoc+(250,230)) into myRectangle //Uses the location stored in VideoLoc, and an offset of 250 pixels to the right and 230 pixels down, to create a rectangle
CaptureScreen(name:"VideoCheck",rectangle:myRectangle) //Uses the rectangle stored in myRectangle to specify the subset of the screen captured by CaptureScreen
put the result into imageFilePath
set the searchRectangle to myRectangle //Limits the search area to the same subset of the screen used by the CaptureScreen
repeat until not imagefound (image:imageFilePath, waitfor:0) //Repeats searching for the CaptureScreen image until it is not showing
Wait 3 //Allows the element some additional time to change before searching again for the CaptureScreen image
if the repeatIndex is greater than 10 then Throw "Video stream exception","The video is not advancing." //Provides a way to stop the script execution in the event that the element doesn't change after 10 iterations of the repeat loop
set the searchRectangle to empty //Sets the searchRectangle back to full screen
Tip: The result of CaptureScreen is the full file path of the screenshot.
put the result
Behavior: Returns the color value of the pixel in the given location. Coordinates are relative to the top-left corner of the SUT. The top-left corner is (0,0), with the x value increasing toward the right, and the y value increasing toward the bottom.
Parameters: One coordinate location, image, or text (OCR) reference.
Returns: The color value of the pixel in the given location. For more information, see Working with Color, in the
if imagefound(Text:"Document") then
put colorAtLocation( the topleft of FoundImageInfo().imagerectangle) into myColor //Determine the RGB value of a pixel 5 pixels to the left of the top left corner of string "Document" on the SUT
if myColor=color("black") then //Uses the color function to return the RGB value of black and compare it to the actual RGB value
LogSuccess "Black background"
LogWarning "Background color" && myColor && "unexpected."
Set the ColorFormat to "HTML" //Changes the color format returned by ColorAtLocation() and Color() to hexadecimal
put ColorAtLocation ("StatusIndicator") into myColor
If myColor isn't "#AFF7DF" then LogError "Color is not #AFF7DF. Color is " &myColor&period
Behavior: Use CompareScreen as either a command or function to check for changes to a screen or window. Typically, you would call CompareScreen when the script has just performed an action on the SUT that results in displaying a new screen or window. When differences are detected, you use the Compare Screen panel in Eggplant Functional to determine how to proceed.
CompareScreen visually compares the current SUT screen against a baseline image, which is specified as the first parameter. If the baseline image doesn't exist, it is automatically captured the first time CompareScreen is executed for that image name.
If you use CompareScreen without specifying a rectangle, the comparison applies to the entire current SUT screen. Optionally, you can specify a rectangle so that only a region of the screen (such as a single application window) is included in the comparison. If the rectangle given (or the size of the current SUT screen) is different from the baseline reference image, an exception is thrown.
When CompareScreen is executed, the script locates the baseline image in the special [Baseline_Screens] folder within the Images folder of the suite (or in an active helper suite). If no image by that name is found, an initial baseline image is captured and stored automatically, then the script proceeds without any further action.
Note: The [Baseline_Screens] folder is created automatically the first time you run CompareScreen in a suite. You do not need to create this folder yourself.
If an existing baseline image is present, that image is compared to the current SUT screen (or to the specified rectangle of the SUT screen). The comparison is performed using the search criteria (e.g., search type, tolerance) of the baseline image. Note that you can open the baseline image from the Images pane of the suite and adjust these properties if desired, which will affect future comparisons.
When you call CompareScreen as a command, the behavior following the comparison is controlled by the Compare Screen Action mode, which can be set on the Run menu in Eggplant Functional (Run > Compare Screen) or by using the CompareScreenAction global property. The Compare Screen Action mode can be set to:
- Show Panel: If a screen difference is detected, the Compare Screen panel opens.
- Log Warning: If a screen difference is detected, a warning message is entered in the log.
- Log Error: If a screen difference is detected, an error message is entered in the log.
- Show Panel Always: The Compare Screen panel opens with every CompareScreen command, regardless of whether any screen difference is detected.
For information about using the Compare Screen panel, see Using Compare Screen in Eggplant Functional for UI Testing.
When you call CompareScreen as a function, the setting of the Compare Screen Action mode is ignored. The Compare Screen panel is never displayed. If no significant differences are found, the function returns an empty list.
If there are differences between the baseline image and the current SUT screen, the function records information about the differences in the run log so that the differences can be reviewed later from the Results pane. It also returns a list of property lists, where each screen difference is one property list with these properties:
- difference: New, Missing, Moved, Changed
- differenceType: Text, Element, Area, DynamicText, DynamicContent
- source: Baseline, Current, Combined
- rectangle: SenseTalk rectangle on baseline
When differenceType is Text, these properties are also present:
- text: baseline text
- textNow: current text
When difference is Moved, this property is also included:
- rectangleMovedTo: SenseTalk rectangle of destination on current screen
The following combinations of difference and differenceType are possible:
- New: differenceType can be Text or Element
- Missing: differenceType can be Text, Element, DynamicText, or DynamicContent
- Moved: differenceType can be Text or Element
- Changed: differenceType can be Text, Element, or Area
Note: When you call CompareScreen as a command, the list of differences is available within the script by calling the result() function on the very next line of the script.
Parameters: An image name and (optional) specified rectangle.
CompareScreen "baselineName", rectangle:<rectangle>
CompareScreen name: "baselineName", rectangle:<rectangle>
put CompareScreen("baselineName", rectangle:<rectangle>)
put CompareScreen(name: "baselineName", rectangle:<rectangle>)
// CompareScreen command for a full-screen compare, captures the image the first time it's run
// CompareScreen command for a specific rectangle comparison
CompareScreen name: "tutorialSut_menu", rectangle: (22,575,415,640)
// CompareScreen command for a specific rectangle defined based on images, then uses the result() funtion to save a list of changes
CompareScreen "calcHist", rectangle:("Calculator_A","Calculator_B")
put the result.archive into ChangeHist
// The results of CompareScreen are stored in the variable
put CompareScreen("HomeScreen") into homeDiffs
// Logs a message if differences are detected
if CompareScreen("LoginPanel", rectangle:(100,100,500,400)) isn't empty then log "There were differences"
// Creates a log message showing the results from CompareScreen
log "Diffs: " & CompareScreen(Name: "NewAccount", Rectangle:("AcctWindowTopLeft","AcctWindowBottomRight"))
Behavior: Creates an entry in the script’s Log file. If you set multiple message parameters, each message is inserted as a separate entry. (If the ScriptLogging global property value is Minimal or Silent, this command does nothing.)
Parameters: One or more Log messages.
Log "Beginning the login sequence now."
log "Today is" &&the long date & period //Concatenates a string with the output from the long date function
log "This is my first message.", "This is my second message" //Logs each message with a separate log entry in the results
Behavior: Behaves just like the Log and LogWarning commands, but displays the log entries in red text, and logs the script as a failure.
Parameters: One or more Log messages.
LogError "This is my error message."
If imageFound("ExceptionDialog") then
CaptureScreen //Creates a screenshot at the point of failure, as LogError does not do so automatically like the ImageNotFound exception does
LogError "Known critical error occurred. Ending execution."
exit all //Terminates execution of the current handler, the handler that called it, and all other handlers further up the call stack
Behavior: Behaves just like the Log commands, but adds one to the success count for the script.
Parameters: One or more Log messages.
LogSuccess "The page loaded properly at" &&the short time&"."
Behavior: Behaves just like the Log commands, but displays the log entries in orange text.
Parameters: One or more Log messages.
LogWarning "This script is executing at "&the long time&"."
if ImageFound("CommunicationMessage") then LogWarning "Known bug #18922 has occurred." //Logs a warning message only if a particular image is found during a particular step in the automation
Behavior: Returns a results property list from every run of the given script, in chronological order. See the Quick Reference: Property Lists page for a description of the results property list.
Parameters: One script name. If no script is named, the function defaults to the current script.
Returns: A list of results property lists from every run of the given script.
put ScriptResults("Login") into LoginResults //Puts the list of the results property lists for all runs of Login into a variable named LoginResults
put the logfile of the last item of scriptResults("DemoScript") into myResult //Takes the value of the logfile property of the most recent (last) set of results for script "DemoScript", and stores it in a variable
delete "/LogFile.txt" from myResult //Manipulates the string for the file path to the logfile such that it becomes the string for the results folder path
copy folder myResult to "C:\Users\Carrie\Desktop\TodaysResults" //Copies the result folder to a new location on the desktop
Log "These are the historic results of Login"
repeat with each item Run of scriptResults("Login") //Iterates through the property lists for each run of script "Login"
Log the date of Run.RunDate&&the time of Run.RunDate&&Run.status //Logs the date, time, and status for each run of Login
put the logFile of the last item of scriptResults("MasterScript") into myResult //*Puts the results property list for the most recent execution of MasterScript into a variable.*
delete "Logfile.txt" from myResult //Manipulates the string for the file path to the logfile such that it becomes the string for the results folder path
CaptureScreen(name:myResult&"WelcomeScreenUI") //Specifies the save location of the CaptureScreen screenshot based on myResult
Tip:The ScriptResults() results property list is automatically passed to Post-Run scripts. Sometimes the return is best used in the context of a Post-Run script.
//This is the code for the main script
insert the result after returnVar //Adds "the result" (the file path) for the capturescreen as a new item to the end of the list in variable returnVar
return returnVar //Returns the value of returnVar, so that it is included with the ReturnValue property of the ScriptResults(). Note that the ReturnValue property can only return strings and not true lists, so later processing is required to make the return appear as a list
//The main script ends
//This is the code for the post-run script
params scriptPath, myScriptResults//Receives the two default parameters passed into the post-run script, the second of which is scriptResults()
//The next 3 lines of code do some processing of the value in returnList to format it as a list
put myScriptResults.ReturnValuesplit by","into returnList
delete "(" from returnList
delete ")" from returnList
put myScriptResults.Errors into errors//Stores the value of the Errors property into a variable called errors
SendMail(to: "firstname.lastname@example.org", Subject:title & " - Errors= " & errors, Message:"Find the screenshots from the Eggplant test script attached. Number of errors: " & errors, attachment:returnList) //Sends an email with the number of errors in the subject and with the screenshots from the main script as attachments
Behavior: Sends email from within a script. The parameters are contained in a SendMail property list. See the Quick Reference: Property Lists page for a description of the SendMail property list.
Parameters: One property list, described below.
sendmail(to: "email@example.com, firstname.lastname@example.org", subject: "Test failed", body: "The very important test script generated an error. The log file is attached.", attachment: logfile)
put the result into Outcome //Stores the result property list from the RunWithNewResults in variable Outcome
//Displaying the different sendmail properties on different lines can make the code more readable
subject:"Login script results:"&&Outcome's status,\
body:"Find attached the logfile for the script run.",\
RunWithNewResults TestScript // TestScript is a variable containing the name of the script to run
put the result into Outcome
put the long name of each of the files of the folder of Outcome's logfile where each ends with ".png" into myfiles // Creates a list containing the file paths to all of the screenshots in the results
SendMail (To:"email@example.com",Subject:title & "---Test results" ,body:TestScript & "Results"& Outcome,attachment:Outcome's Logfile &&& myFiles)
Behavior: Starts recording a movie of the SUT. If you do not designate a file path, the movie is saved in the suite’s Results directory. Movies are saved with file extension .mkv. If you do not designate a file path, the movie is saved in the results folder for the script run. If you do not specify a frame rate, movies are recorded at 10 frames per second.
ExtraTime continues the recording for a given number of seconds after the StopMovie command executes. The default is five seconds. ImageHighlighting is a boolean value that determines whether or not search rectangles are highlighted in the movie. This setting overrides the Image Highlighting setting from the Run menu.
Parameters: A file name for the movie you are capturing and optional file path. FramesPerSecond, ExtraTime, and imageHighlighting are also optional.
StartMovie "/User/Documents/BugMovie" //Begins recording a movie with the name BugMovie
StartMovie "/User/Documents/BugMovie", framesPerSecond:5, extraTime:10, imageHighlighting:Yes
if runningFromCommandLine() then //Determines whether the script is running via command line only execution
Behavior: Ends a movie recording that was started by a script.
Behavior: Turns TraceScreen mode on and off. When TraceScreen mode is on, a full image of the SUT is captured into the results immediately before the execution of each command that acts on the SUT. These screen captures are available in the Log Area of the Suite Editor’s Results pane, and are stored in the result's folder for the script.
Parameters: On or Off.
Run "SearchStore" //Only events that occur in the SearchStore script will have screen tracing