Results and Reporting

Use these commands and functions to customize the results created by SenseTalk scripts in Eggplant Functional. (Also see The ScriptLogging in Global Properties.)

Assert Command

Behavior: When you run an assert command, Eggplant Functional evaluates the specified statement as true or false, and logs a success, failure, or warning message, or an exception depending on the settings of the Assert Command Properties. See the Quick Reference: Property Lists page for a description of the Assert Command Properties.

Parameters: A statement you want Eggplant Functional to evaluate. To improve readability, you can insert the word that into an assert command as shown in the first example below.

Syntax:

assert that statement

Example:

set the AssertionBehavior to "warning" //Changes the default behavior of assertions to generate a warning instead of an exception

assert that x < 10 //Creates a LogWarning message if the value of x is less than 10 due to the AssertionBehavior global property set previously

Example:

assert that ConfirmationNumber is "128901823" with Exception "Validating the order confirmation number." //Customizes the assertion message

Example:

params Platform, Version, DebuggingMode

assert paramCount() is greater than 2 with error //Asserts that at least 3 parameter values were passed into the script, and if not, logs an error

Log the assertionErrorCount&&"assertions have failed." //Logs the total number of assertion errors that have occurred during the execution

Example:

Assert that("iOS","Android","Windows","MacOS") contains SUTPlatform //Asserts that the value stored in SUTPlatform is one of 4 acceptable values and if not, throws an exception to stop the execution

Example:

put ReadText("ResultsUpperLeft","ResultsLowerRight") into UnfilteredResults //Uses OCR to read a number from the SUT and stores it in variable UnfilteredResults

Tap "Filter" //Performs an action on the SUT that causes a filter to be turned onWaitFor 10, "FilterOn"

put ReadText("ResultsUpperLeft","ResultsLowerRight") into FilteredResults

assert FilteredResults is less than UnfilteredResults with Warning //Asserts that the number read by OCR after applying a filter on the SUT is smaller than before applying the filter

BeginTestCase and EndTestCase Commands

Behavior: The BeginTestCase command is used to open a test case, and the EndTestCase command is used to close one. Each test case is given a name, put in quotes after the command as shown in the examples below.

Syntax:

BeginTestCase "CaseName"

action

EndTestCase "CaseName"

Example:

BeginTestCase "SubmitLogin"

Login //Runs a script named Login. Note that calling the script with RunWithNewResults instead would automatically create a separate test case for Login

EndTestCase "SubmitLogin"

Example:

Click "SearchButton"

BeginTestCase "ResultsLoaded"

WaitFor 20, "Results"

EndTestCase "ResultsLoaded"

set myResult to the result

put myResult.testcase & comma & myResult.duration & comma & myResult.starttime & CRLF after file ResourcePath ("myTimings.csv") //Writes the duration of the test case for the "Results" image appearing out to a csv file

Example:

BeginTestCase "A" //Marks the start of the reporting period for case A

//Other commands here

EndTestCase "A" //Ends the reporting period for case A and logs the information that was gathered

BeginTestCase "B" //Starts recording information for case B

// other commands here

BeginTestCase "C" //Starts recording information for case C

// other commands here

EndTestCase "B" //Ends case B and reports information gathered since case B was opened

//Other commands here

//If the run ends with test case C still open, it will be ended and reported automatically

Tech Talk

A test case is used to block out any section of code within a script (or across scripts during a run) for purposes of gathering information about that portion of the test. A script run may include multiple test cases, which may be sequential portions of a script or may overlap with one another as desired. Information about open test cases for the current script run can be obtained at any time using the OpenTestCases global property.

The information gathered for each test case includes:

  • The duration of the test case (in seconds)
  • The number of Errors logged while the test case is open
  • The number of Successes logged while the test case is open
  • The number of Warnings logged while the test case is open
  • The number of Exceptions logged while the test case is open
  • The name of the test case
  • The date and time at which the test case started
Note: One test case is created automatically at the beginning of each run, with the name of the script being run.
Note: At the end of the run, any test cases that are still open are ended and reported automatically.
Note: The initial handler (the script itself) is considered a test case and will be reported as one If no declared test cases are open or no test cases are declared at all, the script will be reported as the only open test case.
Note: Test cases can overlap.
Tip: Beginning in version 5.2, Eggplant Manager automatically parses test case data from script results and displays the duration for the test cases in a dashboard chart. Parsed test case data is exportable from Eggplant Manager.

CaptureScreen Command

Behavior: Captures a snapshot of the entire SUT screen, or a rectangle indicated in the property list. When CaptureScreen is executed as part of a script (versus as a selection or in the Ad Hoc Do Box), the screenshot defaults to saving into the results folder for the script. You can customize this command with a list of any number of the CaptureScreen Properties. See the Quick Reference: Property Lists page for a description of the CaptureScreen Properties.

Parameters: An optional property list.

Syntax:

CaptureScreen

Example:

CaptureScreen(Name: "ImageFileName", Rectangle: ((67, 33), imagelocation("OtherCorner")), increment:true)

Example:

//Use a script like this to determine a page has changed from the previous state, when there is no appropriate element to tell you that you are seeing the new state.

 

Repeat at least once until not imagefound(image:RxResult, waitFor:0) //A special repeat loop that ensures at least one iteration, ignoring the "not imagefound()" condition on the first iteration only. On subsequent iterations, the imagefound function will search for the screenshot captured by the CaptureScreen command

put ImageLocation("RxNumberUpperLeft") into RxLocation //Stores the hot spot location of image RxNumberUpperLeft in variable RxLocation

CaptureScreen(name:"PrescriptionNumber", Rectangle:(RxLocation, RxLocation + (140,100))) //Captures a screenshot of the unique number displayed to the right of the RxLocation

put the result into RxResult //Stores the full file path of the CaptureScreen screenshot

Click "ClosePrescription" //Performs an action that will cause the screen to change

wait 1

if the repeatIndex = 5 then Throw "Image not Found","Closing the current prescription did not work." //Throws an exception if the repeat loop repeats 5 times

end repeat

Example:

CaptureScreen(name:SuiteInfo().ImagesFolder&slash&"CurrentLogo",rectangle:("WelcomeImage","Border"))

Example:

//Use a script like this to detect when you've scrolled to the bottom of the page

put RemoteScreenSize() into DUTScreenSize //Stores the resolution of the SUT in a variable called DUTScreenSize

put (.25*DUTScreenSize.x,.25*DUTScreenSize.y,.75*DUTScreenSize.x,.75*DUTScreenSize.y) into ClippingRectangle //Creates a rectangle that is a subset of the full SUT size. Using a subset helps avoid capturing dynamic elements on the screen, such as the clock or notifications area

CaptureScreen (Name: "state", Rectangle: ClippingRectangle) //Captures a screenshot of the SUT using the ClippingRectangle coordinates

put the result into refImage //Stores the file path of the saved CaptureScreen screenshot into a variable for easy referencing later

repeat at least once while not imageFound (image:refImage, searchRectangle:ClippingRectangle, waitFor:0) //Searches for the CaptureScreen image in the same location on the screen where it was originally captured and ensures that at least one scrolling action is executed against the SUT. Exits the repeat loop in the event that the CaptureScreen image is present inside the searchRectangle following a scrolling event

CaptureScreen (Name: "state", Rectangle: ClippingRectangle) //Creates a new CaptureScreen image to reflect the fact that a scroll action has occurred

put the result into refImage

SwipeUp //Repeats a scrolling action to scroll down the page

Wait 2.5 //A wait time is generally required to allow the UI to stop moving before continuing

end repeat

Example:

//Use a script like this to determine whether an element on the screen, such as a video player, is changing

put imageLocation("VideoNameHeader") into VideoLoc //Stores the location of an reference image for the changing element

put (VideoLoc,VideoLoc+(250,230)) into myRectangle //Uses the location stored in VideoLoc, and an offset of 250 pixels to the right and 230 pixels down, to create a rectangle

CaptureScreen(name:"VideoCheck",rectangle:myRectangle) //Uses the rectangle stored in myRectangle to specify the subset of the screen captured by CaptureScreen

put the result into imageFilePath

set the searchRectangle to myRectangle //Limits the search area to the same subset of the screen used by the CaptureScreen

repeat until not imagefound (image:imageFilePath, waitfor:0) //Repeats searching for the CaptureScreen image until it is not showing

Wait 3 //Allows the element some additional time to change before searching again for the CaptureScreen image

if the repeatIndex is greater than 10 then Throw "Video stream exception","The video is not advancing." //Provides a way to stop the script execution in the event that the element doesn't change after 10 iterations of the repeat loop

end repeat

set the searchRectangle to empty //Sets the searchRectangle back to full screen

Note: Running the CaptureScreen command as part of a selection or in the Ad Hoc Do Box results in the screenshot saving in your local user's document directory (e.g., C:/Users/<username>/Documents/)
Tip: Incrementing is helpful if you are capturing several snapshots within a script (as in a loop or frequently called handler). The images can all have the same image name, with the incremented number distinguishing them; otherwise, each image would overwrite the previous image with the same name.

Tip: The result of CaptureScreen is the full file path of the screenshot.

Example:

CaptureScreen

put the result

ColorAtLocation Function

Behavior: Returns the color value of the pixel in the given location. Coordinates are relative to the top-left corner of the SUT. The top-left corner is (0,0), with the x value increasing toward the right, and the y value increasing toward the bottom.

Parameters: One coordinate location, image, or text (OCR) reference.

Returns: The color value of the pixel in the given location. For more information, see Working with Color, in the SenseTalk Reference Manual.

Syntax:

log ColorAtLocation(coordinate1,coordinate2)

Example:

log ColorAtLocation(593,110)

Example:

if imagefound(Text:"Document") then

put colorAtLocation( the topleft of FoundImageInfo().imagerectangle) into myColor //Determine the RGB value of a pixel 5 pixels to the left of the top left corner of string "Document" on the SUT

if myColor=color("black") then //Uses the color function to return the RGB value of black and compare it to the actual RGB value

LogSuccess "Black background"

else

LogWarning "Background color" && myColor && "unexpected."

end if

end if

Example:

Set the ColorFormat to "HTML" //Changes the color format returned by ColorAtLocation() and Color() to hexadecimal

put ColorAtLocation ("StatusIndicator") into myColor

If myColor isn't "#AFF7DF" then LogError "Color is not #AFF7DF. Color is " &myColor&period

CompareScreen Command or Function

Behavior: Use CompareScreen as either a command or function to check for changes to a screen or window. Typically, you would call CompareScreen when the script has just performed an action on the SUT that results in displaying a new screen or window. When differences are detected, you use the Compare Screen panel in Eggplant Functional to determine how to proceed.

CompareScreen visually compares the current SUT screen against a baseline image, which is specified as the first parameter. If the baseline image doesn't exist, it is automatically captured the first time CompareScreen is executed for that image name.

If you use CompareScreen without specifying a rectangle, the comparison applies to the entire current SUT screen. Optionally, you can specify a rectangle so that only a region of the screen (such as a single application window) is included in the comparison. If the rectangle given (or the size of the current SUT screen) is different from the baseline reference image, an exception is thrown.

When CompareScreen is executed, the script locates the baseline image in the special [Baseline_Screens] folder within the Images folder of the suite (or in an active helper suite). If no image by that name is found, an initial baseline image is captured and stored automatically, then the script proceeds without any further action.

Note: The [Baseline_Screens] folder is created automatically the first time you run CompareScreen in a suite. You do not need to create this folder yourself.

If an existing baseline image is present, that image is compared to the current SUT screen (or to the specified rectangle of the SUT screen). The comparison is performed using the search criteria (e.g., search type, tolerance) of the baseline image. Note that you can open the baseline image from the Images pane of the suite and adjust these properties if desired, which will affect future comparisons.

When you call CompareScreen as a command, the behavior following the comparison is controlled by the Compare Screen Action mode, which can be set on the Run menu in Eggplant Functional (Run > Compare Screen) or by using the CompareScreenAction global property. The Compare Screen Action mode can be set to:

  • Show Panel: If a screen difference is detected, the Compare Screen panel opens.
  • Log Warning: If a screen difference is detected, a warning message is entered in the log.
  • Log Error: If a screen difference is detected, an error message is entered in the log.
  • Show Panel Always: The Compare Screen panel opens with every CompareScreen command, regardless of whether any screen difference is detected.

For information about using the Compare Screen panel, see Using Compare Screen in Eggplant Functional for UI Testing.

When you call CompareScreen as a function, the setting of the Compare Screen Action mode is ignored. The Compare Screen panel is never displayed. If no significant differences are found, the function returns an empty list.

If there are differences between the baseline image and the current SUT screen, the function records information about the differences in the run log so that the differences can be reviewed later from the Results pane. It also returns a list of property lists, where each screen difference is one property list with these properties:

  • difference: New, Missing, Moved, Changed
  • differenceType: Text, Element, Area, DynamicText, DynamicContent
  • source: Baseline, Current, Combined
  • rectangle: SenseTalk rectangle on baseline

When differenceType is Text, these properties are also present:

  • text: baseline text
  • textNow: current text

When difference is Moved, this property is also included:

  • rectangleMovedTo: SenseTalk rectangle of destination on current screen

The following combinations of difference and differenceType are possible:

  • New: differenceType can be Text or Element
  • Missing: differenceType can be Text, Element, DynamicText, or DynamicContent
  • Moved: differenceType can be Text or Element
  • Changed: differenceType can be Text, Element, or Area

Note: When you call CompareScreen as a command, the list of differences is available within the script by calling the result() function on the very next line of the script.

Parameters: An image name and (optional) specified rectangle.

Syntax:

As command:

CompareScreen "baselineName"

CompareScreen "baselineName", rectangle:<rectangle>

CompareScreen name: "baselineName", rectangle:<rectangle>

As function:

put CompareScreen("baselineName")

put CompareScreen("baselineName", rectangle:<rectangle>)

put CompareScreen(name: "baselineName", rectangle:<rectangle>)

Example:

// CompareScreen command for a full-screen compare, captures the image the first time it's run

CompareScreen "tutorialSut"

Example:

// CompareScreen command for a specific rectangle comparison

CompareScreen name: "tutorialSut_menu", rectangle: (22,575,415,640)

Example:

// CompareScreen command for a specific rectangle defined based on images, then uses the result() funtion to save a list of changes

CompareScreen "calcHist", rectangle:("Calculator_A","Calculator_B")

put the result.archive into ChangeHist

Example:

// The results of CompareScreen are stored in the variable

put CompareScreen("HomeScreen") into homeDiffs

Example:

// Logs a message if differences are detected

if CompareScreen("LoginPanel", rectangle:(100,100,500,400)) isn't empty then log "There were differences"

Example:

// Creates a log message showing the results from CompareScreen

log "Diffs: " & CompareScreen(Name: "NewAccount", Rectangle:("AcctWindowTopLeft","AcctWindowBottomRight"))

Related:

Log Command

Behavior: Creates an entry in the script’s Log file. If you set multiple message parameters, each message is inserted as a separate entry. (If the ScriptLogging global property value is Minimal or Silent, this command does nothing.)

Parameters: One or more Log messages.

Example:

Log "Beginning the login sequence now."

Example:

log "Today is" &&the long date & period //Concatenates a string with the output from the long date function

Example:

log "This is my first message.", "This is my second message" //Logs each message with a separate log entry in the results

Tip: Log entries can make it easier to determine where particular log events are occurring in the run.

LogError Command

Behavior: Behaves just like the Log and LogWarning commands, but displays the log entries in red text, and logs the script as a failure.

Parameters: One or more Log messages.

Syntax:

LogError "This is my error message."

Example:

LogError "This is my error message."

Example:

If imageFound("ExceptionDialog") then

CaptureScreen //Creates a screenshot at the point of failure, as LogError does not do so automatically like the ImageNotFound exception does

LogError "Known critical error occurred. Ending execution."

exit all //Terminates execution of the current handler, the handler that called it, and all other handlers further up the call stack

end if

Note: If the ScriptLogging global property is set to Silent, no error messages are inserted into the log file, but the error count for the run is still be incremented, and the ultimate status of the run is Failure.

LogSuccess Command

Behavior: Behaves just like the Log commands, but adds one to the success count for the script.

Parameters: One or more Log messages.

Syntax:

LogSuccess

Example:

LogSuccess "The page loaded properly at" &&the short time&"."

LogWarning Command

Behavior: Behaves just like the Log commands, but displays the log entries in orange text.

Parameters: One or more Log messages.

Syntax:

LogWarning "Warning text"

Example:

LogWarning "This script is executing at "&the long time&"."

Example:

if ImageFound("CommunicationMessage") then LogWarning "Known bug #18922 has occurred." //Logs a warning message only if a particular image is found during a particular step in the automation

Tip: The Results pane of the Suite Editor displays the number of warnings associated with each script run. Warnings are counted even when ScriptLogging is set to Minimal or Silent.

ScriptResults Function

Behavior: Returns a results property list from every run of the given script, in chronological order. See the Quick Reference: Property Lists page for a description of the results property list.

Parameters: One script name. If no script is named, the function defaults to the current script.

Returns: A list of results property lists from every run of the given script.

Syntax:

put ScriptResults("Results") into Results_name

Example:

put ScriptResults("Login") into LoginResults //Puts the list of the results property lists for all runs of Login into a variable named LoginResults

Example:

put the logfile of the last item of scriptResults("DemoScript") into myResult //Takes the value of the logfile property of the most recent (last) set of results for script "DemoScript", and stores it in a variable

delete "/LogFile.txt" from myResult //Manipulates the string for the file path to the logfile such that it becomes the string for the results folder path

copy folder myResult to "C:\Users\Carrie\Desktop\TodaysResults" //Copies the result folder to a new location on the desktop

Example:

Log "These are the historic results of Login"

repeat with each item Run of scriptResults("Login") //Iterates through the property lists for each run of script "Login"

Log the date of Run.RunDate&&the time of Run.RunDate&&Run.status //Logs the date, time, and status for each run of Login

end repeat

Example:

put the logFile of the last item of scriptResults("MasterScript") into myResult //*Puts the results property list for the most recent execution of MasterScript into a variable.*

delete "Logfile.txt" from myResult //Manipulates the string for the file path to the logfile such that it becomes the string for the results folder path

CaptureScreen(name:myResult&"WelcomeScreenUI") //Specifies the save location of the CaptureScreen screenshot based on myResult

Tip:The ScriptResults() results property list is automatically passed to Post-Run scripts. Sometimes the return is best used in the context of a Post-Run script.

Example:

//This is the code for the main script

Login

capturescreen increment:yes

insert the result after returnVar //Adds "the result" (the file path) for the capturescreen as a new item to the end of the list in variable returnVar

return returnVar //Returns the value of returnVar, so that it is included with the ReturnValue property of the ScriptResults(). Note that the ReturnValue property can only return strings and not true lists, so later processing is required to make the return appear as a list

//The main script ends

Example:

//This is the code for the post-run script

params scriptPath, myScriptResults//Receives the two default parameters passed into the post-run script, the second of which is scriptResults()

//The next 3 lines of code do some processing of the value in returnList to format it as a list

put myScriptResults.ReturnValuesplit by","into returnList

delete "(" from returnList

delete ")" from returnList

put myScriptResults.Errors into errors//Stores the value of the Errors property into a variable called errors

SendMail(to: "test@gmail.com", Subject:title & " - Errors= " & errors, Message:"Find the screenshots from the Eggplant test script attached. Number of errors: " & errors, attachment:returnList) //Sends an email with the number of errors in the subject and with the screenshots from the main script as attachments

SendMail Command

Behavior: Sends email from within a script. The parameters are contained in a SendMail property list. See the Quick Reference: Property Lists page for a description of the SendMail property list.

Parameters: One property list, described below.

Syntax:

SendMail(to: "address1,address2", subject: "Subject", body: "Message body", attachment: file_name)

Example:

sendmail(to: "testmanager@somecompany.com, tester1@somecompany.com", subject: "Test failed", body: "The very important test script generated an error. The log file is attached.", attachment: logfile)

Example:

RunWithNewResults "Login"

put the result into Outcome //Stores the result property list from the RunWithNewResults in variable Outcome

//Displaying the different sendmail properties on different lines can make the code more readable

SendMail(To: "test@gmail.com",\

subject:"Login script results:"&&Outcome's status,\

body:"Find attached the logfile for the script run.",\

attachment:(Outcome's Logfile))

Example:

RunWithNewResults TestScript // TestScript is a variable containing the name of the script to run

put the result into Outcome

put the long name of each of the files of the folder of Outcome's logfile where each ends with ".png" into myfiles // Creates a list containing the file paths to all of the screenshots in the results

SendMail (To:"test@gmail.com",Subject:title & "---Test results" ,body:TestScript & "Results"& Outcome,attachment:Outcome's Logfile &&& myFiles)

Important: The SMTP server settings in Eggplant Functional preferences are user-level preferences only, so when running sendMail from the command line or Eggplant Manager script executions, specifying the SMTP server details, such as smtp_user and smtp_host, with the SendMail command is generally required.

StartMovie Command

Behavior: Starts recording a movie of the SUT. If you do not designate a file path, the movie is saved in the suite’s Results directory. Movies are saved with file extension .mkv. If you do not designate a file path, the movie is saved in the results folder for the script run. If you do not specify a frame rate, movies are recorded at 10 frames per second.

ExtraTime continues the recording for a given number of seconds after the StopMovie command executes. The default is five seconds. ImageHighlighting is a boolean value that determines whether or not search rectangles are highlighted in the movie. This setting overrides the Image Highlighting setting from the Run menu.

Parameters: A file name for the movie you are capturing and optional file path. FramesPerSecond, ExtraTime, and imageHighlighting are also optional.

Syntax:

StartMovie "/Path/to/Movie"

Example:

StartMovie "/User/Documents/BugMovie" //Begins recording a movie with the name BugMovie

Example:

StartMovie "/User/Documents/BugMovie", framesPerSecond:5, extraTime:10, imageHighlighting:Yes

Example:

if runningFromCommandLine() then //Determines whether the script is running via command line only execution

startmovie "BugMovie"

login

stopmovie

end if

Tip: Movies are recorded with no compression using file format .mkv. Be sure to allow ample disk space when you are recording movies.
Tip: We recommend using VLC Player to view movies created by the StartMovie command.

StopMovie Command

Behavior: Ends a movie recording that was started by a script.

Parameters: None.

Syntax:

StopMovie

Example:

StopMovie

TraceScreen On/Off Command

Behavior: Turns TraceScreen mode on and off. When TraceScreen mode is on, a full image of the SUT is captured into the results immediately before the execution of each command that acts on the SUT. These screen captures are available in the Log Area of the Suite Editor’s Results pane, and are stored in the result's folder for the script.

Parameters: On or Off.

Syntax:

TraceScreen On

Example:

TraceScreen On

Run "SearchStore" //Only events that occur in the SearchStore script will have screen tracing

TraceScreen Off

Tip: TraceScreen captures full size screenshots of the SUT, which means it can consume large amounts of disk space. Use TraceScreen carefully.
Note: Because TraceScreen is intended as a diagnostic tool, it creates a final screen capture when you issue the TraceScreen Off command. Even if TraceScreen is already off, issuing the TraceScreen Off command creates a screen capture.

 

This topic was last updated on October 11, 2019, at 11:18:54 AM.

Eggplant icon Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2019 Eggplant