Text-Reading Functions

The following information describes the SenseTalk commands and functions you can use in Eggplant Functional that work with text recognition through optical character recognition (OCR) searches or by using the ReadCharacters() Function for character searches.

ReadText() Function

Behavior: Returns the text in a given screen rectangle.

Parameters: A required pair of images, single image, rectangle, or point indicating the region of the screen to read. Using a pair of images usually produces the best results. The ReadText() function uses an optional property list that can improve text search results. SenseTalk includes many options, or properties, that you can use with readText() and other text searches. The following properties are typically the most useful in helping the OCR engine recognize text:

  • Contrast: When Contrast is on, the OCR engine treats the ReadText rectangle as a flat, two-color image. The primary color is taken from the top left pixel of the rectangle, or the ContrastColor property. Pixels that fall within the ContrastTolerance value of that color are considered to have that color. All other pixels are assigned the secondary color. Text can be read in either color. Default value: Off.
  • ContrastColor: When Contrast is on, ContrastColor is the color that is treated as the primary color of the ReadText rectangle. If you do not set the ContrastColor property, contrast color is taken from the top left pixel of the rectangle.
  • ValidCharacters: The string contains the set of characters that should be returned by the ReadText() function. The ValidCharacters property overrides the Language property. This override means that characters that are not part of the ValidCharacters property are not returned by the ReadText function. If the OCR determines that characters are present in the defined area but they do not match characters provided in the ValidCharacters string, it will return "^".
  • IgnoreSpaces: When the IgnoreSpaces property is on during a text search, the search ignores any spaces within the search.
  • IgnoreUnderscores: When the ignoreUnderscores property is on it causes OCR text searches to treat underscores as spaces during searches. For example, the string "My_Computer" would match My_Computer or My Computer. The ignoreUnderscores property is on by default because the OCR engine sometimes fails to recognize underscores.
  • Language: This property determines which character sets are valid for your ReadText() function return value. The Language property can include multiple languages, separated by commas, such as "English, French, German." The default value is English. For a list of all languages supported for use with Eggplant Functional, see OCR Language Support.

See Using OCR Properties in Searches for more information about using these commont text properties.

The following additional properties, listed alphabetically, can also be used with readText(). You might need to experiment with these properties in your specific environment to see which ones provide the best results for your text searches:

  • CaseSensitive: OCR text searches are not case sensitive by default. They should return results whether the letters are capital or lowercase. Use the CaseSensitive:yes parameter to force searches to respect case and return only results that match your text string's capitalization exactly. Use the CaseSensitive:no parameter to ignore case and return all results, ignoring case.
  • ContrastTolerance: When Contrast is on, ContrastTolerance is a measure of how much a pixel can differ from the RGB value of the ContrastColor property and still be considered the primary color. Default value: 45.
  • DPI: Dots per inch (DPI) of the SUT screen. Default value: 72 DPI.
  • EnableAggressiveTextExtraction: Enable this property if you want OCR to extract as much text from the image as possible.
  • EnhanceLocalContrast: Enable this property if you want OCR to automatically increase the local contrast of the image.
  • ExtraWords: Set this property to a list of words to supplement the built-in dictionary for the current language.

  • IgnoreNewLines: Enable this property to have OCR searches ignore new lines during a search while not affecting the text actually read by OCR.
  • MultiLine: This property only applies when reading text near a point, as opposed to reading text within a rectangle. When MultiLine is on, the ReadText() function returns the line of text associated with your point, and any subsequent lines that appear to belong to the same text block. When MultiLine is off, the ReadText() function only returns the line of text associated with your point. Default value: Off.
  • PreferDictionaryWords: Enable this property to tell the OCR engine to read each word as a word defined in its dictionaries, if possible. If no suitable dictionary words are found, the engine returns a nondictionary word, using its best interpretation of each character.

  • PreferredWords: This property is a string or list that determines the words that can be returned by the ReadText() function, but will also return words that do not match those specified.
    Note: The ValidWords property defined below can be used to filter out all words that do not match those specified.
  • ProhibitedWords: Set this property to a list of words to exclude from recognition. If the engine sees something that looks like one of these words, it falls back to a different interpretation of the characters to get a different word instead.

  • SingleColumnMode: When SingleColumnMode is on, the OCR engine presumes that there is only one column of text on the screen. When SingleColumnMode is off, the OCR engine can detect multiple columns and order lines of text accordingly in the return value.
  • TextRotation: One of four values:
    • clockwise. Rotated 90 degrees to the right
    • counter-clockwise. Rotated 90 degrees to the Left
    • upside-down. Rotated 180 degrees
    • none. Not rotated

    When this property is set,the ReadText() function identifies words at the degree of rotation specified.

  • Trim: When Trim is on, the OCR engine trims each edge of the ReadText() function rectangle until a non-background pixel is encountered. The background color is taken from the top left pixel of the rectangle, or the TrimColor property. Default value: Off.
  • TrimBorder: When Trim is on, TrimBorder, an integer, is the pixel-width of background that is not trimmed from the ReadText() function rectangle. The TrimBorder can be set to a negative number, to trim non-background edges from the rectangle. Default value: 0.
  • TrimColor: Color. When Trim is on, TrimColor is the color that is considered the background of the ReadText() function rectangle. If you do not set the TrimColor property, the background color is taken from the top left pixel of the rectangle.
  • TrimTolerance: When Trim is on, TrimTolerance, an integer, is a measure of how much a pixel can differ from the RGB value of the TrimColor and still be considered background. Default value: 0.
  • TrimWhitespace: When TrimWhitespace is on, all whitespace characters are removed from the beginning and end of returned text. When TrimWhitespace is off, the ReadText() function can return text that starts or ends with whitespace characters. Default value: On.
  • ValidCharacters: This parameter limits the characters that the OCR engine recognizes. By default, the engine uses the entire set of characters for the language (or languages) in the current text platform. Sometimes, by limiting the characters, you can force the engine to see your text string. You can use the asterisk as a wildcard so that the OCR engine looks only for the characters in your text string.
  • ValidPattern: This property is a string that takes a regular expression value as outlined in the table below and returns only characters or words that match the pattern specified.

    The following table outlines the regular expression characters available for use with the OCR:

    Item Name Conventional Regular Expression Sign Usage Example/ Explanation
    Any Character . c.t - denotes words such as “cat” and “cot”
    Character from a character range [] [b-d]ell - denotes words such as “bell”, “cell”, “dell”
    [ty]ell - denotes words “tell” and “yell”
    [A-Z] - denotes any uppercase alpha character
    [a-z] - denotes any lowercase alpha character
    [A-Я] - denotes any uppercase Cyrillic character
    [а-я] - denotes any lowercase Cyrillic character
    [0-9] - denotes any numeric character
    [0-9a-zA-Z] - denotes any single character, including alpha and numeric characters
    Character out of a character range [^] [^y]ell - denotes words such as “dell”, “cell”, or “tell”, but not “yell”
    Or | c(a|u)t - denotes words “cat” and “cut”
    0 or more occurrences in a row * 10* - denotes numbers 1, 10, 100, 1000, etc.
    1 or more occurrences in a row + 10+ - allows numbers 10, 100, 1000, etc. but not 1
    [0-9a-zA-Z]+ - allows any word

    Notes:

    • Some characters used in regular expressions are used for system purposes. As seen in the table above, these characters include square brackets, periods, etc.
    • If you wish to enter an auxiliary character as a normal one, put a backslash (\) before it. Example: [t-v]x+ denotes words such as "tx", "txx", "txxx", etc., and "ux", "uxx", etc., but \[t-v\]x+ denotes words such as "[t- v]x", "[t-v]xx", "[t-v]xxx" etc.
    • If you need to group certain regular expression elements, use parentheses. For example, (a|b)+|c denotes "c" and any combinations such as "abbbaaabbb", "ababab", etc. (a word of any non-zero length in which there can be any number of a's and b's in any order), while a|b+|c denotes "a", "c", and "b", "bb", "bbb", etc.
  • ValidWords: This property is a string or list that determines the words that can be returned by the ReadText function. The validWords property overrides the Language property. This override means that words that are not part of the validWords property are not returned by the ReadText() function.

Example:

put the last character of readtext("UpperLeft","LowerRight",contrast:on, contrastColor:(0,0,128),contrastTolerance:25) // Prints only the last character returned by the readText() function

Example:

Log trimAll(ReadText(("UpperLeft","LowerRight"), ValidPattern: "[A-Za-z]+\.py")) // Enforces that readText use a regular expression to read from the screen, and trims all white space, such as tabs and carriage returns, from the output

Example:

//Use code similar to this to read to the left of a label

 

function readResultsNumber ObjectLabel // Declares a custom function named readResultsNumber with parameter ObjectLabel

 

set ResultsRectangle to ImageRectangle(text:ObjectLabel) // Uses OCR to find the label on the screen and then stores the ImageRectangle in a variable

set ResultsNumber to ReadText((0,Top of ResultsRectangle),BottomLeft of ResultsRectangle) // Uses OCR to read to the left of the label, to the edge of the SUT's screen

delete comma from ResultsNumber // Removes all commas from the value stored in ResultsNumber

return ResultsNumber // Returns ResultsNumber so it can be retrieved by the calling handler

 

end readResultsNumber

Practice:

Use the class schedule under the readTable() Function to practice the next three examples.

Example:

log readText(imagelocation("ReadingAssignmentHeader").x, imagelocation (text:"Introduction").y,singlecolumnmode:true) // Reads around a single point based on the x coordinate of the column and y coordinate of the row

Example:

// Use code similar to this to read every row in a particular table column

put ImageLocation("ClassSchedule") into ILC // Stores the location of the hot spot of "ClassSchedule" in a variable. The hot spot is located at the center of the image.

put ((ILC+(20,60)), (imageRectangle("ReadingAssignmentHeader").BottomLeft+(-10,34))) into TableRectangle // Use the locations of ClassSchedule and ReadingAssignmentHead to set establish the location of the first table cell you want to read

repeat 7 times // Indicates how many rows will be read

log ReadText(TableRectangle) // Logs the content of the cell

add ((0,30),(0,30)) to TableRectangle // Shifts the read area 30 pixels down to cover the next cell

end repeat

Example:

//Use code similar to this to read columns for specific rows

set Characters to 0..9 &&& dash &&& "Chapters" // Puts all numbers, -, and the letters in "Chapters" into a variable as a list

put the readTextSettings into RTS // Stores the current readTextSettings in a variable

set the readTextSettings to (validCharacters:Characters) // Sets the readTextSettings based on the list of validCharacters stored in Characters. Using the readTextSettings is appropriate when executing multiple readText() or readTable() functions that require the same set of readTextSettings

put (2,4,7) into Weeks // Stores a list of the row identifiers of interest into a variable

put imageRectangle("WeekHeader") into WeekHeader // Stores the rectangle for the found image "WeekHeader"

put imageRectangle("ReadingAssignmentHeader") into ReadingHeader // Stores the rectangle for the found image "ReadingAssignmentHeader"

repeat with each Week of Weeks // Repeats once for each row identifier in Weeks

put (WeekHeader.BottomLeft, (WeekHeader.Right,remoteScreenSize().y)) into WeekColumn // Creates a rectangle based on location of the Weeks column in the table

put imageRectangle (text:Week,searchRectangle:WeekColumn) into RowNum // Stores the rectangle for the row identifer into a variable

put readText(ReadingHeader.Left,RowNum.Top-4, ReadingHeader.Right, RowNum.Bottom+4) into Reading // Reads a rectangle based on the row position and the column position

If Reading is not empty then // Checks whether the return of readText() is empty

Log Reading

else

Log "No chapter."

end if

end repeat

set the ReadTextSettings to RTS // Sets the ReadTextSettings back to the original property list

ReadTable() Function

Behavior: Returns the text of a table as a list. The returned list contains one sublist per row of values, with each sublist containing one value per cell detected within that row. The sublists for the rows might or might not contain the same number of values.

Parameters: One rectangle in which you want to read text. An optional property list that includes any number of the properties listed under the ReadText() function.

Example:

Log ReadTable("TableUpperLeft","TableLowerRight")

Practice:

Use this class schedule to practice the following example.

Example table image for ReadTable() and ReadText() examples

Example:

put (("Week", "Topic", "Reading Assignment"), (1, "Course Introduction", "Chapter 1")) into ClassAssignments // Creates a list of lists of the expected table contents

put readtable("TableUpperLeft", "TableLowerRight") into ClassAssignmentsTable // Stores the output of readtable into a variable

put the first item of ClassAssignments into AssignmentsHeader // Stores the content in a variable

put the first item of ClassAssignmentsTable into TableHeader // Stores the content in a variable

Assert that AssignmentsHeader=TableHeader // Asserts whether the expected header and the actual table header are the same

repeat for each item 2 to -1 of ClassAssignmentsTable // Iterates from the second to last item. Each item represents a row in the table

if the second item of it is not "Review, final exam" then // Checks the value of the second column of the table row

Log "Assignment" && the repeatIndex && "is" && the third item of it && colon && the second item of it & period // Concatenates strings, the repeatIndex, and values from different row columns, and logs the message

end if

end repeat

Note: Generally, the ReadTable() function performs best when the bounds of the table are included in the rectangle. If no table is found within the given rectangle, an error is thrown.
Tip: If the ReadTable() function doesn't provide the desired results or doesn't detect the table, experiment with different boundaries for the given rectangle around the table.
Tip: The ReadTable() function works best with very regular, clearly defined tables. If the ReadTable() function struggles to read your table as desired, use the ReadText() function to read each row or cell individually.

ReadCharacters() Function

Behavior: Use the ReadCharacters() function with character collections that you capture. Use this function in your scripting to return a character from a character collection (in a given screen rectangle) as a string or multiline string.

Parameters: A required pair of images, single image, rectangle, or point indicating the region of the screen to read. Using a pair of images usually produces the best results. The region parameter is followed by an extra parameter, which is the name of the character collection to use for reading the text on the screen. You can omit the name of the character collection if you set the CurrentCharacterCollection global property.

  • asList: When asList is on (asList:Yes), the ReadCharacters() function returns a list of strings, one for each group of recognized characters.
  • characterPriority: This string contains priority characters, in high-to-low order. The default setting is ";:.". This setting controls which character is recognized if two character images are found in the same location. For example, using characterPriority:"OC" gives the "O" character image a higher priority than the "C"character image. The result is that the ReadCharacters() function uses the letter "O" rather than the letter "C" if both the O and C images are found at the same location on the screen.
  • minimumVerticalOverlap: This value sets the minimum overlap, in pixels, allowed for two characters to be considered to be on the same line. If the two characters overlap more than the minimumVerticalOverlap value, the ReadCharacters() function considers these characters to reside on the same line.
  • maximumHorizontalOverlap: This value sets the maximum overlap, in pixels, allowed for adjacent characters to be considered as occupying the same space. If two adjacent characters overlap more than this setting, the ReadCharacters() function considers these characters to be occupying the same space.
  • maximumAdjacentGap: This value sets the maximum gap, in pixels, allowed between two adjacent characters. This setting assumes no space between characters. If two adjacent characters reside further apart than this setting, the ReadCharacters() function considers these characters to be non-adjacent characters.
  • maximumSpaceGap: This value sets the maximum gap, in pixels, for an implied space to exist between two characters. If two characters reside closer together than this setting, the ReadCharacters() function does not consider these characters to be separated by a space.
  • spaceWidth: This value sets the nominal width, in pixels, of a space character. The ReadCharacters() function uses this setting to determine the number of spaces returned for wide space gaps.

In addition to the properties listed here, you can specify any of the image search properties. These special properties override the corresponding search property for every character image in the collection. For example, specifying "tolerance:60" causes the ReadCharacters() function to use a tolerance value of 60 when searching for every character image rather than the individual tolerances that are set for each image. See Image Property List for more information.

Syntax:

ReadCharacters (<rectangle>, <option1Name>:<option1Value>, <option2Name>:<option2Value>,...<optionNName>:<optionNValue>)

Example:

put ReadCharacters(the RemoteScreenRectangle, CharacterCollection:"Receipt") into myString // Puts the returned value into a string

Example:

set the currentCharacterCollection to "Receipt" // Sets the currentCharacterCollection global property.

put ReadCharacters(the RemoteScreenRectangle) into myString // There is no need to use the characterCollection property as the property is set using the currentCharacterCollection global property in the previous line

Example:

put ReadCharacters(the RemoteScreenRectangle, "Receipt", asList:true) into stringList // Specifies asList:true to have the ReadCharacters() function return a list of strings

Example:

ReadCharacters(the RemoteScreenRectangle , characterPriority:"OC") // Sets the characterPriority property so that if two character images, O and C, are found in the same location the ReadCharacters() function uses the O character

Example:

put ReadCharacters(the RemoteScreenRectangle, minimumVerticalOverlap:6) // Sets the minimumVerticalOverlap property to 6 pixels. If two characters overlap (vertically) by more than 6 pixels, the ReadCharacters() function considers them to reside on the same line.

Example:

ReadCharacters(the RemoteScreenRectangle , maximumHorizontalOverlap:2) // Sets the maximumHorizontalOverlap property to 2 pixels, so that if the horizontal locations of two adjacent characters overlap by more than 2 pixels, the ReadCharacters() function considers these characters to be occupying the same space

Example:

ReadCharacters(the RemoteScreenRectangle , maximumAdjacentGap: 3) // Sets the maximumAdjacentGap property so that if two adjacent characters reside further apart than 3 pixels, the ReadCharacters() function considers them to be non-adjacent characters

Example:

ReadCharacters(the RemoteScreenRectangle , maximumSpaceGap:2) // Sets the maximumSpaceGap property to 2 pixels so that if two characters reside closer together than 2 pixels, the ReadCharacters() function does not consider these characters to be separated by a space

Example:

ReadCharacters(the RemoteScreenRectangle , spaceWidth:2) // Sets the spaceWidth property to 2 pixels. The ReadCharacters function uses this value to determine the number of spaces occupying wide space gaps

Related:

See Working with Character Collections for more information.

 

This topic was last updated on June 21, 2019, at 02:49:49 PM.

Eggplant icon Eggplantsoftware.com | Documentation Home | User Forums | Support | Copyright © 2019 Eggplant