The OCR Update Panel
The OCR Update panel can help you find text matches when your script search is failing. This feature presents you with diagnostics for the search on your system under test (SUT) where the search is failing. Then it lets you change parameters for the Optical Character Recognition (OCR) text search to something that might succeed.
Your choices when using this menu are as follows:
- Throw Exception (Update Off): The OCR Update panel is not used.
- Show Panel (Manual Update): This option opens the OCR Update panel when an OCR search fails during runtime so that you can interactively work with settings to find an appropriate match.
- Auto Update: This setting engages the OCR Update panel internally by trying additional search parameters to create a match and continue the script without user interaction.
The Auto Update option is a good choice for unattended script runs. Frequently, it finds a match and continues the script run when otherwise the search would throw an exception and fail. However, if your parameters are too broad, you could end up matching the wrong text, with unintended results.
The Show Panel (Manual Update) option lets you use the OCR Update panel interactively, which gives the most complete control over text matches. As this method is suited for attended test runs, it will not be right in all cases.
Using the OCR Update Panel
When you are using the manual option, the OCR Update panel opens automatically when a script fails on an OCR text search. The panel consists of several tabs, each described below.
At the bottom of each tab, you'll find the same three buttons:
- Abort: Use this button to abort the script run.
- Proceed: Use this button to cause the script to proceed to its next step. Note that unless the script is written in a way to handle the failed search exception, using this option causes the script to fail.
- Try Again: Use this option to return to the script by retrying the search that caused the exception. If the new search is successful, the script continues to run.
On the bottom left side of each tab you can choose one of the following if you want to change the criteria for when the OCR Update panel appears:
Update Off: The OCR Update panel is not used.
Manual Update: This option opens the OCR Update panel when an OCR search fails during runtime so that you can interactively work with settings to find an appropriate match.
Auto Update: This setting engages the OCR Update panel internally by trying additional search parameters to create a match and continue the script without user interaction.
When you use the Auto Update option, changes made during the script run are applied in memory only. You must apply updates to the image from the Results panel to retain changes.
It is good to track how often your scripts are engaging the Auto Update option. To see how often this is happening, do the following:
- Click the script in question in the Results pane in the Suite window. This action displays a Results pane that shows records of the tests you have run in the current suite.
- Click a row in the top of the Results pane.
- Review the results in the bottom of the Results pane, looking for the AutoUpdate string in the text and message columns to see how many times auto update has been engaged and how long it took to resolve that case. For example, look for the AutoUpdate strings marked by the purple rectangles in the following screenshot:
You can adjust the timer in Preferences by going to Eggplant > Preferences > Run > System. In the Auto-proceed countdown on Image/OCR Update section, input the desired time (in seconds) in the Auto-proceed Delay field. If you use a value of 0, the timer won't show at all and the manual OCR Update panel won't continue automatically back to your script.
The Start Tab
The Start tab is the first part of the panel that shows when an OCR text search fails. This tab provides you with several diagnostic tools to help you adjust your OCR search parameters.
At the top of the window, the text string from the failed search is displayed. Beneath the text string, you will see the list of parameters that were used in the search.
When the OCR Update panel opens, the Start tab might display a countdown timer. If you take no action, the panel closes and the script continues after the countdown runs out, which can be useful for unattended script runs. However, unless your script is prepared to handle the exception of the failed search, it fails after it continues.
Beneath the buttons that access the different tabs, the bottom half of the window presents actions you can take when an OCR text search has failed. You have two primary choices shown in the drop-down menu:
- Use Information: When a diagnostic search finds a potential match, you can use that information to let your script try the search again. Select the diagnostic in the upper part of the panel, then select Use Information from the drop-down list and click Do This. The script resumes running by performing the original search again, temporarily using the indicated parameter. The actual script code is not changed.
- Copy Information: You can also select a successful diagnostic in the upper part of the panel, then select Copy Information from the drop-down list. Taking this action copies the indicated parameter to the clipboard so that you can easily update your script if you decide the parameter will be helpful for future matches.
After choosing which action you want to take for this OCR search, click Do This to test the chosen action.
The Diagnostics tab shows a variety of diagnostic searches to see what parameters can be adjusted in order to find the text string on the screen with an OCR search.
The Status column shows <Not Found> if a match could not be made with the parameter, or it shows information about the match if a possible match was found, including multiple matches. The Info column shows the parameter value necessary to make a match where one is possible.
The diagnostic searches conducted are:
- Standard: Searches for the text again using the original specifications. A result for this search means that the text has appeared since the first failure to find it. Most likely this means that the timing of the script at the step needs to be adjusted to allow more time for the desired text to appear.
- CaseSensitive: OCR text searches are not case sensitive by default. They should return results whether the letters are capital or lowercase. You can use the caseSensitive parameter to force searches to respect case and return only results that match your text string's capitalization exactly. In most cases, the diagnostic shows a successful match only if your script specified caseSensitive: yes, in which case you might be able to make a match if the search isn't case sensitive.
- IgnoreSpaces: This diagnostic looks for matches without respect to blank spaces, either in the text string you're searching for or in potential matches on the SUT screen. For instance, the OCR engine would match either "Egg plant" or "Eggplant" if your search string was "Eggplant." This parameter is on by default for OCR text searches, so the diagnostic generally suggests a possible match only if you've used ignoreSpaces:"No" in your script.
- TextDifference: This diagnostic attempts to find a match by looking for text that can have individual characters that are different from your original string. This parameter can be useful because the OCR engine can sometimes misread text on the screen, mistaking a 0 for an O, for instance. The "text difference" is the number of characters that would need to change between your search string and the text found on screen to make an exact match.
- ValidCharacters: The ValidCharacters parameter limits the characters that the OCR engine will recognize. By default, the engine uses the entire set of characters for the language (or languages) in the current text platform. Sometimes, by limiting the characters, you can force the engine to "see" your text string. You can use the asterisk as a wildcard, validCharacters:"*", so that the OCR engine looks only for the characters in your original text string.
- ValidWords: The ValidWords parameter limits the words that the OCR engine will recognize. By default, the engine uses the entire set of words for the language (or languages) in the current text platform. Sometimes, by limiting the words, you can force the engine to "see" your text string. You can use the asterisk as a wildcard, validWords:"*", so that the OCR engine looks only for the words in your original text string. The validWords parameter overrides the Language property. This override means that words that are not part of the validWords property are not returned by the ReadText() Function.
- DPI: By default OCR searches use 72 DPI for searches. However, if the resolution on the SUT is significantly different, you might be able to find your text by using the DPI parameter to set a different resolution for the search.
- Language: This diagnostic indicates if a match might be made with a language setting other than the default text platform.
When a diagnostic search finds a potential match, you can use that information to let your script try the search again. Select the diagnostic in the list, click Use Info, then click Try Again. The script resumes running by performing the original search again, temporarily using the indicated parameter. The actual script code is not changed.
You can select a successful diagnostic, then click Copy to copy the indicated parameter to the clipboard so that you can easily update your script if you decide the parameter will be helpful for future matches as well.
If you have changed conditions in your SUT, or changed search options on the Properties tab of the OCR Update panel, click the Search button to create a new search and refresh the diagnostics.
The Properties Tab
The Properties tab lets you try different parameters for your OCR search, or combinations of parameters, to see if a match can be made.
When you change a value, the original value from the script appears in red beside the altered field. At the bottom of the fields, in blue text, the result of using the new value is displayed. Note that your changes are cumulative, so if you change multiple properties, each of them is applied to the search to attempt to make a match.
If you want to resume your script with new parameters you've selected, click Use Settings, then click Try Again. The script resumes running by performing the original search again, temporarily using the indicated parameters. The actual script code is not changed.
The Tuner tab lists the parameters you can fine-tune to improve your OCR searches.
Text Read: As you make changes in the Capture Area from the Viewer window, view the text that OCR is reading in the Text Read area of the UI.
OCR Settings: OCR contains many tunable parameters that can help you improve your OCR searches. Spend time understanding the OCR properties so that you can make appropriate adjustments and achieve better search results in your test environment. The information shown in OCR Settings contains choices of predetermined text styles. You can change the individual attributes as needed using the following selections. Use the information shown here to resolve or improve your OCR searches:
DPI: By default, OCR uses 72 DPI screen resolution for searches. If the resolution on your SUT is significantly different, you might need to adjust the DPI for the search.
Valid Words: This parameter limits the words that the OCR engine recognizes. By default, the engine uses the entire set of words for the language (or languages) in the current text platform. You can use the asterisk as a wildcard so that the OCR engine looks only for the words in your original text string.
Valid Characters: This parameter limits the characters that the OCR engine recognizes. By default, the engine uses the entire set of characters available for the language (or languages) in the current text platform. You can use the asterisk as a wildcard so that the OCR engine looks only for the characters in your original text string.
Valid Pattern: This property is a string that takes a regular expression value and returns only characters or words that match the specified pattern.
Contrast: Select this checkbox if you want OCR to treat the ReadText() rectangle as a flat, two-color image. This option can be useful when working with anti-aliased text. It filters the screen of the SUT so that it appears two-toned to the OCR engine. Contrast relies on the following settings:
- Contrast Tolerance: A measure of how much a pixel can differ from the RGB value of the selected color and still be considered the primary color.
- Color: Click the color well and select a new color using the Colors panel. OCR treats this color as the primary color of the ReadText() rectangle. It is generally advised to select the background color for this selection.Tip: Use the color picker in the Colors panel to copy a color from any place in your display, including the Viewer window. Click the magnifying glass in the Colors panel, then click again wherever you see the color you would like to copy.
Enhance Local Contrast: Select this checkbox if you want OCR to automatically increase the local contrast of the image.
Enable Aggressive Text Extraction: Select this checkbox if you want OCR to extract as much text from the image as possible.
Case Sensitive: Select this checkbox to enable case-sensitive searching.
Ignore Spaces: Select this checkbox to have OCR searches ignore spaces between characters during a search.
Ignore Lines: Select this checkbox to have OCR searches ignore new lines during a search.
Language: Use this drop-down menu to add or remove OCR recognition languages. You can select multiple languages for use in OCR searches. To enable additional languages, navigate to Eggplant > Preferences, then click Text and scroll to Enabled Languages.
Language: Use this drop-down menu to add or remove OCR recognition languages. You can select multiple languages for use in OCR searches. To enable additional languages, navigate to eggPlant > Preferences, then click Text and scroll down to Enabled Languages area to make your selections.
Click Copy to save the settings you made into the clipboard area for later use.
Enter a style name, then click Save Style to save your style changes for reuse.