Overview
Overview
You can specify an image as the target UI using the features provided by the ImageRecognition library. If a user captures an image, a library action is performed by recognizing the same UI factor as this image. You can perform actions, such as clicking a specific point in the image or converting the characters inside an image into text. Also, using the Optical Character Recognition (OCR) server, you can convert the text and tables on images into text strings and array data. The Optical Character Recognition features require the OCR server for operation. Therefore, they can be used only when you are logged in to the Designer account.
The features provided by the ImageRecognition library are as follows:
Mouse action features: ImageClick, ImageHover, ImageRightClick, ImageDoubleClick
Image recognition features: IsExist, MatchImages, WaitImageAppear, WaitImageDisappear, GetImageBounds
Image capturing features: CaptureBounds, CaptureScreen, CaptureWindow, CaptureFromFile
Features related to image color: GetImageColor
Image text input features: ImageTextInput
Image using Lable Information: GetImageLabel, GetImageLabelOnFile
Some activity cards support the ability to select a target to use as the final result when there are multiple image-based search results.
Supports multi-image result index navigation : GetImageBounds, ImageClick, ImageRightClick, ImageHover, ImageDoubleClick, IsExist, WaitImageAppear, ImageTextInput, etc.
Use Designer or File Explorer to refer to the samples for the ImageRecognition library.
• On Designer: Menu > Help > Sample > Sample > ImageRecognition
• On File Explorer: C:\Users\user\AppData\Roaming\Brity RPA Designer\samples\ImageRcognition
[ImageRcognition sample project names and related activity cards]
• Capture Screen: CaptureBounds, CaptureFromFile, CaptureScreen, CaptureWindow
• Mouse Action: ImageClick, ImageDoubleClick, ImageHover, ImageRightClick
• Wait Image Target: WaitImageAppear, WaitImageDisappear
• Other Samples: GetImageBounds, GetImageColor, ImageTextInput, IsExist, MatchImages
• ImageLabel: GetImageLabel, GetImageLabelOnFile
Common Properties
Common Properties
IMAGE
Shows the information about the captured image (size and position).
Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|
Image | Image | Y | - | Y | View captured images. If you double-click the image, you can change the image settings using ImageEditor. |
Bounds | Number | N | - | Y | Shows the specified position and size of the target UI object as relative coordinates based on the screen. (Unit: Pixel) X: Horizontal, Y: Vertical, W: Width, H: Height Height |
Image Index Method | Combo Box | N | - | N | Set how to sort the results when there are multiple results of image-based object search. - JustBest : Use only one with the highest comparison score- RowFirst : Sort by row-first order on the screen (Left->Right)- ColumnFirst : Sort by column priority order (top->bottom) - ScoreOrderRow : Sorts in order of highest comparison score , and in case of a tie, in the order of row priority (left->right). - ScoreOrderColumn : Sorts in the order of the highest comparison score, and in the case of a tie, the column priority (top->bottom) order. |
Select Image Index | Number | N | - | N | When there are multiple results of image-based object search, set the index order of the last result to be used in the sorted list. It is an integer value starting from 0. Values outside the result count range are automatically converted and operated according to the result count. |
The Image Index Method property and the Select Image Index property are only available for 8 cards that support multi-image result index navigation. See [Overview]
To set the image shape of the Image property for image-based navigation, double-click the picture part of the Image property to open the image property detail view window.
Image recognition Lib2
In the image detail view window, you can reduce the image comparison area candidate size to be used for actual image comparison. You can also specify the location to click with the mouse. While reducing the image comparison area candidate size, you can change the mouse click position with Apply after confirming the resizing with Crop.
For accurate image-based comparisons, set the image area to be compared as clearly as possible.
MORE OPTIONS
Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|
Timeout(Sec) | Number | Y | sec | Y | Enter the search time for the target object in seconds. If the target object can be searched within the set time, “True” will be returned. If the target object cannot be searched within the set time, “False” will be returned. |
Object Path | Toggle button | Y | - | Y | The Windows path information of the target object is automatically populated. Use this property to search for and identify an object with the path information. For more information about the Windows path information, see Tool > Inspector in the Designer menu. If “@” is entered, it will be regarded as an Xpath syntax. |
SupportedPatterns | - | N | - | Y | The automation pattern available for the target object is automatically entered. |
Focus Before | Toggle button | N | - | Y | You can set the focus on the target object before the activity card is operated. If the focus cannot be set on the target object, the focus is set to the main screen of the target app. If there are multiple target apps running, the app at the front is scanned first. If the target object cannot be found, the app in the back is brought to the front for a rescan. |
Draw Bounds | Toggle button | N | - | Y | Set to display a box (border) to identify the object. |
Raw | Toggle button | N | - | Y | Use a low level keyboard event to determine whether to enter a string. (You can enter only English, Korean, or special characters that can be typed with the keyboard.) |
On Error | Combination box | N | - | N | Select the processing method for occurred errors. - --Ignore--: Ignore the error and proceed. - --Retry--: Try the task one more time. - --GoTo--: Try the scenario for the specified time if the activity fails. - --Event--: Select an event created within the project. |
DESCRIPTION
Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|
DESCRIPTION | Text | N | - | N | Enter the description for the activity card. The description entered in the DESCRIPTION field is used as the description of the activity. A representative value will be displayed in the absence of the DESCRIPTION input. |
ImageClick
ImageClick
Overview
This activity card clicks a specific point on an image.
Application procedures
Double-click No Target on the activity card.
Specify the range of the target image to click.
On the range of the image, click the position for the mouse click.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Click Point | Text | N | - | Y | Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Opens the Amazon website (www.amazon.com) with Chrome, waits for the Amazon logo to be displayed, and verify if the "Returns & Orders” menu image exists. Click the menu if the menu exists. Otherwise, click the “Returns & Orders” menu image. Then, click the Amazon logo and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageClick_Test
Used variables
openBrowser = Variable to specify the launched browser
imgWait = Variable that stores the verification result of whether the Amazon logo and image exist
imgExist = Variable that stores the verification result of whether the "Returns & Orders" menu image exist
Used properties
Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True
Related/Similar activities
ImageHover, ImageRightClick, ImageDoubleClick
ImageTextInput
ImageTextInput
Overview
This activity card identifies a specified image and enters text. (TextInput feature)
Application procedures
Double-click No Target on the activity card.
Specify the range of the target image to click.
On the specified boundary of the image, click the position for the mouse click.
In the Input Text field, enter the text to enter in the specified area.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Input Text | Other | Y | - | N | Enter the string to input within single quotation marks. |
N | Click Point | Text | N | - | Y | Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed. |
N | Clear Before | Toggle button | N | - | Y | Select whether to delete all the text in text field before entering text. (Default: On) |
N | Done(Enter) | Toggle button | N | - | Y | Set whether to press the Enter key after entering the text. (Default: False) |
N | [OPTION] Before Delay(ms) | Text | N | Milliseconds | Y | Specify the delay time before the text input. (Default: 0) |
N | [OPTION] After Delay(ms) | Text | N | Milliseconds | Y | Specify the delay time after the text input. (Default: 0) |
N | [OPTION] Interval(ms) | Text | N | Milliseconds | Y | Set the interval between character inputs. (Default: 0) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome, wait for three seconds until the Google logo is displayed. Then, enter “RPA” at the specified position, wait for three seconds, and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_ImageTextInput_Test
Used variables
Browser = Variable to specify the launched browser
Used properties
Input Text = 'RPA'
Done(Enter) = On
ImageHover
ImageHover
Overview
This activity card hovers the mouse pointer over a specific point on an image.
Application procedures
Double-click No Target on the activity card.
Specify the range of the target object over which to hover the mouse pointer.
On the specified range of the image, click the position to place the mouse pointer.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Click Point | Text | N | - | Y | Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed. |
N | ActionDelay(ms) | Text | N | - | Y | Enter the amount of time to wait after finding the image before performing the actual action. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Amazon website (www.amazon.com) with Chrome and wait for one second. Place the mouse pointer over the three different menu images, wait for one second, then close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageHover_Test
Used variables
browser1 = Variable to specify the launched browser
Used properties
Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True
Related/Similar activities
ImageClick, ImageRightClick, ImageDoubleClick
ImageRightClick
ImageRightClick
Overview
This activity card right-clicks a specific point on the image.
Application procedures
Double-click No Target on the activity card.
Specify the range of the target object to right-click.
On the specified range of the image, click the position to right-click.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Click Point | Text | N | - | Y | Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome and click the Gmail logo. Right-click the Gmail logo image and wait for two seconds. Then, press the “ESC” key to close the context menu and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageRightClick_Test
Used variables
browser1 = Variable to specify the launched browser
Used properties
Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True
Related/Similar activities
ImageClick, ImageHover, ImageDoubleClick
ImageDoubleClick
ImageDoubleClick
Overview
This activity card double-clicks a specific point on an image.
Application procedures
Double-click No Target on the activity card.
Specify the range of the target image to double-click.
On the specified range of the image, click the position for the double-click.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Click Point | Text | N | - | Y | Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Samsung SDS Brity RPA Help website and wait until the menu image is displayed. Then, double-click the menu image three times, wait for two seconds, and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageDoubleClick_Test
Used variables
None
Used properties
Threshold = 98
Timeout(Sec) = 5
Related/Similar activities
ImageClick, ImageRightClick, ImageHover
GetImageBounds
GetImageBounds
Overview
This activity card searches for a specific image on the screen and fetches the coordinates data.
Application procedures
Double-click No Target on the activity card.
On the specified range of the image, click the position to fetch the coordinates data.
Specify the range of the image to fetch the coordinates data.
In the [Output] Output field, enter the variable to store the boundary data of the searched image.
In the Properties window, specify other required properties.
Card output properties
Property | Type | Additional comments | Description | Example |
|---|---|---|---|---|
Output | Rect | Variable type in class X : double Y : double Width : doubleHeight : double | Returns the area information of the image searched for. | X: 774 Y: 575 Width: 52 Height: 19 |
Card input properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Search Bounds | Text | Y | - | Y | Enter the boundary data (x, y, width, height) on the screen to search for the image. The boundary data will be automatically filled in when you specify the boundary by clicking No Target. |
N | Threshold | Number | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome and search for the specified image and fetches the coordinates data. Then, output the coordinate data in a message box and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_GetImageBounds_Test
Used variables
point = Variable to store the boundary data of the searched image
Used properties
[Output] Output =
this.pointThreshold = 96
Remarks
By specifying a position while pressing down the "Alt" key and specifying the image boundary, you can specify the relative coordinates of the image from a specific position (offset feature).
Related/Similar activities
GetImageColor
GetImageColor
Overview
This activity card fetches the color data of the specified coordinates of an image.
Application procedures
Double-click No Target on the activity card.
Specify the range of the image to fetch the color.
On the specified range of the image, click the position to fetch the color.
In the [Output] Output field, enter the variable to store the fetched color.
In the Properties window, specify other required properties.
Card output properties
Property | Type | Additional comments | Description | Example |
|---|---|---|---|---|
Output | String | - | Returns the color value of the specified coordinate in the searched image in RGB format. | '232,232,233' |
Card input properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Search Bounds | Text | Y | - | Y | Enter the boundary data (x, y, width, height) on the screen to search for the image. The boundary data will be automatically filled in when you specify the boundary by clicking “No Target.” |
N | Offset | Number | N | - | Y | Enter the relative coordinates to retrieve the color from. The coordinates will be automatically populated when you select the image. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome, search for the specified image, and fetch the color at the specified coordinates. Then, output the color in a message box and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_GetImageColor_Test
Used variables
openBrowsser = Variable to specify the launched browser
getImageColor = Variable to store the color of the specified coordinates of the image
waitImage = Variable to store the verification result of whether the image exists
Used properties
[Output] Output =
this.getImageColorSearch Bounds = '1606,155,52,60'
Related/Similar activities
IsExist
IsExist
Overview
This activity card verifies whether a specific image exists on the screen.
Application procedures
Double-click No Target on the activity card.
Specify the range of the target image to double-click.
In the [Output] Output field, enter the variable to store the execution result.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | [Output] Output | Variable | Y | - | Y | Specify the variable to save the OCR operation result. (Default: RESULT) “True” is returned if the image is found on the screen within the time entered in the “Timeout” field. Otherwise, “False” is returned. |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (www.google.com) with Chrome. When the search menu icon (magnifier) is displayed, hover the mouse pointer over the Google logo. Wait for two seconds and close the Chrome browser if the Google image is displayed. Otherwise, open the Google website again with Chrome.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_IsExist_Image_Test
Used variables
isExist = Variable that stores the verification result of whether the search menu icon (magnifier) exists
Used properties
[Output] Output =
this.isExitThreshold = 98
Draw Bounds = True
On Error = Ignore
Related/Similar activities
WaitImageAppear, WaitImageDisappear
MatchImages
MatchImages
Overview
This activity card returns a filename by finding an image file that matches the image inside a specific folder within a specified range.
The size of the search area to find a matching target must be larger than the maximum size (width. height) of each image file in the folder.
Application procedures
Double-click No Target on the activity card.
Specify the range of the image to compare with the image in the specified folder.
In the [Output] Output field, enter the variable to store the searched file name.
In the SourceImagePath field, enter the path and name of the image file to compare.
In the Properties window, specify other required properties.
Card output properties
Property | Type | Additional comments | Description | Example |
|---|---|---|---|---|
Output | String | - | Returns the path to an image file that matches an image within a specific folder.※ If there is no matching image, an empty string is returned. | 'C:/IPA_Temp_Imgage/image_sample.png' |
Card input properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
N | Search Bounds | Text | N | - | N | Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen. |
N | SourceImagePath | Text | Y | - | N | Enter the complete path and file name of the image file to compare. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open one image file with the Chrome browser and compare it with the specified image. Output the comparison result, “Matching file: compared_file_name,” in a message box and close the Chrome browser. Repeat the procedure to compare another image with the specified image. Then, output the “I found a similar file in the given folder (C: \\ IPA_Temp_Image). Press OK, and the C: \\ IPA_Temp_Image folder will be deleted. ” message in a message box and delete the copied files and directories.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_MatchImage_Test
Used variables
matchImages = The name of the searched file through the comparison
bid = Variable to specify the launched browser
Used properties
[Output] Output =
this.matchImagesThreshold = 96
SourceImagePath = 'C:\\IPA_Temp_Image'
Draw Bounds = True
On Error = Ignore
WaitImageAppear
WaitImageAppear
Overview
This activity card waits until a specific image appears on the screen.
Application procedures
Double-click No Target on the activity card.
Specify the boundary for the image to search for.
In the [Output] Output field, enter the variable to store the search result of the image.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | [Output] Output | Variable | Y | - | Y | Specify the variable to store the search result of the image. (Default: RESULT) “True” is returned if the image is found on the screen within the time entered in the “Timeout” field. Otherwise, “False” is returned. |
N | Search Bounds | Text | N | - | N | Specify the range of the target screen to search for the image. Without additional input, the search will be performed for the entire screen. |
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome and wait until the Google logo appears. Then, output the result of the image appears in a message box and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Wait Image Target > P_WaitImageAppear_Test
Used variables
appear = Variable to store the result of the image appearance
Used properties
[Output] Output =
this.appearThreshold = 96
Related/Similar activities
WaitImageDisappear
WaitImageDisappear
Overview
This activity card waits until a specific image disappears from the screen.
Application procedures
Double-click No Target on the activity card.
Specify the boundary for the image to search for.
In the [Output] Output field, enter the variable to save the result of disappearing images.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | [Output] Output | Variable | Y | - | Y | Specifies a variable to store the result of the image disappearing (default: RESULT). ※ Returns True if the image disappears from the screen within the time entered in the Timeout field, and False if it does not disappear. |
N | Search Bounds | Text | N | - | N | Specify the range of the target screen to search for the image. Without additional input, the search will be performed for the entire screen. |
N | Threshold | Variable | Y | - | Y | Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome and wait until the Google logo appears. Open the Amazon website (http://www.amazon.com) with a new Chrome browser and wait until the Amazon logo appears. Then, wait until the Google logo disappears, output the result in a message box, and close all Chrome browsers.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Wait Image Target > P_WaitImageDisappear_Test
Used variables
browser1, browser2 = Variables that specify the launched browsers
appear = Variable that stores the result of the image appearance
disAppear = Variable that stores the result of the image disappearance
Used properties
Threshold = 96
Related/Similar activities
CaptureBounds
CaptureBounds
Overview
This activity card captures a specified area on a PC screen.
Application procedures
Ensure that you know the coordinates and size of the captured area, which can be acquired with the GetImageBounds activity card.
In the CaptureBounds field, enter the boundary data of the captured area in the form of “x, y, width, and height.”
Set the SaveClipboard property to select whether to store the captured area on the clipboard.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | CaptureBounds | Text | Y | - | Y | Enter the boundary to capture in the order of Left, Top, Width, and Height. You can also specify the boundary fetched by the GetBounds or GetImageBounds activity cards as the variable. Without additional input, the “RESULT” variable is selected by default, under the assumption that the information acquired from the previous process will be used. |
N | Capture Path | Text | Conditional | - | N | Enter the download path and file name of the captured image. This property becomes available when you disable the SaveClipboard property. |
N | SaveClipboard | Toggle button | Y | - | Y | Set whether you want to save the captured area on the clipboard or as a file. (Default: On) - On: Save the captured area on the clipboard. - Off: Save the captured area as a file. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome. Capture the area with coordinates “500, 200, 600, 200” (x, y, width, height), store it on the clipboard, and then close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureBounds_Test
Used variables
openBrowser = Variable to specify the launched browser
waitImage = Variable that stores the result of image appearance
Used properties
CaptureBounds = '500,200,600,200'
SaveClipboard = true
Remarks
The GetBounds and GetImageBounds activity cards are often used together to verify the coordinates and size of the captured area.
The captured image will be saved on the clipboard when the SaveClipboard property is turned on and saved as a file when it is turned off.
Related/Similar activities
CaptureScreen, CaptureWindow, CaptureFromFile
CaptureScreen
CaptureScreen
Overview
This activity card captures the entire screen of a local PC.
Application procedures
In the Capture Path field, specify the path and name for the captured image file of the local PC screen.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Capture Path | Text | Conditional | - | N | Enter the download path and file name of the captured image. (File format: png) |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Open the Google website (http://www.google.com) with Chrome. Then, capture the entire PC screen, save it as “C:\\CaptureScreenTest.png,” and close the Chrome browser.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureScreen_Test
Used variables
openBrowser = Variable to specify the launched browser
waitImage = Variable that stores the result of image appearance
Used properties
Capture Path = 'C:\\CaptureScreenTest.png'
Related/Similar activities
CaptureBounds, CaptureWindow, CaptureFromFile
CaptureWindow
CaptureWindow
Overview
This activity card captures a specified object or window on a PC screen.
Application procedures
Double-click No Target on the activity card.
Select a target object or a window to capture.
In the Path field, enter the path and file name to save the captured image.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Capture Path | Text | N | - | N | Enter download path and file name of the captured image. (File format: png) |
N | IsOnlyTarget | Toggle button | Y | - | Y | Specify whether to capture the selected window only or to capture the top-most application of the object. (Default: On) - On: Save the specified object as an image file. - Off: Save the window that includes the specified object as an image file. |
Y | - | - | - | - | Set properties related to the target (UI object). | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Run Notepad, capture the object for the “File” menu, save it as “C:\\CaptureWindowTest.png,” and then close Notepad.
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureWindow_Test
Used variables
None
Used properties
Capture Path = 'C:\\CaptureWindowTest.png'
IsOnlyTarget = On
Related/Similar activities
CaptureBounds, CaptureScreen, CaptureFromFile
CaptureFromFile
CaptureFromFile
Overview
This activity card captures a specific part of a specified file and saves the captured image on the clipboard or image file.
Application procedures
In the CaptureBounds field, enter the boundary data of the captured area in the form of “x, y, width, and height.”
In the SourceImagePath field, enter the path and name of the image file to capture.
Set the SaveClipboard property to select whether to store the captured area on the clipboard.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | CaptureBounds | Text | Y | - | N | Enter the boundary to capture in the order of Left, Top, Width, and Height. |
N | SourceImagePath | Text | Y | - | N | Enter the path and name for the image file to capture. |
N | SaveClipboard | Toggle button | Y | - | Y | Set whether you want to save the captured area on the clipboard or as a file. (Default: On) - On: Save the captured area on the clipboard. - Off: Save the captured area as a file. |
N | Capture Path | Text | Conditional | - | N | Enter download path and file name of the captured image. This property becomes available when you disable the SaveClipboard property. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Capture the specified area of the image file, save it on the clipboard, and save the image on the clipboard as a separate file. Next, create a new Excel file, open it, and paste the image on the clipboard into cell “A1.”
Excel_NewExcel_1K
Sample file
Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureFromFile_Test
Used variables
browser = Variable to assign the opened Excel document
Used properties
CaptureBounds = '0,100,200,200'
SourceImagePath =
Global.ResourcesDir+ 'IU.png'SaveClipboard = On
Remarks
To save the captured image in the clipboard and as a file at the same time, you can enable the SaveClipboard property of the CaptureFromFile activity card and use the ClipboardToFile activity card to save the clipboard image.
Related/Similar activities
CaptureBounds, CaptureScreen, CaptureWindow
GetImageLabel
GetImageLabel
Overview
This activity card recognizes labels in specified images based on pre-made image files and labeling information.
Application procedures
In the [Output] Output field, enter a variable to store the execution result.
In the LabelNum field, enter the number of labels to read from the image.
Specify an XML file containing labeling information in ImageLabelPath.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Output | Variable | Y | - | Y | Specify a variable to store the execution result. |
N | LabelNum | Number | Y | - | N | Enter the number of labels to read from the image. |
N | ImageLabelPath | Text | Y | - | N | Specifies an XML file containing labeling information. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
Executes the image file to be recognized, captures a specific area, identifies the image's labels in the area from the labeling information, and outputs it as a message.
Sample file
Menu > Help > Samples > Sample > ImageRecognition > ImageLabel_Sample > P_GetImageLabel_Test
Used variables
readBound = area from which to read the label
Used properties
LabelNum = 3
ImageLabelPath = Global.ResourcesDir + 'Training\\ImageLabel.xml'
Related/similar activities
How to create label information
Create a folder for each image to be recognized as a label, and write the XML format as shown in the figure below. By specifying the label values recognized in each folder, the final label information file is created and saved in the same location as the folder.
GetImageLabelOnFile
GetImageLabelOnFile
Overview
This activity card recognizes labels from pre-made and specified image files based on the labeling information.
Application procedure
In the [Output] Output field, enter a variable to store the execution result.
In the LabelNum field, enter the number of labels to read from the image.
Specify an XML file containing labeling information in ImageLabelPath.
Inthe SourceImagePath field, specify the image file whose label is to be recognized.
In the Properties window, specify other required properties.
Card properties
Common | Properties | Type | Required | Unit | Auto-setting | Description |
|---|---|---|---|---|---|---|
N | Output | Variable | Y | - | Y | Specify a variable to store the execution result. |
N | LabelNum | Number | Y | - | N | Enter the number of labels to read from the image |
N | ImageLabelPath | Text | Y | - | N | Specifies an XML file containing labeling information. |
N | SourceImagePath | Text | Y | - | N | Specifies the image file whose label is to be recognized. |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. | |
Y | - | - | - | - | Common property of the ImageRecognition library. |
Example of utilization
It reads the image file to be recognized, identifies the image's labels in the labeling information, and outputs it as a message.
Sample file
Menu > Help > Samples > Sample > ImageRecognition > ImageLabel_Sample > P_GetImageLabelOnFile_Test
Variable used
None
Used properties
LabelNum = 3
ImageLabelPath = Global.ResourcesDir + 'Training\\ImageLabel.xml'
SourceImagePath = Global.ResourcesDir + 'Label_Test.png'
Related/similar activities