Customized & Scalable Vision Guidance Product
ABSTRACT
This paper illustrates one repeatable, scalable, affordable, and easy to use vision guidance product for users of all levels. Vision Guidance, which means to guide the executable unit to the desired position to perform the required action, is widely used in the industrial automation manufacturing application. Our repeatable product is one standard product, we developed once, and there is no need for the customer to develop it. The critical component for vision guidance product is the machine vision algorithm. Our affordable product provides the customized and fully self-innovated machine vision algorithm without purchasing any commercial product in the market, which greatly reduce the cost of the machine vision product. Meanwhile the performance of our machine vision platform is competitive with the commercial product in the market. Our scalable product is able to satisfy different vision guidance applications according to the customers requirements. Our easy-to-use product works like the smart camera with good usability, the users can deploy the vision guidance application quickly by just dragging and clicking the mouse. We validated our product by performing vision guidance with several kinds of industrial robots such as UR robot and EPSON robot, the test result is good.
PROBLEM STATEMENT
Vision guidance is one of the most popular applications for machine vision in industrial automation manufacturing. Theoretically, vision guidance means to guide the executable unit to the desired position to perform the required actions. The executable units include robot, cylinder, motor, etc. The required actions include the pick & place, the soldering, the assembly and so on. As shown in Fig. 1, vision-guided robots is very popular in automation. Like mounting the eye to the robot, vision guidance greatly increases the accuracy of robot movement and enlarges application fields of robot.
To realize the vision guidance application, there are several needed machine vision technologies: a) object location, b) calibration, c) communication. These technologies of vision guidance are also widely used in the industrial fields. As shown in Fig. 2, we will use object location technology in inspection application because before performing inspection, we normally need to locate the position of object though object location algorithm. We also use the object location in dimensional measurement application for the same reason. So vision guidance is the most widely used industrial machine vision technology.
There are several commercial available products in the market to perform vision guidance, which can be catogrized into two groups, product based on the smart camera and product based on the traditional camera. The representatives of smart camera are Cognex and Halcon. The representative of traditional camera is the Halcon. In table 1, we list cost to use such kind of machine vision product. The smart camera has the good usability, high product cost, low development cost. The traditional camera has bad usability, low product cost and high development cost. In Fig. 3, we can see the using of smart camera is mainly by the mouse drag and click. For traditional camera we need the professional engineer to perform the advanced programming. So the total cost for both kinds of products are high. Also for both smart camera and traditional camera, if we want to use the vision guidance function, we need to buy the whole algorithm packages which is also the reason for high cost.
One customized vision guidance product is necessary to fill the gap between customer’s need and the commercial products in the market. Several challenges are confronted for the development:
- Self-innovated machine vision algorithm: the reason for high cost of commercial vision guidance product is the machine vision algorithm. To reduce the cost of the system, we can’t buy any commercial machine vision library and must develop the algorithm by ourselves, innovations generate the core competence for our product
- Low cost with competitive performance: the customer will not accept one product if the performance is bad even if with the low cost, so our self-innovated machine vision algorithm should have competitive performance compared with commercial product
- Configurable for different kinds of guidance application: we need to guide different kinds of executable units, such as scara robot, 6-axia robot, cylinder, motor and so on, so we need one configurable system which means the user can config the product according to the requirement for different applications
- Friendly use for the operators: the product should has the smiliar usability like the smart camera
- Industrial-level quality to run in the production line: the low cost should not degrade the quality of the product
- Have the potential to support digital factory: digital factory is important for TE, and many customers are interested in it, so we should consider the potential to connect to network and exchange information
We innovated the configurable and customized vision guidance solution. We developed all the machine vision algorithm for vision guidance by ourselves, and don’t purchase any commercial available product in the market, which greatly reduce the cost of the system. The performance of our system is comparable with commercial product. We provide the configurable product which can be configed for different applications. Our solution has the similar operation method like smart camera, the user can config the software just by mouse drag and click to realize one vision guidance application. We validated our product by perfroming vision guidance with several kinds of induatiral robots such as UR robot and EPSON robot and the test result is good. Comparing the cost of commercial product, the cost saving per one set is about 5, 000 USD dolloars, and considering the potential deployment opportunity is 200+, the cost saving for this technology is more than 1M USD dolloars.
METHODS AND RESULTS
AMT Vision Guidance Product
Our product provides the whole solution for vision guidance applications. As shown in Fig. 4, our product package includes the software, the miniature controller & the 2-mega-pixel industrial camera. The camera is used to capture the image, the software can process the image and feedback the processd result, the miniature controller is able to run the software as the host.
The software is the key for the whole system, which is not only integrated with the customized machine vision algorithms, but also provides the configurable operation for the users. To provide the complete solution for the customer, we also provide the camera and controller. The customer doesn’t need to spent time on camera selection and host selection.
The Structure and Functions of Software
To develop one configurable software is one complex project, so we firstly construct the conceptual structure of the software as shown in Fig 5. It has three layers, each layer only serves the layer above it and is served by the layer below it. The function modules are characterized in each layer. To improve the robustness of each layer, the function module in each layer is also responsible for handling errors related to the function module.
The bottom is the Input/Output (I/O) layer which is responsible for the communication between software and the environment. It has four function modules. The camera interface module is used to trigger camera, capture image and read/set the parameters of the camera. The TCP/UDP module implements the TCP and UDP protocals which enables communication with the device which supports TCP/UDP protocal. Right now, almost all the industrial robots support TCP/UDP protocal. The digital I/O module enables the communication through the traditional digital I/O. The Archieve Module is used to archive the information, such as images, system log, the programm recipe, the inspection result.
The middle is the Algorithm & Control layer which receives image from I/O layer, perform the required processing and feedback to the I/O layer and GUI layer. Also this layer is responsible for the logical control of the whole software to schedule the complex tasks between each layer.
The top layer is the GUI layer, which is used for image and result display, also user can adjust the setting of the software by interactive operations.
The Structure and Functions of Machine Vision Library
In the Algorithm & Control Layer, there is one self-developped machine vision algorithm library inside. The structure and function of the library is shown in Fig. 6. Two main functions of the machine vision library is how to represent the image data and how to process the image.
The image representation of library influences the efficiency to read/write image data in computer’s phsicial memory, so it greatly influences the efficiency of machine vision algorithm. For example, for one 2-mege-pixel image, when the image want to read the image data, the algorithm will perform the pixel reading operation 2 million times. Normally, each algorithm will read the image data several times, if the image repsentation is bad, the efficiency of algorithm will be greatly reduced. When tested our image representation, the speed of good representation is at least 3 times faster than that of bad representation.
The image data from the I/O layer doesn’t have the image representation we defined. Normally, there are three kinds of data formats from the I/O layers, they are raw image data from the camera, the QT format – Qimage and the Visual Studio C++ format – Cimage. So we specially provide the format conversion algorithm to convert the imcoming image data to the specificed image representation.
About the image processing algorithm, the image processing library provides three main function modules: 1) image processing, 2) Mathematic Processing and 3) License
1. Image processing: which has the customized machine vision algorithm, and within this module, the algorithms on vision guidance is grouped in several subgroups. Each subgroup is responsible for each type of image processing algorithms. For example, the object location subgroup provides the algorithms on template matching, Calibrate subgroup provides the algorithms on calibration function, etc.
2. Mathematic Processing: mathematic operation is the base for image processing algorithm, which is frequently used in the library. So we realized our high-efficent machine vision library inside the machine vision library. This module has four subgroups, such as Trigonometry subgroup provides the algorithm to calculate the trigonometric values. The Search subgourp provides several kinds of look-up algorithm
3. License: the self-innovated algorithm is the property of TE, so we especially provide the encrypt and decrypt function to control the authorization of the product
Discussion and Key Points
This paper illustrates one repeatable, scalable, affordable, and easy to use vision guidance product for users of all levels. We validated our product by perfroming vision guidance with several kinds of induatiral robots such as UR robot and EPSON robot and the test result is good.
ACKNOWLEDGEMENTS
We are sincerely thankful to Mr. Josef Sinder who shared lots of information and technology on embedded machine vision system.
Customized & Scalable Vision Guidance Product
ABSTRACT
This paper illustrates one repeatable, scalable, affordable, and easy to use vision guidance product for users of all levels. Vision Guidance, which means to guide the executable unit to the desired position to perform the required action, is widely used in the industrial automation manufacturing application. Our repeatable product is one standard product, we developed once, and there is no need for the customer to develop it. The critical component for vision guidance product is the machine vision algorithm. Our affordable product provides the customized and fully self-innovated machine vision algorithm without purchasing any commercial product in the market, which greatly reduce the cost of the machine vision product. Meanwhile the performance of our machine vision platform is competitive with the commercial product in the market. Our scalable product is able to satisfy different vision guidance applications according to the customers requirements. Our easy-to-use product works like the smart camera with good usability, the users can deploy the vision guidance application quickly by just dragging and clicking the mouse. We validated our product by performing vision guidance with several kinds of industrial robots such as UR robot and EPSON robot, the test result is good.
PROBLEM STATEMENT
Vision guidance is one of the most popular applications for machine vision in industrial automation manufacturing. Theoretically, vision guidance means to guide the executable unit to the desired position to perform the required actions. The executable units include robot, cylinder, motor, etc. The required actions include the pick & place, the soldering, the assembly and so on. As shown in Fig. 1, vision-guided robots is very popular in automation. Like mounting the eye to the robot, vision guidance greatly increases the accuracy of robot movement and enlarges application fields of robot.
To realize the vision guidance application, there are several needed machine vision technologies: a) object location, b) calibration, c) communication. These technologies of vision guidance are also widely used in the industrial fields. As shown in Fig. 2, we will use object location technology in inspection application because before performing inspection, we normally need to locate the position of object though object location algorithm. We also use the object location in dimensional measurement application for the same reason. So vision guidance is the most widely used industrial machine vision technology.
There are several commercial available products in the market to perform vision guidance, which can be catogrized into two groups, product based on the smart camera and product based on the traditional camera. The representatives of smart camera are Cognex and Halcon. The representative of traditional camera is the Halcon. In table 1, we list cost to use such kind of machine vision product. The smart camera has the good usability, high product cost, low development cost. The traditional camera has bad usability, low product cost and high development cost. In Fig. 3, we can see the using of smart camera is mainly by the mouse drag and click. For traditional camera we need the professional engineer to perform the advanced programming. So the total cost for both kinds of products are high. Also for both smart camera and traditional camera, if we want to use the vision guidance function, we need to buy the whole algorithm packages which is also the reason for high cost.
One customized vision guidance product is necessary to fill the gap between customer’s need and the commercial products in the market. Several challenges are confronted for the development:
- Self-innovated machine vision algorithm: the reason for high cost of commercial vision guidance product is the machine vision algorithm. To reduce the cost of the system, we can’t buy any commercial machine vision library and must develop the algorithm by ourselves, innovations generate the core competence for our product
- Low cost with competitive performance: the customer will not accept one product if the performance is bad even if with the low cost, so our self-innovated machine vision algorithm should have competitive performance compared with commercial product
- Configurable for different kinds of guidance application: we need to guide different kinds of executable units, such as scara robot, 6-axia robot, cylinder, motor and so on, so we need one configurable system which means the user can config the product according to the requirement for different applications
- Friendly use for the operators: the product should has the smiliar usability like the smart camera
- Industrial-level quality to run in the production line: the low cost should not degrade the quality of the product
- Have the potential to support digital factory: digital factory is important for TE, and many customers are interested in it, so we should consider the potential to connect to network and exchange information
We innovated the configurable and customized vision guidance solution. We developed all the machine vision algorithm for vision guidance by ourselves, and don’t purchase any commercial available product in the market, which greatly reduce the cost of the system. The performance of our system is comparable with commercial product. We provide the configurable product which can be configed for different applications. Our solution has the similar operation method like smart camera, the user can config the software just by mouse drag and click to realize one vision guidance application. We validated our product by perfroming vision guidance with several kinds of induatiral robots such as UR robot and EPSON robot and the test result is good. Comparing the cost of commercial product, the cost saving per one set is about 5, 000 USD dolloars, and considering the potential deployment opportunity is 200+, the cost saving for this technology is more than 1M USD dolloars.
METHODS AND RESULTS
AMT Vision Guidance Product
Our product provides the whole solution for vision guidance applications. As shown in Fig. 4, our product package includes the software, the miniature controller & the 2-mega-pixel industrial camera. The camera is used to capture the image, the software can process the image and feedback the processd result, the miniature controller is able to run the software as the host.
The software is the key for the whole system, which is not only integrated with the customized machine vision algorithms, but also provides the configurable operation for the users. To provide the complete solution for the customer, we also provide the camera and controller. The customer doesn’t need to spent time on camera selection and host selection.
The Structure and Functions of Software
To develop one configurable software is one complex project, so we firstly construct the conceptual structure of the software as shown in Fig 5. It has three layers, each layer only serves the layer above it and is served by the layer below it. The function modules are characterized in each layer. To improve the robustness of each layer, the function module in each layer is also responsible for handling errors related to the function module.
The bottom is the Input/Output (I/O) layer which is responsible for the communication between software and the environment. It has four function modules. The camera interface module is used to trigger camera, capture image and read/set the parameters of the camera. The TCP/UDP module implements the TCP and UDP protocals which enables communication with the device which supports TCP/UDP protocal. Right now, almost all the industrial robots support TCP/UDP protocal. The digital I/O module enables the communication through the traditional digital I/O. The Archieve Module is used to archive the information, such as images, system log, the programm recipe, the inspection result.
The middle is the Algorithm & Control layer which receives image from I/O layer, perform the required processing and feedback to the I/O layer and GUI layer. Also this layer is responsible for the logical control of the whole software to schedule the complex tasks between each layer.
The top layer is the GUI layer, which is used for image and result display, also user can adjust the setting of the software by interactive operations.
The Structure and Functions of Machine Vision Library
In the Algorithm & Control Layer, there is one self-developped machine vision algorithm library inside. The structure and function of the library is shown in Fig. 6. Two main functions of the machine vision library is how to represent the image data and how to process the image.
The image representation of library influences the efficiency to read/write image data in computer’s phsicial memory, so it greatly influences the efficiency of machine vision algorithm. For example, for one 2-mege-pixel image, when the image want to read the image data, the algorithm will perform the pixel reading operation 2 million times. Normally, each algorithm will read the image data several times, if the image repsentation is bad, the efficiency of algorithm will be greatly reduced. When tested our image representation, the speed of good representation is at least 3 times faster than that of bad representation.
The image data from the I/O layer doesn’t have the image representation we defined. Normally, there are three kinds of data formats from the I/O layers, they are raw image data from the camera, the QT format – Qimage and the Visual Studio C++ format – Cimage. So we specially provide the format conversion algorithm to convert the imcoming image data to the specificed image representation.
About the image processing algorithm, the image processing library provides three main function modules: 1) image processing, 2) Mathematic Processing and 3) License
1. Image processing: which has the customized machine vision algorithm, and within this module, the algorithms on vision guidance is grouped in several subgroups. Each subgroup is responsible for each type of image processing algorithms. For example, the object location subgroup provides the algorithms on template matching, Calibrate subgroup provides the algorithms on calibration function, etc.
2. Mathematic Processing: mathematic operation is the base for image processing algorithm, which is frequently used in the library. So we realized our high-efficent machine vision library inside the machine vision library. This module has four subgroups, such as Trigonometry subgroup provides the algorithm to calculate the trigonometric values. The Search subgourp provides several kinds of look-up algorithm
3. License: the self-innovated algorithm is the property of TE, so we especially provide the encrypt and decrypt function to control the authorization of the product
Discussion and Key Points
This paper illustrates one repeatable, scalable, affordable, and easy to use vision guidance product for users of all levels. We validated our product by perfroming vision guidance with several kinds of induatiral robots such as UR robot and EPSON robot and the test result is good.
ACKNOWLEDGEMENTS
We are sincerely thankful to Mr. Josef Sinder who shared lots of information and technology on embedded machine vision system.
