May 17, 2022

Machine Vision Drives Smart Robot Assistant – Metrology and Quality News

Due to the industry standard 4.0, the digitization, automation and networking of systems and plants are becoming prominent topics in production and therefore also in logistics. Industry 4.0 pursues the increasing optimization of processes and workflows in favor of productivity and flexibility and thus saving time and costs. Robotic systems have become the engine of process automation. Thanks to the Internet of Things (IoT), robots are becoming increasingly sensitive, autonomous, mobile and easy to use. Increasingly, they are becoming a daily help in factories and warehouses. Intelligent imaging techniques play an increasingly important role
in this.

To meet the increasing demands for shifting and evolving production environments towards fully automated and intelligently networked production, ONTEC Automation GmbH of Germany has developed a self-driving robotic assistance system. The ‘Smart Robot Assistant’ uses the synergies of mobility and automation: it consists of a powerful and efficient intralogistics platform, a flexible robotic arm and a robust 3D stereo camera system from the Ensenso N series from IDS Imaging Development Systems.

The solution is versatile and supports monotonous and heavy configuration and placement tasks, for example. The self-contained transport system is suitable for lifting Euro-pallets up to container or industrial format as well as mesh pallets of various sizes at ground level with a maximum load of 1,200 kilograms. For a customer in the textile industry, the AGV (Automated Guided Vehicle) is used for the automated loading of bobbin creels. For this purpose, he picks up pallets with spools of yarn, transports them to the designated creel and loads them for further processing. Thanks to a specially developed gripper system, up to 1,000 spools of thread per 8-hour shift are gripped and pushed onto a mandrel of the creel. The sizing scheme and the position of the coils are captured by an Ensenso 3D camera (N45 series) installed on the gripper arm.

For this task, mobile use for 3D acquisition of moving and static objects on the robot arm, the Ensenso 3D camera is suitable thanks to its compact design. The 3D stereo electronics of the Ensenso N 45 are completely decoupled from the housing, allowing the use of a lightweight plastic composite as the housing material. The low weight facilitates the use on robot arms such as the intelligent robotic assistant. The camera can also cope with demanding environmental conditions. “The challenges of this application can be found primarily in the different lighting conditions that are evident in different parts of the room and at different times of the day,” explains Tim Böckel, software developer at ONTEC. Even in difficult lighting conditions, the built-in projector projects a high-contrast texture onto the object to be imaged by means of a pattern mask with a random dot pattern, thereby complementing structures on smooth, featureless surfaces.

The matching process allows the Ensenso 3D camera to recognize a very large number of pixels, including their change of position, by means of the auxiliary structures projected on the surface and to create complete and homogeneous depth information from the scene to from these. This in turn ensures the necessary precision with which the Smart Robot Assistant proceeds. Other criteria for selecting the camera were, among others, the standard Gigabit Ethernet vision interface and the 1.3 MP global shutter sensor. “The camera only takes a single pair of images of the entire palette in favor of faster processing time, but it must provide the coordinates at a relatively large distance with millimeter precision to allow the robot arm to grasp with precision”, explains Matthias Hofmann, IT specialist for application development at ONTEC. “So we need the high resolution of the camera to be able to safely record the edges of the coils with the 3D camera.” The location of the edges is important in order to be able to transmit the position of the center of the roll to the gripper as precisely as possible.

For more information: