Industrial systems have seen large shifts in their wired and wireless infrastructure this past decade with the shift from traditional fieldbus technologies to industrial Ethernet, with protocols such as Ethernet/IP, ProfitNET, and EtherCAT overtaking legacy networks. Now, the introduction of Industry 4.0 can be readily seen with industrial wireless sensor networks (IWSNs) implemented across the industry verticals of oil and gas, pharmaceuticals, process monitoring/control, fleet management, inventory management, and industrial automation. Beyond simple connectivity of assets on the plant floor, more detailed analysis and predictive maintenance can be performed to optimize operation of machine equipment. This article discusses the implementation and benefits of leveraging artificial intelligence (AI) and machine learning (ML) in relation to complex robotics used in industrial automation.
Understanding robotics used in industrial automation applications
Industrial robot manipulators have been utilized in a wide array of industries to perform repetitive, strenuous, and precise tasks. Robotics have also eliminated the need for human intervention in tasks that could be potentially hazardous to humans; for instance, six-axis industrial robotics can perform car painting efficiently and effectively. Other robotics-based applications include automated packaging, machine tending, circuit-board testing, pick-and-place circuit assembly, metalworking, and shield welding. The high precision of these machines has allowed them to be highly successful at their tasks with relatively low failure rates, especially when compared with assembly lines leveraging human labor. The internet of robotic things (IoRT) brings IIoT to its cutting edge with the need for a combination of relatively high-throughput, low-latency, and high-reliability communications.
The typical hierarchy of control
This concept developed during the ’90s with “networked robotics,” wherein a collection of robotics could be remotely controlled via a wired or wireless communication network. This could vary between the control of a singular robot for several functions to a collection of robots performing various functions. The IoRT involves five layers within its architecture, including the:
- Hardware layer
- Network layer
- Service and application layer
Figure 1 illustrates the hardware and software revolving around these various aspects of IoRT. The hardware layer involves various sensors, actuators, and robotics that can be remotely managed to monitor and control equipment (e.g., robots, vehicles, electronic warfare equipment, home appliances, industrial sensor nodes, etc.). The network layer includes routers, controllers, cloud data storage, and various forms of wireless protocols that can be used to connect to other nodes within the network where gateways and/or base stations connect to the cloud for more complex data processing. These protocols can vary between cellular communications (e.g., 2G/3G, 4G LTE, LTE-A, or 5G) to short-range connections with Wi-Fi, BLE, Zigbee, Z-wave, 6LoWPAN, and near-field communication. Some IIoT-specific protocols include WirelessHART and ISA 100.11a. Both of these protocols are medium-range (>200 m), medium-throughput (250 kbps) protocols with guaranteed latencies as low as 10 ms. Longer-range protocols can also consume low power with small payloads and specialized narrowband (NB) and ultra-narrowband (UNB) modulation schemes. These characteristics are found in low-power wide-area networks (LPWANs) such as LoRaWAN, Sigfox, Weightless, and cellular-specific LPWANs such as NB-IoT and LTE-M.
The service and application layer involves the centralized collection of data from each node with cloud routers, smartphones, AI, and ML for the analysis and management of data for short-term and long-term operation and maintenance. IoT protocols have been developed for lightweight data processing for low latency, energy efficiency, and localized communication. Some of the data protocols used for robotics include MQTT, CoAP, XMPP, IPv6, UDP, DTLS, AMQP, uLP, and LLAP. This layer also involves the infrastructure to support and track all this data, which is critically important to smooth automation processes with IoT business cloud services, enterprise resource planning software, big data services, and robotic platform support. This middleware (IaaS, PaaS, SaaS) provides a platform of support to more readily employ IIoT.
Figure 1: The breakdown of products and services used in IIoT applications
Bringing AI and ML to industrial automation
With the large data rates and large number of sensor nodes, vast amounts of data must be processed in an IoRT system. Machine-learning algorithms can uniquely serve this niche due to their inherent ability to give computers the ability to learn from data. The layer of abstraction allows for data-driven forecasts as opposed to responding only to direct programming. This, in turn, allows for more complex, longer-term processing that has applications in predictive maintenance with parts and device life cycles. Robotic applications that have integrated ML include:
- Robot vision
- Robot navigation
- Field robotics
- Humanoid robotics
- Legged locomotion
- Modeling vehicle dynamics
- Medical and surgery robotics
- Off-road rough-terrain mobile robot navigation
Machine learning for correcting robot interaction with dynamic environments
Legacy robotics were traditionally designed for static environments with predictable outcomes with preset robot positions and direct programming. However, newer challenges emerge when a robot must interact with another dynamic object and may have to predict a position based upon visual or sensory feedback. There are various types of approaches for programming a robot to serve this environment, including imitation learning and reinforcement learning. In imitation learning, the robot can discern how to optimally move based upon feedback from previous movements and interactions with its environment. In kinesthetic teaching, the robot’s manual motions can be recorded and logged for feedback and learning. The robot can also learn through imitation learning by remote control through teleoperation, with the operator located at a large geographical distance. Some examples of teleoperation include the Mars rover and remote medical surgery. In this case, the robot can gain data from both learning its manual positions through sensor/video feedback (e.g., snapshots, motion/position detection, etc.) and from the additional information gained from the controller (e.g., haptic feedback, joystick, augmented gloves, etc.). In reinforcement learning, the robot can start with a rough program for accomplishing a task and self-improvement based upon positive and negative feedback from the environment. In this case, a robot can learn new tasks not previously demonstrated with merely an indirect goal (e.g., jumping, fast motions, etc.).
Machine-learning applications in industrial settings
Many industrial applications merely require repetitive tasks to be accomplished with little variance and degrees of freedom with respect to motion. In many cases, human-robot cooperation can be accomplished whereby humans and robots complete prespecified parts of a certain task. The future of IIoT also includes the use of fully automated factories that can be remotely monitored through a secure wireless connection. Both of these factories have the benefit to greatly benefit from optimization through AI and ML. Machine-vision applications can accomplish visual inspections via the pixel-based or feature-based approach wherein either pixels are manipulated to gain information on flaws such as scratches, surface roughness, and bubbles or general features are used for pass/fail inspection. In either case, a suitable machine-learning algorithm can be employed (e.g., decision tree) to train the classifier not only for fault detection but to accurately classify faults of each individual piece.
Pick-and-place machines also rely heavily on machine vision for feature extraction and real-time object recognition as well as additional learning algorithms for predicting grasp locations based on trial-and-error experimentation. In this case, there is not only the consideration of object recognition but the requirement for a rapid processing time in order to grasp an object on a conveyor belt. In the oil and gas industry, robotics has a high utility, as it bypasses the need to incorporate human effort in order to operate dangerous equipment in harsh environments. For instance, unmanned underwater robots are leveraged for the visual inspection of seabed wellheads in offshore rigs. In the oil and gas application, teleinspection and teleoperation can be incorporated to inspect and maintain equipment that would traditionally require a human. Mobile deliveries and motions can also be leveraged to shuttle materials around a facility with deep-learning algorithms to prevent collisions in dynamic environments. Simultaneous location and mapping algorithms can support the real-time 3D mapping of an environment in automated guided vehicles (AGVs) and autonomous mobile robots (AMRs) (Figure 2). Using both AGVs and AMRs in tandem with real-time location systems both simplifies and automates asset management on the plant floor.
Figure 2: AMRs and AGVs have been used in warehouses and factories to rapidly transport materials and streamline processes
Predictive maintenance and automated control of a plant with AI and ML algorithms
Traditionally, maintenance of industrial equipment has been based upon standard schedules and practices. An IIoT-based system removes the need for a schedule for plant maintenance, repairs, or replacements by dynamically assessing all machines through an IWSN. This eliminates the risk of factory downtime while eliminating the additional cost of unnecessary scheduled inspections. This, for instance, can be applied in oil and gas refineries to survey equipment regularly without human intervention, or to regularly assess moving parts in industrial equipment such as large motors through vibrational analysis. Robot equipment itself can be assessed and maintained through these analytics. The vast repertory of data collected over an extended period of time allows for learning and predictive capabilities with a fine-tuned perspective on the degradation process of any and all machines. Information such as usage history, remaining useful lifetime, classification models, and sensor data are all critical to industrial predictive maintenance.
Benefits for AI/ML for robotics
Deep-learning algorithms are heavily integrated in the future of IIoT. A fully automated plant floor is not possible without the use of learning algorithms for smooth day-to-day operations as well as long-term operation of the entire plant. The applications for AI/ML in robotics are numerous, all of which go into a great deal of depth with particular algorithms and data to be tracked for any specific industrial use case. Wireless technologies and AI processing are intrinsically tied together for complex industrial automation applications.
 and machine learning (ML) in relation to complex robotics used in industrial automation.){kind=link}