Visual data, be it 2D pictures or full 3D models, are unquestionably one of the cornerstones of artificial intelligence today as it enables everything from self-driving cars to full automation in smart factories. In the field of autonomous manufacturing especially, the value of visual data cannot be overstated. Beyond simply training a model to recognize parts or detect defects, having full 3D models of products is a vital tool for virtual AI training in the digital twin of production. However, commensurate with its value, visual data is highly challenging to capture and work with as well. While methods exist to help alleviate this, such as creating synthetic image data or using less, tuned data, there aren’t many ways to reduce the hefty requirements for 3D models, but in the future that might change. Recently researchers have created an AI model capable of generating 3D models from simple 2D images, opening a door for companies to economically created models efficiently and cost-effectively.
Recently, researchers at Google developed a method to use AI to create realistic 3D models from simple 2D pictures or even extend images into short videos. While the technology is still in its infancy and will likely never replace hand-made, full-detail models, the ability to do something that once took days or even weeks in hours shouldn’t be dismissed out of hand. For products or components that an AI model would only be required to recognize as a basic shape, these AI-generated models may prove sufficient for training purposes. On the other hand, for objects where more detail is required the AI-generated 3D models could serve as a starting point for human workers to flesh out more complete versions which include physical properties such as material or weight. In either case, taking a single or even multiple photos is far easier than the work required to generate highly detailed 3D – especially when the added fidelity might be wasted. For companies looking to take the next step in their digital transformation and reap the benefits of advanced AI in the factory, AI-generated 3D models may become a valuable tool on the path to digitization. This is especially true for companies who may internally lack the necessary skills to create the models in-house or find it impractical for a variety of reasons.
When training an AI system to pick parts, having a complete 3D model of the part is vital for the training process when using a digital twin. Although many companies will already have these models prepared, that is not something that can be taken for granted. For companies that work with a large number of parts from 3rd party vendors or focus on logistics, requiring them to handle goods from a large number of vendors, it is simply impractical to manually create full 3D models for each component an AI system would encounter. This is where AI-generated models come into play. By taking a simple picture, the AI model can create a 3D model suitable for training and when combined with techniques such as synthetic data, could generate an entire training data set from just a handful of photos. This enables fast and efficient training when combined with a digital twin of production on which comprehensive models of AI-controlled systems exist.
In the world of artificial intelligence, there is rarely a single method or solution that handles every situation, rather a wide variety of techniques bubble up from different industries and research groups, each providing some benefit but few presenting a complete picture. When combined with other techniques, such as synthetic data, using AI to generate 3D models from pictures could become another valuable tool in bringing advanced AI-controlled robotics to the factory by enabling efficient training on the digital twin of production. This could be a stepping stone for companies to easily import the real world into the virtual one without requiring specialized skills – a vital component in the puzzle of true autonomous factories.
Courtesy: Siemens
