“내일 오는 상품 골라주네”…네이버 쇼핑 AI, 배송 조건 따져 추천

A team of researchers from the Gwangju Institute of Science and Technology (GIST) has developed artificial intelligence (AI) robotics technology that not only identifies objects but also operates them. Professor Kim Won-hwan's research team at GIST's AI Department unveiled on the 17th that they have created an artificial intelligence technology capable of recognizing not only the objects around a robot but also the detailed components and functions like handles and buttons.

The existing technology used by AI to understand its environment is known as '3D Scene Graph' (3D Scene Graph, 3DSG), which is a sort of 'robotic 3D knowledge map' that recognizes objects in the space to understand and move around or plan the order of work. However, this technology has limitations as it can only identify objects like chairs and microwaves, and it cannot specifically identify small parts or their functions.

The GIST research team proposed a 'Unified 3DSG' (Unified 3D Scene Graph) to overcome this limitation, which not only recognizes the position of objects and parts but also their functions and operation methods. They developed an AI technology called 'Open-Vocabulary Part-Aware 3D Scene Graph Generation' (OP3DSG) that automatically constructs this unified 3D scene graph from the video footage of the robot's camera.

Using this technology, the robot can find the main objects from the camera footage and then search for the detailed parts that are expected to be part of that object. For example, when a robot discovers an oven, it needs to find the buttons, door, and handles that make up the oven to navigate to those objects. The research team explained that this works by stating, "For instance, when an oven is found, the robot needs to find the buttons, door, and handles that make up the oven."

The team utilized a large language model (LLM) to review and enhance the functional relationships and possible actions between parts and objects. The research team demonstrated the possibility of robots independently performing tasks by using this technology on a mobile robot named 'Stretch 3.' They showed how the robot could locate a remote control to turn on a television or pick up a towel and put it in a basket when given the instruction, "Please tidy up my laundry."

Professor Kim emphasized that the key of this research is linking 'what is where' and 'which parts can be used and how they can be used' together, and stated that this technology could become a foundational technology for service robots to understand complex real-world environments and carry out human commands in the future. The participating graduate student Kim Eun-mo noted that if this technology is combined with manipulation and control technologies for robots, it could be practically utilized, helping elderly people or those with limited mobility by simply giving simple instructions like "Please turn on the light."