Augmented Reality Instruction for Object Assembly based on Markerless Tracking




Li-Chen Wu, I-Chen Lin, Ming-Han Tsai

Abstract

Conventional object assembly instructions are usually written or illustrated in a paper manual. Users have to associate these static instructions with real objects in 3D space. In this paper, a novel augmented reality system is presented for a user to interact with objects and instructions. While most related methods pasted obvious markers onto objects for tracking and constrained their orientations or shapes, we adopt a markerless strategy for more intuitive interaction. Based on live information from an off-the-shelf RGB-D camera, the proposed tracking procedure identifies components in a scene, tracks their 3D positions and orientations, and evaluates whether there are combinations of components. According to the detected events and poses, our indication procedure then dynamically displays indication lines, circular arrows and other hints to guide a user to manipulate the components into correct poses. The experiment shows that the proposed system can robustly track the components and respond intuitive instructions at an interactive rate. Most of users in evaluation are interested and willing to use this novel technique for object assembly.

Demo Video

Experiments

The proposed system was built on a PC with a quad-core, 3.4 GHz CPU and 12 GB RAM. Currently, only two threads are invoked. We adopted the ASUS Xtion Pro Live [ASUSTek Computer Inc. ] as our RGB-D camera device.

Figure. The illustration of the indication lines. (a) The blue circle is the indication circle (b) The red lines are the alignment lines. (c)(d)(e) shows each set of the components and their representative matched points.

Figure. The design of our user interface.

Publication

Li-Chen Wu, I-Chen Lin, Ming-Han Tsai, "Augmented Reality Instruction for Object Assembly based on Markerless Tracking," Proc. ACM SIGGRAPH Symp. on Interactive 3D Graphics and Games (I3D'16), pp. 95-102, Redmond, WA, Feb. 2016.

Paper: preprint_version (about 2.68MB), published version (link to the ACM digital library)

BibTex

@inproceedings{WuI3D16,
author = {Wu, Li-Chen and Lin, I-Chen and Tsai, Ming-Han},
title = {Augmented Reality Instruction for Object Assembly Based on Markerless Tracking},
year = {2016},
publisher = {Association for Computing Machinery},
doi = {10.1145/2856400.2856416},
booktitle = {Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games},
pages = {95–102},
numpages = {8},
location = {Redmond, Washington},
series = {I3D '16}
}

Acknowledgements

The authors appreciate helpful comments from reviewers. This paper was partially supported by the Ministry of Science and Technology, Taiwan under grant no. MOST 104-2221-E-009-129-MY2 and 104-2218-E-009-008.