Maintenance is a required process in many commercial domains. Commonly, paper-based instruction manuals are digitised to improve accessibility to maintenance information. Among the different forms of digital media, augmented reality (AR) combined with machine learning has the potential to provide real-time user guidance and interaction. In this paper, we present an Augmented Reality Visual Intelligence (ARVI) framework, which combines visual perception with cognitive task reasoning to monitor user performance and provide contextualised guidance for maintenance tasks. The framework is implemented in Microsoft Hololens for a wheel removal operation. The implemented software can recognise aspects of the environment, objects and a user‟s hands, as well as infer the task being completed. The evaluation of visual perception models for object detection using a single task video for training obtained relatively high F1 scores, and a small-scale usability study reported some positive feedback on the application development.
License type:
PublisherCopyrights
Funding Info:
Agency for Science Technology and Research (A*STAR) Aerospace Programme Grant No. 162150016