A team of researchers from the Massachusetts Institute of Technology, Hong Kong University of Science and Technology and the Swiss Graduate School of Engineering Zurich has unveiled an innovative tool that can create digital replicas of physical objects and bring them to life in mixed reality (MR). The technology, called InteRecon, can recreate simple toys as well as complex gadgets, giving them interactive functions.

InteRecon offers a new way to preserve memories of favorite items. The researchers also note that the technology will help museums provide visitors with the opportunity to study exhibits in more detail.

The process of creating an interactive digital item involves 4 steps: 3D reconstruction of the object, adding physical transformations to simulate its operation, creating an interface with controls, and integrating multimedia content.

To work with the technology, you can use an iPhone app that scans an object and turns it into a 3D model. The model is then imported into the InteRecon MR interface, accessible through headsets such as Meta Quest. Virtual controls allow you to segment the object and customize your interaction with it.

The team will present their work at the ACM CHI conference in Japan. In the future, the researchers plan to improve InteRecon's physics-based modeling engine and use generative AI models to recreate lost personal items