Reality Forge: Interactive Dynamic Multi-Projection Mapping

Creating digitals assets, especially surface textures is a tedious, and often unintuitive task. We built a fun to use, interactive projection-mapping system that aims to simplify this process. This demonstration is based on our fully dynamic multi-projection mapping system. It enables us to bring virtual worlds into shared physical spaces, as it allows for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world to continuously adapt the color output of each projector pixel. This real-time solution uses off-the-shelf graphics hardware, depth cameras and projectors. Based on this system we present a non-destructive, immersive, fully dynamic mixed-reality painting system for real-world objects that combines the workflow of a traditional airbrush artist with the power of digital media. In contrast to a real-world airbrush artist, we are not limited to pigmented paint but can apply elaborate material and lighting effects. Since our system interactively changes the appearance of any Lambertian real-world object, designing the surface texture of a 3D-Object becomes a collaborative experience. We leverage this new tool in the context of cultural heritage and design helping teachers, researchers and designers to experiment with different materials, surroundings and painting techniques in the real-world.