Dynamic Projection Mapping onto Non-rigid Objects using Markers and Skeletal Information

Costumes are one of the major elements of the excitement in a live music performance. Although it is ideal to change the costumes to match the music theme each time, it is not practical because of the time and effort required to change the costume. In this paper, we propose a dynamic projection mapping method onto the costumes instead of “real” costume changes. Our method generates mapping images in advance, which are deformed and/or occluded in accordance with the dance choreography. Then, the projection positions are slightly adjusted in real time based on the difference between expected and actual standing position. This technique enables dynamic projection mapping onto a non-rigid object. Several experimental results verified the feasibility of the dynamic projection mapping.