In this project is designed a gamification-based application for the cultural heritage sector that aims to enhance the learning and fruition of museum artworks. The application encourages users to experience history and culture in the first person, based on the idea that the artworks in a museum can tell their own story, thus improving the engagement of the museums and providing information on the artwork itself.

We propose an application that allows museum visitors to create a deepfake video of a sculpture directly through their smartphone. More in detail, starting from a few live frames of a statue, the application generates in a short time a deepfake video where the statue talks by moving its lips synchronized with a text or audio file. The application exploits an underlying generative adversarial network technology and has been specialized on a custom statues dataset collected for the purpose. Experiments show hat the generated videos exhibit great realism in the vast majority of cases, demonstrating the importance of a reliable statue face detection algorithm.

 The final aim of our application is to make the museum experience different, with a more immersive interaction and an engaging user experience, which could potentially attract more people to deepen classical history and culture.

We created a dataset of statues by acquiring videos from the Gipsoteca Museum of Sapienza University of Rome (Rome, Italy). In the Gipsoteca ("gypsotheque") is possible to admire plaster casts of Greek sculptures from the Archaic period to Hellenism. In our project, we consider different sculptures (i.e. Poseidon, Hercules, Zeus, Minerva Tritonia, etc.) among the 1200 plaster casts. We demonstrate the power of our approach on the collected dataset, which will be expanded in future works.