On January 21, the MIT Filmmakers Association hosted a hackathon - the MIT AI for Filmmaking Hackathon.
We, along with a talented group of screenwriters, composers, actors, and other AI enthusiasts, created the film MINDVAULT.
In MINDVAULT, AI symbolizes the illusion of control over one’s sensorial self, and transports the protagonist–and the viewer–to an alternative reality. AI becomes not a tool for the film’s visual construction, but another actor, co-existing–and sometimes merging–with the protagonist.
The visual effects in MINDVAULT are created with custom software built on top of Stable diffusion v1.5, developed by NikuAI. It encompasses various known techniques, such as Deforum for frame animation, Dreambooth for actress model finetuning, Real-ESRGAN for spatial upsampling the generated images to 4k resolution to match the rest of the film, and Google’s FILM for temporal interpolation to animate the diffusion results. Our custom python based framework is built on top of a fork of InvokeAI wrapper around Stable Diffusion, and includes multiple tools for end to end management of frame to frame video creation.
Finally, to perform region masking, we initially used Davinci Resolve’s MagicMask tool, but when we compared the results with those we got from RunwayML’s tracking tool, we saw a clear difference. The RunwayML results were significantly more accurate and the project flow was super smooth, enabling our team to quickly improve upon the quality of the final production.
MINDVAULT was featured in the 2023 Student World Impact Film Festival.
Paris Myers, Minoz, Sarah Markovic
Paris Myers, Carlos Centeno, Niko Dellic
Jackson Myers, Dana Reason