MindPainter: Efficient Brain-Conditioned Painting of Natural Images via Cross-Modal Self-Supervised Learning
Published in The 39th Annual AAAI Conference on Artificial Intelligence 2024, 2024
Recommended citation: Muzhou Yu, Shuyun Lin, Hongwei Yan, Kaisheng Ma*. "MindPainter: Efficient Brain-Conditioned Painting of Natural Images via Cross-Modal Self-Supervised Learning." The 39th Annual AAAI Conference on Artificial Intelligence 2024 https://arxiv.org/pdf/xxxx.xxxxx
This paper proposes MindPainter, a method for efficient brain-conditioned image editing using brain signals of visual perception as prompts. By employing cross-modal self-supervised learning, it directly reconstructs masked images with pseudo-brain signals generated by the Pseudo Brain Generator, enabling seamless cross-modal integration. The Brain Adapter ensures accurate interpretation of brain signals, while the Multi-Mask Generation Policy enhances generalization for high-quality editing in various scenarios, such as inpainting and outpainting. MindPainter is the first to achieve efficient brain-conditioned image painting, advancing direct brain control in creative AI.
Recommended citation: Muzhou Yu, Shuyun Lin, Hongwei Yan, Kaisheng Ma*. “MindPainter: Efficient Brain-Conditioned Painting of Natural Images via Cross-Modal Self-Supervised Learning.” The 39th Annual AAAI Conference on Artificial Intelligence 2024