Newsletter
Apple researchers have introduced a groundbreaking AI model, MLLM-Guided Image Editing (MGIE), capable of editing images based on text prompts. Developed in collaboration with researchers from the University of California, Santa Barbara, this model represents a significant advancement in image editing technology. Unlike existing models, MGIE reportedly handles a wide range of editing scenarios, from simple color adjustments to complex object manipulations.
The core of the MGIE is a Multimodal Large Language Model (MLLM), which interprets user requests and provides concise instructions for image editing. This approach enables the model to address ambiguous commands effectively, achieving reasonable editing results. For instance, the MLLM understands a request to “make a pizza more healthy”, and connects the term “healthy” with “vegetable toppings,” instructing the diffusion model to edit the image accordingly.
The MGIE can edit images from your text description
What sets MGIE apart from existing models like LLM-Guided Image Editing (LGIE) is its enhanced visual perception. While LGIE is confined to a single modality, MLLM within MGIE has access to the input image and cross-modal understanding, allowing for more descriptive instructions. This capability enables the model to identify specific regions in the image that need adjustment, such as brightening certain areas for a desired effect.
MGIE is now available as an open-source project on GitHub, offering code, data, and pre-trained models for download. Additionally, a web demo hosted on Hugging Face spaces allows users to experience the image editing capabilities of the model firsthand. However, Apple has not yet disclosed its plans for integrating MGIE into its products beyond research projects.
During Apple’s recent quarterly earnings call, CEO Tim Cook confirmed the company’s ongoing work on AI features for its devices. The company is likely to announce the results later this year. Business Standard expects these AI enhancements to extend to various Apple services, including Siri, Messages, and Apple Music. With the incorporation of generative AI features, users can anticipate improvements such as text summarization, personalized suggestions, and enhanced functionality across Apple’s ecosystem.