Apple MGIE – AI for text-based image editing

Almost a month ago We introduced you to Apple’s multimodal language model “Ferret” here in the blog and in the podcast. This revealed for the first time the iPhone manufacturer's AI efforts being carried out in secret. With MGIE there is now a new Apple AI that brings together images and text input. Because MGIE stands for “Multimodal large language model Guided Image Editing”. A multimodal language model (MLLM) is used here for image processing. Corrections, filters, effects on individual points or objects and so on should be possible using simple prompts. You can try the whole thing out using an online tool.

On the left you can see the image I uploaded and the corresponding prompt. On the right is the output image and the interpretation of the prompt. It can be said that some work is still needed on MGIE.
On the left you can see the image I uploaded and the corresponding prompt. On the right is the output image and the interpretation of the prompt. It can be said that some work is still needed on MGIE.

“Remove the clouds and give the image a sunny yellow cast”

The commands you can give MGIE to beautify landscape photos are something like this. Changing hair color in selfies can also be used to predict the outcome in the hair salon. After initial tests, the specialist press for MGIE even says that some commands transmitted via text for changes to the input image could replace manual editing - for example in Photoshop or similar programs. This certainly also applies to the filters and effects integrated into the Photos app that Apple offers on Mac, iPhone and iPad. You can find a successful test example here: b. at 9to5Mac.

Use Apple MGIE yourself: GitHub and HuggingFace

If you are looking for more background information on MGIE, you will find, among other things find it on GitHub. Because Apple made the AI ​​model available open source there. In addition to the raw data, there are also some explanations of how it works and how it is used. If you don't know what to do with the data and can't create your own user interface (like me), then just use the web tool equipped with MGIE at HuggingFace. Like Ferret, MGIE also comes with a scientific paper that you download here can. There is a summary with the most important points at VentureBeat.

For everyone who would prefer to have the information about MGIE translated, here are the most important things:

  • Expressive text-based editing: MGIE can create concise and clear instructions that effectively guide the editing process. This not only improves the quality of the edits, but also increases the overall usability.
  • Photoshop-like modification: MGIE can perform common Photoshop-like edits such as: B. Crop, resize, rotate, flip and add filters. The model can also perform more advanced edits, such as: E.g. change the background, add/remove objects and blend images.
  • Global photo optimization: MGIE can optimize the overall quality of a photo, e.g. B. Brightness, contrast, sharpness and color balance. The model can still apply artistic effects such as sketching, painting and cartooning.
  • Local editing: MGIE can edit specific regions or objects in an image, such as: B. Faces, eyes, hair, clothing and accessories. The model can also change the attributes of these regions or objects, such as shape, size, color, texture, and style.

We're excited about what's new in iOS 18

It is assumed that Apple will build various AI functions into iOS 18 and probably also into iPadOS 18 and macOS 15. In addition to an improvement for the voice assistant Siri, Apple now seems to be concentrating more and more on the area of ​​image recognition and image processing. It is possible that the focus is not only on creative purposes, but also on operating aids. The recognition of the surroundings is already part of the iPhone camera functions to a certain extent - but it could now be significantly expanded. We'll know exactly from June when Apple holds WWDC24.

Did you like the article and did the instructions on the blog help you? Then I would be happy if you the blog via a Steady Membership would support.

Post a comment

Your e-mail address will not be published. Required fields are marked with * marked

In the Sir Apfelot Blog you will find advice, instructions and reviews on Apple products such as the iPhone, iPad, Apple Watch, AirPods, iMac, Mac Pro, Mac Mini and Mac Studio.