Apple plans to run ChatGPT-like Siri AI locally

Apple is working on what appears to be a technically impressive development that could change the way we interact with our iPhones for the better. The company plans to run advanced language models (LLMs), similar to ChatGPT, directly on iPhones.

This step could significantly improve Siri's functionality and intelligence and finally make the previously relatively "dumb" Siri smarter - without sending data to the cloud and raising privacy issues. Apple's research team has figured out how this should work technically in this paper described.

LLMs on the iPhone – the theory

Apple's research team recently published the paper mentioned above, which describes an innovative approach to running Large Language Models (LLMs) on iPhones. This research shows that it is possible to run complex AI algorithms, which typically require powerful servers, on the comparatively limited resources of a smartphone.

By using specialized optimization techniques and efficient code management, Apple has found a way to reduce the computational load on these models so that they can run directly on the iPhone without significantly impacting performance or battery life.

The technical implementation

When implementing Large Language Models (LLMs) on iPhones, Apple faces a major challenge: These models require a lot of data and storage space, but iPhones have limited memory compared to servers. To solve this problem, Apple's researchers have developed a clever technology that uses the iPhone's flash memory - the memory in which apps and photos are also stored.

Instead of relying on the commonly used main memory (RAM), this method stores the AI ​​model's data in flash memory. This has two advantages:

• First, flash storage in mobile devices is often more widely available than RAM.
• Second, Apple uses two special techniques that minimize data transfer and maximize flash memory throughput.

Windowing and row-column bundling

The first technique is called “Windowing” and works similarly to recycling: instead of constantly loading new data, the AI ​​model reuses some of the data it has already processed. This reduces the need to constantly access memory and makes the process faster and smoother.

The second technique, “Row-column bundling“, is comparable to reading larger sections of text in a book instead of individual words. This method aggregates data more efficiently so that it can be read from flash memory more quickly, which in turn improves the AI's ability to understand and generate language.

According to the research paper, these methods make it possible to run AI models that are equivalent to up to twice the iPhone's available storage capacity. This method could increase speed by 4x to 5x on standard processors (CPUs) and 20x to 25x on graphics processing units (GPUs). That's why this is a critical breakthrough for deploying complex LLMs in resource-limited environments like the iPhone.

Potential impact for us users

Implementing LLMs on iPhones could have far-reaching impacts on user experience.

First, it would finally allow Siri to handle more complex requests and provide more precise, contextual responses. This could make interacting with the iPhone more intuitive and efficient.

Second, this technology could open up new applications in areas such as personal knowledge management, advanced translation capabilities, and even the development of interactive, education-based applications.

However, these are just speculations and theoretical possibilities for now. There is no question that Apple plans to make some kind of ChatGPT work on the iPhone, iPad and Mac. However, we can't say much about what the implementation will look like at the moment.

Data protection and performance benefits

While the ability to run LLMs on a Mac or iPhone sounds good, it also presents challenges, particularly when it comes to privacy and security. Apple is known for its focus on protecting user data and sending as little of this data as possible from the devices themselves to the Apple servers.

The company must also ensure that these new AI functions comply with strict EU data protection guidelines. This includes handling sensitive data, preventing unwanted data collection and ensuring that the AI ​​models are not used for inappropriate purposes.

But the matter will also require a lot of technical resources. Integrating AIs based on language models that are then used by millions of people every day would put a huge strain on a cloud service. In this respect, too, a local execution of the AI ​​might be the better option for Apple - especially since you could then use many of the smart assistant's functions even without an internet connection.

Our Conclusion

Apple's research into making LLMs run locally on iPhone and Mac marks an exciting milestone in the development of Siri. It shows that Apple - as well as users - has realized that the current version of Siri lags enormously behind chatbots like ChatGPT.

Although we now have a paper here that shows us the technical possibilities for the future and no current features have been addressed in the next iOS, we assume that something like this can possibly be expected in iOS 18.

Jen Kleinholz( Founder & Sir Apfelot )

Jens has been running the blog since 2012. He acts as Sir Apfelot for his readers and helps them with technical problems. In his spare time he rides electric unicycles, takes photos (preferably with the iPhone, of course), climbs around in the Hessian mountains or hikes with the family. His articles deal with Apple products, news from the world of drones or solutions to current bugs.