Apple plans to run ChatGPT-like Siri AI locally

Siri on iPhone featured image

Apple is working on what appears to be a technically impressive development that could change the way we interact with our iPhones for the better. The company plans to run advanced language models (LLMs), similar to ChatGPT, directly on iPhones.

This step could significantly improve Siri's functionality and intelligence and finally make the previously relatively "dumb" Siri smarter - without sending data to the cloud and raising privacy issues. Apple's research team has figured out how this should work technically in this paper described.

Many iPhone users would like to see a smarter Siri with the capabilities of ChatGPT (Photo: Omid Armin).
Many iPhone users would like to see a smarter Siri with the capabilities of ChatGPT (Photo: Omid Armin).

LLMs on the iPhone – the theory

Apple's research team recently published the paper mentioned above, which describes an innovative approach to running Large Language Models (LLMs) on iPhones. This research shows that it is possible to run complex AI algorithms, which typically require powerful servers, on the comparatively limited resources of a smartphone.

By using specialized optimization techniques and efficient code management, Apple has found a way to reduce the computational load on these models so that they can run directly on the iPhone without significantly impacting performance or battery life.

The technical implementation

When implementing Large Language Models (LLMs) on iPhones, Apple faces a major challenge: These models require a lot of data and storage space, but iPhones have limited memory compared to servers. To solve this problem, Apple's researchers have developed a clever technology that uses the iPhone's flash memory - the memory in which apps and photos are also stored.

Instead of relying on the commonly used main memory (RAM), this method stores the AI ​​model's data in flash memory. This has two advantages:

  • First, flash storage in mobile devices is often more widely available than RAM.
  • Second, Apple uses two special techniques that minimize data transfer and maximize flash memory throughput.
Flash memory offers significantly higher capacity, but has a much lower bandwidth compared to DRAM and CPU/GPU caches and registers.
Flash memory offers significantly higher capacity, but has a much lower bandwidth compared to DRAM and CPU/GPU caches and registers.

Windowing and row-column bundling

The first technique is called “Windowing” and works similarly to recycling: instead of constantly loading new data, the AI ​​model reuses some of the data it has already processed. This reduces the need to constantly access memory and makes the process faster and smoother.

The second technique, “Row-column bundling“, is comparable to reading larger sections of text in a book instead of individual words. This method aggregates data more efficiently so that it can be read from flash memory more quickly, which in turn improves the AI's ability to understand and generate language.

According to the research paper, these methods make it possible to run AI models that are equivalent to up to twice the iPhone's available storage capacity. This method could increase speed by 4x to 5x on standard processors (CPUs) and 20x to 25x on graphics processing units (GPUs). That's why this is a critical breakthrough for deploying complex LLMs in resource-limited environments like the iPhone.

Potential impact for us users

Implementing LLMs on iPhones could have far-reaching impacts on user experience.

First, it would finally allow Siri to handle more complex requests and provide more precise, contextual responses. This could make interacting with the iPhone more intuitive and efficient.

Second, this technology could open up new applications in areas such as personal knowledge management, advanced translation capabilities, and even the development of interactive, education-based applications.

However, these are just speculations and theoretical possibilities for now. There is no question that Apple plans to make some kind of ChatGPT work on the iPhone, iPad and Mac. However, we can't say much about what the implementation will look like at the moment.

Data protection and performance benefits

While the ability to run LLMs on a Mac or iPhone sounds good, it also presents challenges, particularly when it comes to privacy and security. Apple is known for its focus on protecting user data and sending as little of this data as possible from the devices themselves to the Apple servers.

The company must also ensure that these new AI functions comply with strict EU data protection guidelines. This includes handling sensitive data, preventing unwanted data collection and ensuring that the AI ​​models are not used for inappropriate purposes.

But the matter will also require a lot of technical resources. Integrating AIs based on language models that are then used by millions of people every day would put a huge strain on a cloud service. In this respect, too, a local execution of the AI ​​might be the better option for Apple - especially since you could then use many of the smart assistant's functions even without an internet connection.

Our Conclusion

Apple's research into making LLMs run locally on iPhone and Mac marks an exciting milestone in the development of Siri. It shows that Apple - as well as users - has realized that the current version of Siri lags enormously behind chatbots like ChatGPT.

Although we now have a paper here that shows us the technical possibilities for the future and no current features have been addressed in the next iOS, we assume that something like this can possibly be expected in iOS 18.

My tips & tricks about technology & Apple

Did you like the article and did the instructions on the blog help you? Then I would be happy if you the blog via a Steady Membership would support.

Post a comment

Your e-mail address will not be published. Required fields are marked with * marked

In the Sir Apfelot Blog you will find advice, instructions and reviews on Apple products such as the iPhone, iPad, Apple Watch, AirPods, iMac, Mac Pro, Mac Mini and Mac Studio.