No one likes when Siri gives up on them just because their iPhone cannot establish a network connection or the cellular signal is weak, but Apple may have a solution for this: offline Siri.
An Apple patent application filing published yesterday, first spotted by AppleInsider, describes an offline personal assistant that would work even when the device was offline.
In order to make Siri more useful when not connected to the Internet, the personal assistant would need to run its voice recognition algorithm entirely on a user’s device rather than upload their spoken queries to the cloud for processing as she normally would.
In order to run speech-to-text processing and validation directly on a mobile device, however, some sort of hardware-accelerated natural language processing would be needed, and I suspect this would run on a specialized low-power coprocessor.
AppleInsider explains:
Apple suggests the use of an onboard system of modules to handle digital assistant queries that does not connect to the outside world. The collection of modules includes elements for speech synthesis, dialog processing, phonetic alphabet conversion based on a default vocabulary and user-created data and a natural language processing module, among other items.
Depending on the recognized words and the structure of the query, the offline digital assistant could then perform a number of predefined tasks stored in the task flow processing module or service processing module. These two stores contain models for commonly-requested tasks, such as setting a timer or to play a song saved to the device, with the appropriate task model performed depending on the request.
The patent filing proposes using machine learning to optimize the process.
Aa you probably now, recent devices that use the A11 Bionic processor or newer take advantage of Apple’s built-in Neural engine to accelerate machine learning in hardware. As an example, the ten times faster Neural engine in the latest A12 Bionic processor allows for features like real-time scene analysis and on-the-fly camera settings optimization.
This offline Siri mode descried in the patent would combine the best of both worlds: as soon as an Internet connection has been re-established, Siri would switch to using its servers.
Not only would Apple’s solution improve Siri on iPhone, but on other devices as well. It may be particularly important in the context of Apple Watch because Siri on the watch not only doesn’t work when your iPhone is out of range but also if the Bluetooth connection is weak.
Surely you’ve seen the annoying “I’ll tap you when I’m ready” message.