The United States Patent and Trademark Office (USPTO) today published an interesting patent application filed for by Apple, which indicates that the Cupertino company might be exploring much deep integration between the Siri personal assistant and its iMessage message platform.
Titled “Intelligent automated assistant in a messaging environment”, the company’s patent application proposes chatbot-style Siri functionality within the context of the Messages app for those times when using voice-based interactions would be impractical, like in noisy environments such as libraries, movie theaters and so forth.
The patent application appears to be an enhanced version of Apple’s prior invention, titled “Virtual assistant in a communication session,” filed for last year and outlining basic Siri-Messages integration via bot-style commands.
Today’s patent application includes deep Siri integration with audio, video and image files, as well as with deep links to websites/apps and more. The system could potentially permit users to use image and video recognition with Siri, asking the personal assistant to, say, send an image of a Volkswagen Beetle to a contact or return a text-based web answer.
Siri cannot process images in its current form.
Other image-related queries could include “Where is this?”, “What insect is this?”, “Which company uses this logo?” and more. Siri would be able to remember user preferences and data, too. In one embodiment, a user sends a picture of a bottle of wine and says, “I like this wine” to have Siri remember this preference.
She would also be able to store custom text for later retrieval.
Lastly, Apple proposes chatbot-like functionality where the AI-driven assistant would offer its services to chat participants in a manner not unlike the AI assistant in Facebook’s Messenger app. The system could intelligently provide relevant information in the text form about stuff like a nearby restaurant while being able to suggest transportation options and even create an entry in each chat participant’s calendar about an upcoming meeting.
Here’s an excerpt from the patent abstract:
User input can be received and in response to receiving the user input, the user input can be displayed as a first message in the GUI. A contextual state of the electronic device corresponding to the displayed user input can be stored. The process can cause an action to be performed in accordance with a user intent derived from the user input. A response based on the action can be displayed as a second message in the GUI.
Siri currently includes a feature that allows you to edit your queries by typing, but it’s not very practical as it’s only available after the user has initially issued a voice-based command to the personal assistant.
Apple’s technology, assuming it sees the light of day, would permit users to review previous Siri interactions in a chronological format. Today’s patent application was first filed for in May 2016 and credits Apple engineers Petr Karashchuk, Tomas A. Vega Galvez and Thomas R. Gruber as its inventors.