Apple’s upcoming Live Captions feature will automatically transcribe any audio content

Apple’s Live Captions feature will auto-transcribe any audio on your iPhone, iPad and Mac, including in phone calls, on video streams and in real life.

A photograph showing an iPhone held in a person's hand, demonstrating the Live Captions feature in action during a FaceTime video call
Image credit: Apple
  • Live Captions is an upcoming new feature for the iPhone, iPad and Mac that uses artificial intelligence to let you follow along with any audio content.
  • It will automatically transcribe audio content whether you’re on a FaceTime call, watching clips in your favorite social media app or having a conversation with a real person next to you.
  • Don’t confuse Live Captions with Live Text, which was introduced alongside iOS 15 to make it easy to copy text from images as if it were raw text.

Your Apple devices will soon read any audio

Live Captions doesn’t care about the audio source. With the feature turned on, any audio gets automatically transcribed whether you’re on a phone call or talking to a person standing next to you. The transcribed text appears in an app as an overlay similar to regular captions, and you can adjust the font size to your liking.

Apple mentioned the following scenarios where Live Captions may come in handy:

  • Phone calls
  • FaceTime calls
  • Video conferencing apps
  • Social media apps
  • Streaming media content
  • Talking to a person next to you

If you’ve used live captions in third-party apps, you know what to expect. The accuracy of speech-to-text has improved greatly over the years though muffled dialogue is still problematic. That’s why Apple cautions that the accuracy of Live Captions may vary. The feature “should not be relied upon in high-risk situations,” notes the company. And rightfully so, perhaps, Live Captions will be launching in beta. Read: How to use live captions in Google Chrome for Mac

Live Captions in group FaceTime: Will it work?

You can even use Live Captions on group FaceTime calls. In that case, all transcribed dialogue will be attributed to call participants so that people with hearing disabilities can follow along easily. And when used on the Mac, a FaceTime call participant can even type out their response and have it spoken aloud in real time to others who are part of the conversation.

A featured image demonstrating how Apple's Live Captions feature attributes auto-transcribed dialogue to call participants during group FaceTime calls on iPhone
Image credit: Apple

Privacy and security

Live Captions owes its magic to machine learning and other advanced technology, such as computer vision and artificial intelligence. Like with other Apple features, Live Captions protects your privacy. This means that live captions are generated on the device. Your data never leaves the device nor is it uploaded or shared. Apple has previewed Live Captions, alongside Apple Watch Mirroring and other upcoming assistive features, in a post published on its Apple Newsroom page so check it out if you’re interested in additional aspects of the upcoming features.

System requirements for Live Captions

The Live Captions feature works on these devices and operating systems:

  • iPhone 11 and later
  • iPad models with the A12 Bionic chip and later
  • Apple silicon Macs

iPads with the A12 Bionic include the third-generation iPad Air, fifth-generation iPad mini and eighth-generation iPad. If your iPad is one of those models or newer, it will run Live Captions like a boss. Language support for Live Captions will be initially limited to English (US, Canada).

When will Apple launch Live Captions publicly?

Image credit: Apple

Apple first previewed Live Captions on Tuesday, May 1. The company provided no firm launch date for the feature aside from saying it’s coming later this year. “These features will be available later this year with software updates across Apple platforms,” Apple notes. Expect Live Captions to launch in the fall alongside iOS 16, iPadOS 16 and macOS 13.0. Read: How to use Live Text to extract text from photos

Not just for hard of hearing

Live Captions is an assistive feature that caters to folks with hearing issues. But as with all accessibility features, Live Captions can be used by anyone regardless of their hearing. This is one of the ways technology improves our lives for the better. Live captions are already present in Google’s Chrome browser and many social media apps. The next time a friend asks you what’s so special about machine learning on the iPhone, impress them by giving them a live demo of Live Captions.