iPhone 6 Plus downsampling artifacts explained

iPhone resolutions (image 002)

With the iPhone 6 Plus, Apple took the unusual route of rendering apps at 3× scale into a backing store of 1,242-by-2,208 “logical” pixels which the GPU then scales down to the device’s native full HD screen resolution of 1,920-by-1,080 pixels.

This is different from every other iOS device to date and required to simplify app development as developers must simply create two times crisper graphics assets for standard Retina screens and three times denser images to account for the new “Retina HD” screens of the iPhone 6 and iPhone 6 Plus.

On the downside, the system down samples everything by approximately 13 percent to the native resolution so some unwanted artifacts do appear which degrade image quality slightly in certain situations.

Berlin-based iPhone and Mac developer Ole Begemann did an awesome job explaining the unwanted artifacts by taking highly detailed photographs of the iPhone 6 Plus screen using a closeup camera rig consisting of an Olympus E-M1 and a Panasonic GH3 using an Olympus M 60 mm f/2.8 lens.

The first test involves rendering a grid of vertical green hairlines with varying spacing on a black background using CoreGraphics, at the logical resolution of 1,242-by-2,208 pixels.

The lines are perfectly crisp when rendered into a backing store of 1,242-by-2,208 logical pixels. But after the iPhone 6 Plus does its downsampling magic for display on the 1,080-by-1,920 pixel screen, the crisp lines lose the black 1-pixel-wide gaps between each green line and a very subtle Moiré pattern is introduced.

iPhone 6 Plus (downsampling artifacts, Ole Begemann 001)

The good news is, apps (mostly games) which tap OpenGL or Apple’s Metal for rendering can opt out of the downsampling routine and draw directly into an image buffer which matches the native device resolution, so no blurring or artifacts occur.

“Rendering the test pattern using OpenGL at the native hardware resolution yields a perfect result,” said Begemann. “The lines are very bright and there’s no bleeding into neighboring pixels.”

However, as UI elements are introduced into the OpenGL view, some blurring occurs. You can observe the effect on your iPhone 6 Plus yourself: simply launch a game and then change the volume or pull down Notification Center.

The same thing happens when watching a video on the device. Here’s a video by Hendrik Kueck which demonstrates the effect using a test pattern image on an iPhone 6 Plus.

Because your iPhone 6 Plus uses hardware-accelerated decoding of iOS-friendly video formats, videos played using the stock Video app are rendered directly at the native screen resolution, skipping the scaling stage.

But the moment you bring up playback controls, the system must blend the blurred toolbars with the video. It does so by — you guessed right — upsampling the video view to the logical 3× scale in order to compute the blur before downsampling everything to the hardware resolution.

As a result, temporary degradation in image quality occurs when the video controls are overlaid over the video. “Pixel-perfect rendering is a thing of the past on the iPhone platform,” Begemann concludes.

The automatic downsampling mostly is not an issue in practice, he says, and developers should ignore the image degradation rather than waster resources accounting for unwanted image artifacts.

“Just do what Apple recommends and treat the iPhone 6 Plus as a 414×736@3× device and you’re gonna be fine in all but the most extreme corner cases,” he summed up.

With the iPhone 6 and iPhone 6 Plus, Apple has inevitably increased fragmentation of the platform in terms device resolution. In turn, developers now must account for non-Retina and Retina screens by providing normal, 2x and 3x assets in their apps, which makes downloads bigger and puts a strain on device storage when installing apps.

iPhone resolutions

Developers behind the PaintCode app have put together this webpage which does an excellent job explaining all the various screen sizes and pixel configurations on Apple’s mobile devices. The image above is part of the larger infographic so check out the webpage to inform yourself on the topic.

Bottom line: the vast majority of iPhone 6 Plus users won’t notice any blurring with naked eye, but those that look closer will notice some artifacts in certain situations, induced by the downscaling.

If Begemann’s findings have piqued your interest, he’s published the code on GitHub that you can download and build the same two demo apps he used to run them on your iPhone 6 Plus and observe image degradation yourself.

By the way, screenshots taken on the iPhone 6 Plus are always saved in the logical 3× size at 1,242-by-2,208 pixels.

[Ole Begemann via The Loop]