#Vernieracuity

[appliancesreviews]

Overview of 8K AI Upscaling Technology

Experts have not yet finished arguing about the prospects for 4K TVs, but companies are already actively offering 8K models. Basically, skeptics of this trend use two fairly convincing arguments.

Introduction

First, many have not forgotten the relatively recent history of 3D TVs. The rapid growth of its popularity began after a demonstration at CES and IFA in 2010. Many experts quite logically predicted its excellent prospects. Unfortunately, the high cost of producing 3D content has become an insurmountable obstacle to its further development. As a result, almost all companies abandoned 3D support in their TVs. The second argument is based on the eye capabilities. Unfortunately, its maximum sharpness is significantly inferior, for example, to eagle vision. Of course, it would never occur to anyone to watch a 75-inch TV from a distance of, for example, up to 1 meter. But at a more adequate distance, the sharpness of our vision does not allow us to distinguish individual pixels smaller than a certain size. That is, vision does not perceive the difference between, for example, Full HD and 4K TVs with the same screen size. Perhaps the owners of a falcon or an eagle will be able to please their pets with a high-quality image, but they themselves will not see the difference. But these arguments have counterarguments. For example, companies are actively improving 8K AI upscaling technology.

At a minimum, 8K TVs with AI upscaling will provide better quality playback of lower resolution content. In addition, the results of experiments open up some new aspects. In particular, South Korean scientists have studied the effect of vernier acuity (hyperacuity) to image formation by our brain. Their results allowed scientists to hypothesize that the brain is less tired when perceiving content with high resolution. At the same time, it watching provides richer emotions due to high realism of the perceived content.

Resolution

Of course, any TV provides maximum playback quality for corresponding native content in pixel-to-pixel mode. Unfortunately, the real situation on the market is often different from the ideal. For example, the capabilities of modern TVs often significantly exceed the resolution of the content offered. But sometimes devices, on the contrary, do not support content resolution. Of course, companies solve this problem. Modern algorithms use several basic technologies. The simplest of them increases the image resolution, replacing each pixel with four copies of it. Bilinear interpolation is more efficient. These algorithms compare the missing pixel with the two nearest ones, forming a linear gradient to sharpen the image.

Even more efficient bicubic interpolation uses 16 closest points in all directions, increasing color accuracy. Unfortunately, it often provokes a halo effect.

Main methods

A list of the most popular modern methods includes supersampling, checkerboard rendering and upscaling. Supersampling is a spatial anti-aliasing method for removing the aliasing in the rendered images. In fact, it first renders the image in 4K, and then displays it in 1080p resolution. As a result, the detail of the final image increases. Of course, rendering requires sufficient processing power. The Insects screenshot from Microsoft on Full HD TV demonstrates this effect.

Checkerboard Rendering reduces computing power requirements. In particular, it removes every second pixel and forms an image from the rest. The diagrams show the differences between Standard Rendering and Checkerboard Rendering.

Removing pixels slightly reduces the image resolution, but significantly reduces the amount of computation. In fact, upscaling only resizes while maintaining its proportions. It can be considered the simplest method of converting image resolution. In this case, the TV processor forms additional pixels, providing the required resolution. Accordingly, the final image is a set of new averaged pixels.

Unfortunately, the final image comparison demonstrates a significant decrease in quality compared to native content.

Upscaling

A few years ago, Samsung was the first to use the marketing term UHD Upscaling. Strictly speaking, almost any smartphone or computer has long supported this feature. Of course, its quality directly depends on the content. For example, the quality of SD - UHD Upscaling is significantly lower compared to Full HD - UHD Upscaling due to the insufficient number of pixels in the original image. The picture illustrates the difference in quality between SD / HD / Full HD - 4K Upscaling.

But its visualization is almost disappearing for Full HD. The first 4K TVs did not have an AI processor and simply scaled the image to the entire screen area using traditional interpolation and increasing the pixels size. Unfortunately, bilinear interpolation provides a fairly low quality due to loss of information, reduced detail, blurring outlines of objects, noise, etc. Moreover, mutual influence complicates their elimination. For example, sharpening increases noise, and conversely, reducing noise reduces detail. But companies have improved technology by developing AI Upscaling. Using a powerful high-performance processor radically improved upscaling quality. Today, almost all leaders use 8K AI Upscaling technology under different names. But, of course, they all use the same principles. Samsung is one of the leaders in the development of this technology. The company uses the powerful 8K Quantum processor, optimized for neural networks and AI processing. Samsung's MLSR (Machine Learning Super Resolution) technology uses machine learning, object recognition algorithms, and software filters.

Samsung AI Upscaling

Simplified, this technology uses a constantly updated database of the images pairs in high and low resolution. It's formed as follows. The system selects an 8K image and lowers its resolution, adding noise and reducing the clarity of outlines. In addition, it performs several compression/decompression cycles with different quality levels to add characteristic artifacts. Generated images pairs of different quality are divided into classes, processed by algorithms based on neural networks and stored in a database. Subsequently, the processor analyzes the low-resolution image, finds the corresponding algorithm in the database with filters, and replaces it with an 8K pattern. In fact, the 8K AI Upscaling technology generates an image by analogy with the creation of Frankenstein from 8K patterns in the database. During processing, AI analyzes image fragments up to 60 times per second, correcting saturation, detail and white balance. CPU uses code lines of patterns base for localization of processing fragments. Such an algorithm increases the recognition accuracy of displayed objects. As a result, the processor almost accurately distinguishes, for example, the sky from its reflection in the windows. Moreover, the technology works quite well even with streaming video. But cloud computing for such content is not possible. Previously, AI machine learning used a formula bank, which is downloaded to the TV and updated periodically. But new Samsung Q950TS supports innovative Deep Learning using a neural network to expand machine learning. In fact, it creates its own formulas and algorithms, using processor-integrated basic training coefficients. Of course, Deep Learning has become a quality leap in AI processing. It is performed on a per-pixel basis with the use of noise reduction, restoration of contours, texture creation and restoration of small details. Deep Learning is primarily intended for small details and complex images.

Sony and LG 8K AI Upscaling

Sony's 8K AI Upscaling use dual database processing.

The first database recognizes compression noise, compares it with the patterns and eliminates artifacts. The second database increases resolution based on patterns. All new LG TVs use AI 8K Upscaling with new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. In addition, it supports AI Picture Pro, AI Brightness and AI Sound Pro technologies.

Conclusions

Probably, experts will continue to argue about the reasonable limits of the screen resolution. But in any case, 8K AI upscaling provides improved quality for lower resolution content. In addition, technology has an added bonus. In particular, Deep Learning of neural networks provides continuous quality improvement due to the expansion of patterns base. Theoretically, the quality of 8K AI Upscaling will constantly improve, creating a paradoxical situation when the new TV works worse than the old model. But of course, the price remains one of the main criteria for the consumer when choosing. Today, this difference reaches several thousand dollars. But many consider it inadequately overpriced compared to the quality gain that provides the modern 8K AI Upscaling. Most likely, companies will continue to improve it performance, while lowering prices for 8K models. This video shows the capabilities of the Samsung 8K AI Upscaling Technology. Read the full article

[appliancesreviews]

What TV resolution do we need?

As usual, CES 2020 clearly demonstrated the main focus areas of companies' efforts to improve TVs. In fact, they cover almost all aspects, including design, audio quality, enhanced functionality, etc. But, improving image quality remains one of the main directions. Today, companies solve this problem using a variety of methods, including: - increasing resolution; - increased efficiency of AI 8K Upscaling; - improving color rendering by improving the backlight; - automatic calibration. TCL demonstrated the 8th series with Mini LED backlight Technology and announced its next generation called Vidrian Mini LED.

Today, this Chinese company is confidently leading the way in improving FALD (full-array local dimming) technology. According to many experts, Mini LED backlight or, already, Vidrian Mini LED Technology will be able to closely approximate the color accuracy of LED TVs to OLED technology. In addition, this year, almost all companies introduced new auto-calibration modes. In particular, Sony developed Ambient Optimization, new Samsung TVs use Adaptive Picture, Adaptive Peak Brightness and Adaptive Tone Mapping, LG TVs have Dolby Vision IQ. But an unprecedented number of 8K models from almost all leading companies demonstrates the dominance of the trend of increasing TV resolution.

Introduction

Disputes over the avalanche spread of 4K are still ongoing, but almost all leading companies already offer 8K TVs. Basically, skeptics of this trend use two fairly convincing arguments. First, many have not forgotten the relatively recent history of 3D TVs. The rapid growth of its popularity began after a demonstration at CES and IFA in 2010. Many experts quite logically predicted its excellent prospects. Of course, the manufacturers adequately responded to demand, significantly improving and cheapening this technology in its models. Unfortunately, the high cost of producing 3D content has become an insurmountable obstacle to its further development. As a result, almost all companies abandoned 3D support in their TVs. Therefore, today experts quite reasonably express doubts regarding the prospects for the production of native 8K content, which today is practically absent. Probably for this reason, companies are actively improving 8K AI Upscaling technology. In the worst case, these TVs will be able to improve quality playback of lower resolution content.

Secondly, they quite reasonably ask about the advisability of increasing resolution in terms of eye capabilities. The screen resolution of smartphones confirms these doubts. As known, many modern smartphones already provide 400, 500 ppi and higher. But according to many ophthalmologists, 600 ppi is close to the limit of perception for our vision. Accordingly, a further increase in resolution does not make much sense. In the same time, many companies continue to increase it. Today, the absolute record belongs to the Sony Xperia Z5 with 806 ppi.

This value corresponds to 4K, but does not make sense due to the limited possibilities of our eye. But this argument has some additional aspects.

Vernier acuity (hyperacuity)

As know, standard measure of visual acuity using Snellen eyechart. The argument of skeptics is based on the limitation of visual acuity, which does not distinguish between 8K vs 4K due to too high pixel density. In particular, its lower limit does not exceed 20/20.

But the Japanese NHK broadcaster did some interesting research to find out the effect of resolution on perception. Of course, visual acuity remains a major factor, but it's not the only one. Brain and Vernier acuity (hyperacuity) also significantly affect image perception. For example, many people will be able to distinguish the difference between two pairs of lines, one of which contains perfectly parallel lines, and the second includes lines with an offset of only one pixel. In this case, Vernier acuity will take this displacement as a stepping stone.

But the spacing the pixels on the 8K screen is half vs 4K matrix. Accordingly, reducing the height of the step increases the smoothness of the lines and we perceive the 8K image as clearer. Moreover, the brain reinforces this difference by trying to create a more analog-like image, increasing its realism. Thus, the argument based on only traditional visual acuity is not convincing enough.

Experiment Results

Moreover, recent studies of Dr. YungKyong Park of Ewha Womans University in Seoul further supports this hypothesis. In this experiment, scientists installed side-by-side 65-inch TVs with 4K and 8K resolution, which were calibrated at 500 nits of peak luminance. Of course, all 120 participants in the experiment had the same 20/20 visual acuity and normal color vision. The experiment was conducted at a distance of 9 feet from the displays in a dark room. Content included the 16 images and 3 videos with visuals. As a result, almost all participants in the experiment noted the difference 8K vs 4K displays.

Interestingly, the participants in the experiment noted not increased sharpness or contrast of the image due to higher resolution. According to them, 8K images looked cooler, warmer, more delicious, heavier, etc, which corresponds to sensory perception. In addition, Dr. Kyoung-Min Lee of Seoul National University investigated 8K resolution in terms of its perception by the brain. As known, increasing resolution reduces information loss. Accordingly, the dynamic signal-to-noise ratio increases. As a result, the level of immersive effect also rises. Thus, the brain is less tired when perceiving 8K content. At the same time, watching provides richer emotions due to high realism of the perceived content.

Conclusions

1. In the absence of native 8K content, 8K TVs will be able to improve quality playback of lower resolution content with 8K AI Upscaling technology. 2. Today, the screen of 75" 8K TV provides up to 120 ppi. But according to ophthalmologists, the eye can perceive up to 600 ppi due to Vernier acuity (hyperacuity). In this case, the brain distinguishes images with different detailing even if they are not visualized. Therefore, increasing the TV resolution will be relevant in the coming years. In fact, increasing resolution reduces information loss. Accordingly, the dynamic signal-to-noise ratio increases. As a result, the process requires less brain effort to interpolate the lost data. In other words, the brain is less tired when perceiving content in high resolution. In addition, the level of immersive effect also rises due to high realism of the perceived content. Thus, higher detail reduces the load on the brain, while increasing the immersive effect. 3. Given these factors, increasing ppi of the TV screen by increasing the number of pixels has good prospects and to continue to be one of the main ways to improve image quality. This video demonstrates the capabilities of the 8K AI Upscaling in the Samsung Q950TS 8K at CES 2020. Read the full article