#bicubicinterpolation

[appliancesreviews]

Overview of 8K AI Upscaling Technology

Experts have not yet finished arguing about the prospects for 4K TVs, but companies are already actively offering 8K models. Basically, skeptics of this trend use two fairly convincing arguments.

Introduction

First, many have not forgotten the relatively recent history of 3D TVs. The rapid growth of its popularity began after a demonstration at CES and IFA in 2010. Many experts quite logically predicted its excellent prospects. Unfortunately, the high cost of producing 3D content has become an insurmountable obstacle to its further development. As a result, almost all companies abandoned 3D support in their TVs. The second argument is based on the eye capabilities. Unfortunately, its maximum sharpness is significantly inferior, for example, to eagle vision. Of course, it would never occur to anyone to watch a 75-inch TV from a distance of, for example, up to 1 meter. But at a more adequate distance, the sharpness of our vision does not allow us to distinguish individual pixels smaller than a certain size. That is, vision does not perceive the difference between, for example, Full HD and 4K TVs with the same screen size. Perhaps the owners of a falcon or an eagle will be able to please their pets with a high-quality image, but they themselves will not see the difference. But these arguments have counterarguments. For example, companies are actively improving 8K AI upscaling technology.

At a minimum, 8K TVs with AI upscaling will provide better quality playback of lower resolution content. In addition, the results of experiments open up some new aspects. In particular, South Korean scientists have studied the effect of vernier acuity (hyperacuity) to image formation by our brain. Their results allowed scientists to hypothesize that the brain is less tired when perceiving content with high resolution. At the same time, it watching provides richer emotions due to high realism of the perceived content.

Resolution

Of course, any TV provides maximum playback quality for corresponding native content in pixel-to-pixel mode. Unfortunately, the real situation on the market is often different from the ideal. For example, the capabilities of modern TVs often significantly exceed the resolution of the content offered. But sometimes devices, on the contrary, do not support content resolution. Of course, companies solve this problem. Modern algorithms use several basic technologies. The simplest of them increases the image resolution, replacing each pixel with four copies of it. Bilinear interpolation is more efficient. These algorithms compare the missing pixel with the two nearest ones, forming a linear gradient to sharpen the image.

Even more efficient bicubic interpolation uses 16 closest points in all directions, increasing color accuracy. Unfortunately, it often provokes a halo effect.

Main methods

A list of the most popular modern methods includes supersampling, checkerboard rendering and upscaling. Supersampling is a spatial anti-aliasing method for removing the aliasing in the rendered images. In fact, it first renders the image in 4K, and then displays it in 1080p resolution. As a result, the detail of the final image increases. Of course, rendering requires sufficient processing power. The Insects screenshot from Microsoft on Full HD TV demonstrates this effect.

Checkerboard Rendering reduces computing power requirements. In particular, it removes every second pixel and forms an image from the rest. The diagrams show the differences between Standard Rendering and Checkerboard Rendering.

Removing pixels slightly reduces the image resolution, but significantly reduces the amount of computation. In fact, upscaling only resizes while maintaining its proportions. It can be considered the simplest method of converting image resolution. In this case, the TV processor forms additional pixels, providing the required resolution. Accordingly, the final image is a set of new averaged pixels.

Unfortunately, the final image comparison demonstrates a significant decrease in quality compared to native content.

Upscaling

A few years ago, Samsung was the first to use the marketing term UHD Upscaling. Strictly speaking, almost any smartphone or computer has long supported this feature. Of course, its quality directly depends on the content. For example, the quality of SD - UHD Upscaling is significantly lower compared to Full HD - UHD Upscaling due to the insufficient number of pixels in the original image. The picture illustrates the difference in quality between SD / HD / Full HD - 4K Upscaling.

But its visualization is almost disappearing for Full HD. The first 4K TVs did not have an AI processor and simply scaled the image to the entire screen area using traditional interpolation and increasing the pixels size. Unfortunately, bilinear interpolation provides a fairly low quality due to loss of information, reduced detail, blurring outlines of objects, noise, etc. Moreover, mutual influence complicates their elimination. For example, sharpening increases noise, and conversely, reducing noise reduces detail. But companies have improved technology by developing AI Upscaling. Using a powerful high-performance processor radically improved upscaling quality. Today, almost all leaders use 8K AI Upscaling technology under different names. But, of course, they all use the same principles. Samsung is one of the leaders in the development of this technology. The company uses the powerful 8K Quantum processor, optimized for neural networks and AI processing. Samsung's MLSR (Machine Learning Super Resolution) technology uses machine learning, object recognition algorithms, and software filters.

Samsung AI Upscaling

Simplified, this technology uses a constantly updated database of the images pairs in high and low resolution. It's formed as follows. The system selects an 8K image and lowers its resolution, adding noise and reducing the clarity of outlines. In addition, it performs several compression/decompression cycles with different quality levels to add characteristic artifacts. Generated images pairs of different quality are divided into classes, processed by algorithms based on neural networks and stored in a database. Subsequently, the processor analyzes the low-resolution image, finds the corresponding algorithm in the database with filters, and replaces it with an 8K pattern. In fact, the 8K AI Upscaling technology generates an image by analogy with the creation of Frankenstein from 8K patterns in the database. During processing, AI analyzes image fragments up to 60 times per second, correcting saturation, detail and white balance. CPU uses code lines of patterns base for localization of processing fragments. Such an algorithm increases the recognition accuracy of displayed objects. As a result, the processor almost accurately distinguishes, for example, the sky from its reflection in the windows. Moreover, the technology works quite well even with streaming video. But cloud computing for such content is not possible. Previously, AI machine learning used a formula bank, which is downloaded to the TV and updated periodically. But new Samsung Q950TS supports innovative Deep Learning using a neural network to expand machine learning. In fact, it creates its own formulas and algorithms, using processor-integrated basic training coefficients. Of course, Deep Learning has become a quality leap in AI processing. It is performed on a per-pixel basis with the use of noise reduction, restoration of contours, texture creation and restoration of small details. Deep Learning is primarily intended for small details and complex images.

Sony and LG 8K AI Upscaling

Sony's 8K AI Upscaling use dual database processing.

The first database recognizes compression noise, compares it with the patterns and eliminates artifacts. The second database increases resolution based on patterns. All new LG TVs use AI 8K Upscaling with new powerful α (Alpha) 9 Gen 3 AI processor with AI Deep Learning algorithms. In addition, it supports AI Picture Pro, AI Brightness and AI Sound Pro technologies.

Conclusions

Probably, experts will continue to argue about the reasonable limits of the screen resolution. But in any case, 8K AI upscaling provides improved quality for lower resolution content. In addition, technology has an added bonus. In particular, Deep Learning of neural networks provides continuous quality improvement due to the expansion of patterns base. Theoretically, the quality of 8K AI Upscaling will constantly improve, creating a paradoxical situation when the new TV works worse than the old model. But of course, the price remains one of the main criteria for the consumer when choosing. Today, this difference reaches several thousand dollars. But many consider it inadequately overpriced compared to the quality gain that provides the modern 8K AI Upscaling. Most likely, companies will continue to improve it performance, while lowering prices for 8K models. This video shows the capabilities of the Samsung 8K AI Upscaling Technology. Read the full article

[appliancesreviews]

New technologies in Samsung TVs 2020 Review

As known, these days companies present their new products at CES 2020. As usual, the attention of experts is primarily attracted by the leaders of the consumer electronics segment, which shape development trends. Of course, Samsung is one of them. The company fully met expectations by presenting three new TVs, including the second generation of Samsung 8K model, innovative MicroLED panels and the experimental The Sero TV. It has a rotary screen for viewing video content in vertical format, for example, from Instagram.

Of course, Samsung Q950TS aroused the greatest interest among experts. In fact, the company created the second generation of its 8K TV.

The company demonstrated that the past year was not in vain, embodying a huge number of innovative solutions in the Q950TS. Moreover, this model demonstrated the company's successes in fundamental directions, including 8K AI upscaling based on Deep Learning with neural network and AI ScaleNet technology.

Upscaling

As known, upscaling increases the number of pixels in the original signal to increase the image clarity on the TV screen. This problem arose about 10 years ago when TVs already provided Full HD (1920 x 1080) resolution, and television channels broadcast the signal in HD format (1280 x 720), or SD format (480p, 576i). Of course, this factor significantly reduced the advisability of buying an expensive Full HD TV. At the first stage, companies successfully solved this problem, showing in advertising or when selling a magnificent image of high-quality video with high resolution. But having bought such a TV, the consumer was quickly disappointed due to the low image quality of low-resolution video from normal traffic. Very quickly, the consumer market realized that even a very expensive TV can provide high quality only when playing high-quality content. Of course, companies could wait for the required content to appear on the market, but there was a risk. For example, many people remember the rise and sunset of the 3D format in TVs. Even with a large number of fans, today almost all companies have abandoned this option due to the shortage of 3D video content. As a result, large companies, including Samsung, began to actively develop upscaling technologies for their models, reducing their commercial risks. Indeed, even in the absence of appropriate native content, upscaling provides at least an increase in the quality of available low-resolution content, stimulating the acquisition of expensive model. As a last resort, 8K TV with high-quality upscaling will improve the playback quality of accessible content.

8K AI upscaling with AI Quantum Processor 8K

Physically, upscaling is similar to frame interpolation. As known, in this mode TV creates an additional intermediate frame based on two neighboring frames, ensuring smooth playback of even highly dynamic scenes. Accordingly, upscaling forms additional pixels to increase resolution. For example, the AI 8K Quantum easily scales content to 33,177,600 pixels. Unfortunately, upscaling can distort the original image, forming artifacts. Therefore, this task requires solving many complex problems, including color accuracy, GPU performance for video processing without lags, etc. As a result, companies developed a lot of upscaling technologies, including Pixel Shift (formation of an additional pixel by shifting), bicubic interpolation (calculation colors by adjacent pixels), etc. Today, AI upscaling is the most promising direction. The Q950TS confirms Samsung’s leadership in the AI upscaling.

This model uses the latest generation of the AI Quantum Processor 8K, which extends AI upscaling capabilities. Previously, AI machine learning used a formula bank, which is downloaded to the TV and updated periodically. New Q950TS supports innovative Deep Learning using a neural network to expand machine learning. In fact, it creates its own formulas and algorithms without using a formula bank. Of course, Deep Learning has become a quality leap in AI processing. It is performed on a per-pixel basis with the use of noise reduction, restoration of contours, texture creation and restoration of small details. Deep Learning is primarily intended for small details and complex images.

AI ScaleNet

Of course, transmitting 8K streaming video content requires a significant increase in the bandwidth of communication channels, which depends on the speed of your Internet connection. To solve this problem, Samsung Research Visual Technology has developed a new AI ScaleNet video codec. As its name implies, the company uses AI to decompress 8K video. In fact, 8K is compressed to 4K using AI-downscaler and transmitted to the user's TV. Then, AI ScaleNet (codec with machine learning elements using different patterns for different image parts) restores 4K to 8K.

Kwang Pyo Choi, developer of this codec, said that AI ScaleNet uses AI Adaptive Bit Rate Streaming, which provides adjustment depending on the bandwidth of the networks. In fact, the codec adjusts the resolution for smooth video streaming. In prospect, this technology will provide the required quality of 8K content transfer even at low bandwidth, for example, during periods of heavy network traffic. Moreover, Samsung has agreed with Amazon Prime Video to jointly test the new technology. Additionally, the Q950TS supports Google's AV1 (AOMedia Video 1) codec for 8K YouTube content.

Object Tracking Sound Plus (OTS+), Active Voice Amplifier (AVA) and Q Symphony

Apparently, Samsung paid a lot of attention to the object-oriented audio, which lies at the heart of Dolby Atmos and DTS: X. As a result, the new TV has OTS+ (Object Tracking Sound Plus) technology that recognizes moving object inside the scene. After identifying the object, OTS+ moves the corresponding sound track in the direction of its movement using internal speakers. As a result, the sound follows the object. This effect works well, for example, in the scene of a passing car or flying helicopter. TV sequentially switches the speakers, creating a wonderful illusion of the engine sound that moves with the car or helicopter. Maybe Object Tracking Sound Plus is not yet ready to compete with Dolby Atmos, but, of course, it significantly surpasses the traditional 10W down-firing speakers in other flagships. In addition, the company offers AVA (Active Voice Amplifier) feature that adjusts the volume of dialogs depending on the noise level in the room. For example, when using a blender or vacuum, AVA will automatically increase the volume. But in case of a conversation in the room, the AVA activation requires pressing a passive-aggressive button on the remote control. Most likely, Q-Symphony technology will appeal to home theater enthusiasts. It combines the speakers in the Samsung TVs of 2020 and Samsung Q800T soundbars into a single audio system, creating a stunning dynamic soundscape. In fact, Q-Symphony creates a 9.1.4 configuration.

Conclusion

Samsung has once again confirmed its innovative leadership. New Samsung Q950TS 8K TV, innovative MicroLED panels and the experimental The Sero TV demonstrated that the company's engineers were able to make significant progress in a year. The Q950TS 8K TV has embodied almost all of the company's innovative solutions, including 8K AI upscaling with AI Quantum Processor 8K, AI ScaleNet, Object Tracking Sound Plus (OTS+), Active Voice Amplifier (AVA) and Q Symphony. In fact, Q950TS became the best advertisement of the innovative successes of a South Korean company. Apparently, Samsung Q950TS 8K will compete with Nano99, Nano97 and Nano95 series of LG NanoCell 8K TVs and Sony Z8H 8K. The video offers a demo of the Q950TS 8K and Sero Vertical TV at CES 2020. Read the full article