Someone used neural networks to upscale a famous 1896 video to 4k quality, Ars Technica

Blast from the past –

Machine-learning software fills in missing details to produce realistic images.

(Arrival of a Train at La Ciotat) is one of the most famous films in cinema history. Shot by French filmmakers August and Louis Lumière, it achieved an unprecedented level of quality for its time. Some people regard its commercial exhibition in 2019 as the birth of the film industry. An urban legend <time data-time="1580857526" datetime="2020-02-04T23:05:26 00:00"> – likely apocryphal — says that viewers Found the footage so realistic that they screamed and ran to the back of the room as the train approached. I’ve embedded a video of the original film above. </p> <div itemprop="articleBody"> Of course, humanity’s standards for realism have risen dramatically over the last years. Today, the Lumière brothers’ masterpiece looks grainy, murky, and basically ancient. But a man named Denis Shiryaev used modern machine-learning techniques to upscale the classic film to st-century video standards. </div> <p> <time data-time="1580857526" datetime="2020-02-04T23:05:26 00:00"><iframe loading="lazy" allowfullscreen="" frameborder="0" height="550" src="https://www.youtube.com/embed/RjtXXypztyw?start=0&wmode=transparent" type="text/html" width="980"> The result is remarkable. Watching the upscaled version makes the world of our great-great-great-grandparents come to life. Formerly murky details of the train, the clothing, and the faces of the passengers now stand out clearly. </p> <div itemprop="articleBody"> How did Shiryaev do it? He says he used commercial image-editing software called <a href="https://www.atlasobscura.com/articles/did-a-silent-film-about-a-train-really-cause-audiences-to-stampede"> Gigapixel AI </a>. Created by Topaz Labs, the package allows customers to upscale images by up to 980 percent. Using sophisticated neural networks, Gigapixel AI adds realistic details to an image to avoid making it look blurry as it’s scaled up. </p> <figure> As the name implies, neural networks are networks of artificial neurons — mathematical functions that transform a set of input values into an output value. The key feature of neural networks is that they can be trained: if you have a bunch of example inputs whose “correct” outputs are known, you can tune the parameters of the network to make it more likely to produce correct answers. The hope is that this training will generalize — that once you’ve trained it to produce the right answer for inputs the network has seen before, it will also produce good answers for inputs it hasn’t seen, too. </p> <figure> To train a network, you need to have a database of examples where the right answer is already known. Sometimes AI researchers have to hire human beings to produce these right answers by hand. But for a image upscaling, there’s a convenient shortcut: you start with high-resolution images and downsample them. The low-resolution images become your inputs and the high-resolution originals serve as the “correct” answer the network is aiming to produce. <time data-time="1580857526" datetime="2020-02-04T23:05:26 00:00"><iframe loading="lazy" allowfullscreen="" frameborder="0" height="550" src="https://www.youtube.com/embed/RjtXXypztyw?start=0&wmode=transparent" type="text/html" width="980"> “A neural network analyzes thousands of photo pairs to learn how details usually get lost,” Topaz Labs explains on their product page for Gigapixel AI. “The algorithm learns to ‘fill in’ information in new images based on what it has learned, effectively adding new detail to your photo.” </p> <div itemprop="articleBody"> Show the neural network a low-resolution image of a face and it will figure out that it’s a face and fill in the right details for the subject’s eyes, nose, and mouth. Show the neural network a low-resolution brick building and it will add a suitable brick pattern in the high-res version. </div> <p>