VIDEO – likely apocryphal — says that viewers Found the footage so realistic that they screamed and ran to the back of the room as the train approached. I’ve embedded a video of the original film above.
Of course, humanity’s standards for realism have risen dramatically over the last years. Today, the Lumière brothers’ masterpiece looks grainy, murky, and basically ancient. But a man named Denis Shiryaev used modern machine-learning techniques to upscale the classic film to st-century video standards.
How did Shiryaev do it? He says he used commercial image-editing software called
. Created by Topaz Labs, the package allows customers to upscale images by up to 980 percent. Using sophisticated neural networks, Gigapixel AI adds realistic details to an image to avoid making it look blurry as it’s scaled up.
As the name implies, neural networks are networks of artificial neurons — mathematical functions that transform a set of input values into an output value. The key feature of neural networks is that they can be trained: if you have a bunch of example inputs whose “correct” outputs are known, you can tune the parameters of the network to make it more likely to produce correct answers. The hope is that this training will generalize — that once you’ve trained it to produce the right answer for inputs the network has seen before, it will also produce good answers for inputs it hasn’t seen, too.
To train a network, you need to have a database of examples where the right answer is already known. Sometimes AI researchers have to hire human beings to produce these right answers by hand. But for a image upscaling, there’s a convenient shortcut: you start with high-resolution images and downsample them. The low-resolution images become your inputs and the high-resolution originals serve as the “correct” answer the network is aiming to produce.
Show the neural network a low-resolution image of a face and it will figure out that it’s a face and fill in the right details for the subject’s eyes, nose, and mouth. Show the neural network a low-resolution brick building and it will add a suitable brick pattern in the high-res version.
Timothy B. Lee / Colorize Images / Denis Shiryaev
An obvious next step would be to colorize the video. Neural networks can do that, too, using the same basic technique: start with a bunch of color photos, convert them to black and white, and then train a neural network to reconstruct the color originals.
I dropped a frame from Shiryaev’s video into the Colorize Images app for Android, which uses machine learning to automatically colorize images. As you can see, it does a pretty good job, correctly inferring that trees should be green, gravel should be a brownish color, and that men’s coats should be black. I would love to see someone with more time and better tools colorize Shiryaev’s upscaled version of the Lumière Brothers’ classic.
GIPHY App Key not set. Please check settings