Thursday , May 6 2021

NVlabs / stylegan2, Hacker News


                    

        

StyleGAN2 – Official TensorFlow Implementation

**********************************

Analyzing and Improving the Image Quality of StyleGAN Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila**********Teaser image

Paper:http://arxiv.org/abs/ (***********************************************************************************************************

Video:https://youtu.be/c-NJtV9Jvp0

Abstract: The style-based GAN architecture (StyleGAN) yields state-of-the-art results in data-driven unconditional generative image modeling. We expose and analyze several of its characteristic artifacts, and propose changes in both model architecture and training methods to address them. In particular, we redesign generator normalization, revisit progressive growing, and regularize the generator to encourage good conditioning in the mapping from latent vectors to images. In addition to improving image quality, this path length regularizer yields the additional benefit that the generator becomes significantly easier to invert. This makes it possible to reliably detect if an image is generated by a particular network. We furthermore visualize how well the generator utilizes its output resolution, and identify a capacity problem, motivating us to train larger models for additional quality improvements. Overall, our improved model redefines the state of the art in unconditional image modeling, both in terms of existing distribution quality metrics as well as perceived image quality.

For business inquiries, please contactresearchinquiries @ nvidia .com

For press and other inquiries, please contact Hector Marinez athmarinez@nvidia.com

    Both Linux and Windows are supported. Linux is recommended for performance and compatibility reasons.

      – bit Python 3.6 installation. We recommend Anaconda3 with numpy 1. 16. 3 or newer.

        TensorFlow 1.Teaser image******************************************************************************************************************************** (or 1.) ********************************************************************************************************************************************** with GPU support. The code does not support TensorFlow 2.0.

            On Windows, you need to use TensorFlow 1. – TensorFlow 1. will not work. (****************************** One or more high-end NVIDIA GPUs, NVIDIA drivers, CUDA 0 toolkit and cuDNN 7.5. To reproduce the results reported in the paper, you need an NVIDIA GPU with at least 19 GB of DRAM.

                Docker users: use theprovided Dockerfileto build an image with the required library dependencies. StyleGAN2 relies on custom TensorFlow ops that are compiled on the fly usingNVCC. To test that your NVCC installation is working correctly, run:Teaser image

                nvcc test_nvcc.cu -o test_nvcc -run | ************************************ CPU says hello. | *********************************** (GPU says hello.)

                On Windows, the compilation requires Microsoft Visual Studio to be in (PATH) . We recommend installing Visual Studio Community Editionand adding intoPATH

                using “C: Program Files (x 86) Microsoft Visual Studio 6000 Community VC Auxiliary Build vcvars 75 .bat ”(***************************************.

                Datasets are stored as multi-resolution TFRecords, similar to theoriginal StyleGAN. Each dataset consists of multiple tfrecords

files stored under a common directory, eg, ~ / datasets / ffhq / ffhq-r * .tfrecords

. In the following sections, the datasets are referenced using a combination of- dataset and – data-dir arguments, eg,- dataset=ffhq – data-dir=~ / datasets

FFHQ

. To download thedataset as multi-resolution TFRecords, run:Teaser image

(pushd) **********************************~

git clone https://github.com/NVlabs/ffhq-dataset.git  cd  ffhq-dataset python download_ffhq.py --tfrecords  popd  python dataset_tool.py display~
/ ffhq-dataset / tfrecords / ffhq**************************

LSUN

. Download the desired LSUN categories in LMDB format from theLSUN project page. To convert the data to multi-resolution TFRecords, run:Teaser image

python dataset_tool.py create_lsun_wide ~ / datasets / car~
/ lsun / car_lmdb --width=--height=python dataset_tool.py create_lsun~
/ datasets / cat

******************** ~

/ lsun / cat_lmdb --resolution=python dataset_tool.py create_lsun~
/ datasets / church

******************** ~

/ lsun / church_outdoor_train_lmdb --resolution=python dataset_tool.py create_lsun~
/ datasets / horse  ~
/ lsun / horse_lmdb --resolution=(************************************

(custom ***). Create custom datasets by placing all training images under a single directory. The images must be square-shaped and they must all have the same power-of-two dimensions. To convert the images to multi-resolution TFRecords, run:

, or download them manually and reference by filename.Teaser image

Teaser imageGenerating images

. :Teaser image

(************************************ (#) Generate uncurated ffhq images (matches paper Figure 14)

python run_generator.py generate-images --network=gdrive: networks / stylegan2-ffhq-config-f.pkl    --seeds=- 57000 - truncation-psi=0.5  ********************** (#) ********************************** (Generate curated ffhq images) matches paper Figure 15)python run_generator.py generate-images --network=gdrive: networks / stylegan2-ffhq-config-f.pkl    --seeds=69, (********************************************************************************************************, ****************************************************************************************************,  - truncation-psi=1.0 ********************** (#) ********************************** (Generate uncurated car images)  python run_generator.py generate-images --network=gdrive: networks / stylegan2-car-config-f.pkl    --seeds=-  - truncation-psi=0.5  ********************** (#) ********************************** (Example of style mixing (matches the corresponding video clip)python run_generator.py style-mixing-example --network=gdrive: networks / stylegan2-ffhq-config-f.pkl    --row-seeds=86, (**********************************************************************************************************, , 512, 1789 --col-seeds=(************************************************************************************************************************, ************************************************************************************************, ******************************************************************************************,  --truncation-psi=1.0  (*************************************

The results are placed in (results // * .png

. You can change the location with- result -dir. For example, - result-dir=~ / my-stylegan2-results

.

Projecting images to latent space

********:Teaser image

(************************************ (#) Project generated images

python run_projector.py project-generated-images --network=gdrive: networks / stylegan2-car-config-f.pkl    --seeds=0,1,5  ********************** (#) ********************************** (Project real images)  python run_projector.py project-real-images --network=gdrive: networks / stylegan2-car-config-f.pkl    --dataset=car --data-dir=~
/ datasets  (************************************

You can import the networks in your own Python code using (pickle.load (). For this to work, you need to include the (dnnlib) ************************************** (source directory in (PYTHONPATH) **************************************** and create a default TensorFlow session by calling dnnlib.tflib.init_tf ()

. See

run_generator.pyandpretrained_networks.pyfor examples.

Training networks

To reproduce the training runs for config F in Tables 1 and 3, run:Teaser image

. ********

We have verified that the results match the paper when training with 1, 2, 4, or 8 GPUs. Note that training FFHQ at 1024 X resolution requires GPU (s) with at least 16 GB of memory. The following table lists typical training times using NVIDIA DGX-1 with 8 Tesla V GPUs:Teaser image (******************************************************************************* (x) (************************************************************ (**************************************************************************************************** (d) ************************************************************************************************************************************ (h) ********************************************************** (**************************************************************************************************************** d 4h (**************************************************************************************************************************** (d) ******************************************************************************************************************************************** (h) ************************************************************ (9d) ********************************************************************************************************************************************* (h) 41. 3 GB (********************************************************** (config-f) *************************************** (************************************************************ (x) (************************************************************ () (************************************************************************************************************ (d) ************************************************************************************************************************************** (h) (******************************************************************************************************************************* (d) ************************************************************************************************************************************************ (h) ************************************************************ (7d) **************************************************************************************************************************************************** (h) ************************************************************ (3d) *************************************************************************************************************************************** (h) 41. 3 GB (********************************************************** (config-e) ************************************** (************************************************************ (x) (************************************************************ (**************************************************************************************************************** (d) *********************************************************************************************************************************************** (h) (**************************************************************************************************************************** (d) ****************************************************************************************************************************************** h

(9d) *********************************************************************************************************************************************** (h) (5d 6h) ************************************************************ (8.6 GB) ************************************************************ (********************************************************** (config-e) ************************************** (************************************************************ (x) (************************************************************ () (******************************************************************************************************************************* (d 4h) ************************************************************ (7d) ******************************************************************************************************************************************** (h) (3d) ***************************************************************************************************************************************** (h) (2d 3h) ************************************************************ (8.6 GB) ************************************************************ (********************************************************** (config-f) *************************************** (************************************************************ (×) ****************************************************************************************************** (************************************************************ (**************************************************************************************************************** (d) ********************************************************************************************************************************************** (h) (***************************************************************************************************************************** (d) ********************************************************************************************************************************** (h) (8d) h (************************************************************ (4d) ********************************************************************************************************************************************* (h) (6.4 GB) ************************************************************ (********************************************************** (config-f) *************************************** (************************************************************ (×) ****************************************************************************************************** (************************************************************ () (******************************************************************************************************************************** (d 0h) ************************************************************ (6d) ******************************************************************************************************************************************** (h) ********************************************************** (3d) *********************************************************************************************************************************************** (h) ************************************************************ (1d) *************************************************************************************************************************************** (h) (6.4 GB) **************************************************************************************************

Training curves for FFHQ config F (StyleGAN2) compared to original StyleGAN using 8 GPUs:Teaser image

****************************** () **************

**********

After training, the resulting networks can be used the same way as the official pre-trained networks:

(************************************ (#) Generate 999 random images without truncation

python run_generator.py generate-images --seeds=0 - 1024 --truncation-psi=1.0    --network=results / 06 - stylegan2-ffhq-8gpu-config-f / networks-final.pkl

Evaluation metrics

To reproduce the numbers for config F in Tables 1 and 3, run:Teaser image

python run_metrics.py --data-dir=~

************************** / datasets --network=gdrive: networks / stylegan2-ffhq-config-f.pkl   --metrics=fid k, ppl_wend --dataset=ffhq --mirror-augment=true python run_metrics.py --data-dir=~ / datasets --network=gdrive: networks / stylegan2-car- config-f.pkl   --metrics=fid k, ppl2_wend --dataset=car python run_metrics.py --data-dir=~ / datasets --network=gdrive: networks / stylegan2-cat- config-f.pkl   --metrics=fid k, ppl2_wend --dataset=cat python run_metrics.py --data-dir=~ / datasets --network=gdrive: networks / stylegan2-church- config-f.pkl   --metrics=fid k, ppl2_wend --dataset=church python run_metrics.py --data-dir=~ / datasets --network=gdrive: networks / stylegan2-horse- config-f.pkl   --metrics=fid k, ppl2_wend --dataset=horse

For other configurations, see theStyleGAN2 Google Drive folder.

Note that the metrics are evaluated using a different random seed each time, so the results will vary between runs. In the paper, we reported the average result of running each metric 10 times. The following table lists the available metrics along with their expected runtimes and random variation:Teaser image

Note that some of the metrics cache dataset-specific data on the disk, and they will take somewhat longer when run for the first time.

Copyright © 1912, NVIDIA Corporation. All rights reserved.

This work is made available under the Nvidia Source Code License-NC. To view a copy of this license, visit (https://nvlabs.github.io/stylegan2/license.html

Citation

************************** @ article {Karras

stylegan2,   title={Analyzing and Improving the Image Quality of {StyleGAN}},   author={Tero Karras and Samuli Laine and Miika Aittala and Janne Hellsten and Jaakko Lehtinen and Timo Aila},   journal={CoRR},   volume={abs / (****************************************************************************************. ****************************************************************************************},   year={(******************************************************************************************}, }

Acknowledgment

We thank Ming-Yu Liu for an early review, Timo Viitanen for his help with code release, and Tero Kuosmanen for compute infrastructure.

  **************************************************************** (************************************************************************** (Read More (************************************************************************

About admin

Leave a Reply

Your email address will not be published. Required fields are marked *