Table of Contents
StyleGAN Notes
styleGAN2-ADA experiments Feb 2021 (and so on and so on…)
- using
foam-repo/F20/web/src/flickr/img- two data folders (prepared with dataset_tool.py)
1024-foam-f20all the images tagged with f15 machine tags1024-zzkt-f20all the images tagged with f15 machine tags and zzkt images
- transfer learning from flickr-1024 network (i.e
ffhq.pkl) via–resume=ffhq1024 - training strategy
- start with
–cfg=auto –aug=ada –metrics=fid50k_full - compare unmirrored with →
–mirror=1 - try
–augpipe=bgcfnc - try various gamma levels
–gamma=10
preparing data
transfer learning using ffhq model for sytleGAN2-ADA https://github.com/NVlabs/stylegan2-ada-pytorch
resize/scale/crop to 512 or 1024 1:1
mogrify -resize 512x512^ -gravity center -extent 512x512 *.jpg
mogrify original.png -resize 1024x1024^ -gravity center -extent 1024x1024 new.png
prepare images if reqd. → png in rgb colorspace
mogrify -resize 512x512^ -gravity center -extent 512x512 -type TrueColor -colorspace sRGB *.jpg
convert -colorspace sRGB -type truecolor *.png
and maybe
mogrify -define png:color-type=2 data/1024x1024-foam-flickr/*.png
python3 dataset_tool.py --source=data/1024x1024 --dest=data/foam-flickr/
python3 train.py --outdir=results/flickr4/ \
--data=data/512x512/ \
--gpus=1 \
--resume=ffhq512 --snap=10
generating
generate images
python3 generate.py --outdir=results/ \
--trunc=1 \
--network=results/flickr3/00009--auto1-resumeffhq1024/network-snapshot-000000.pkl \
--seeds=85,265,297,849
generate.py --outdir=results/flickr5/generated/000041 \ --network=results/flickr5/00002-512x512-auto1-resumecustom/network-snapshot-000040.pkl \ --seeds=1100-1200
style mixing
python3 style_mixing.py --outdir=results/ \
--network=results/flickr3/00009--auto1-resumeffhq1024/network-snapshot-000000.pkl \
--rows=85,100,75,458,1500 \
--cols=55,821,1789,293
style_mixing.py --outdir=results/flickr5/generated/000000 --network=results/flickr5/00001-512x512-auto1-resumecustom/network-snapshot-000000.pkl --rows=600-610 --cols=108-118
latent images
python3 projector.py --outdir=results/flickr6/generated/100000 \
--target=data/_transfer_test/41018347952_71cbe8d49e_k\(1\).jpg \
--network=results/flickr6/00003-512x512-auto1-resumecustom/network-snapshot-000200.pkl
training
python3 train.py --outdir=results/flickr3/ \
--data=data/foam-flickr/ \
--gpus=1 \
--resume=ffhq1024 \
--snap=10
python3 train.py --outdir=results/flickr3/ --data=data/foam-flickr/ --gpus=1 --resume=ffhq1024 --snap=10
Training options:
{
"num_gpus": 1,
"image_snapshot_ticks": 10,
"network_snapshot_ticks": 10,
"metrics": [
"fid50k_full"
],
"random_seed": 0,
"training_set_kwargs": {
"class_name": "training.dataset.ImageFolderDataset",
"path": "data/foam-flickr/",
"use_labels": false,
"max_size": 1140,
"xflip": false,
"resolution": 1024
},
"data_loader_kwargs": {
"pin_memory": true,
"num_workers": 3,
"prefetch_factor": 2
},
"G_kwargs": {
"class_name": "training.networks.Generator",
"z_dim": 512,
"w_dim": 512,
"mapping_kwargs": {
"num_layers": 2
},
"synthesis_kwargs": {
"channel_base": 32768,
"channel_max": 512,
"num_fp16_res": 4,
"conv_clamp": 256
}
},
"D_kwargs": {
"class_name": "training.networks.Discriminator",
"block_kwargs": {},
"mapping_kwargs": {},
"epilogue_kwargs": {
"mbstd_group_size": 4
},
"channel_base": 32768,
"channel_max": 512,
"num_fp16_res": 4,
"conv_clamp": 256
},
"G_opt_kwargs": {
"class_name": "torch.optim.Adam",
"lr": 0.002,
"betas": [
0,
0.99
],
"eps": 1e-08
},
"D_opt_kwargs": {
"class_name": "torch.optim.Adam",
"lr": 0.002,
"betas": [
0,
0.99
],
"eps": 1e-08
},
"loss_kwargs": {
"class_name": "training.loss.StyleGAN2Loss",
"r1_gamma": 52.4288
},
"total_kimg": 25000,
"batch_size": 4,
"batch_gpu": 4,
"ema_kimg": 1.25,
"ema_rampup": null,
"ada_target": 0.6,
"augment_kwargs": {
"class_name": "training.augment.AugmentPipe",
"xflip": 1,
"rotate90": 1,
"xint": 1,
"scale": 1,
"rotate": 1,
"aniso": 1,
"xfrac": 1,
"brightness": 1,
"contrast": 1,
"lumaflip": 1,
"hue": 1,
"saturation": 1
},
"resume_pkl": "https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/transfer-learning-source-nets/ffhq-res1024-mirror-stylegan2-noaug.pkl",
"ada_kimg": 100,
"run_dir": "results/flickr3/00009--auto1-resumeffhq1024"
}
Output directory: results/flickr3/00009--auto1-resumeffhq1024
Training data: data/foam-flickr/
Training duration: 25000 kimg
Number of GPUs: 1
Number of images: 1140
Image resolution: 1024
Conditional model: False
Dataset x-flips: False
Creating output directory...
Launching processes...
Loading training set...
Num images: 1140
Image shape: [3, 1024, 1024]
Label shape: [0]
Constructing networks...
Resuming from "https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/transfer-learning-source-nets/ffhq-res1024-mirror-stylegan2-noaug.pkl"
Setting up PyTorch plugin "bias_act_plugin"... Done.
Setting up PyTorch plugin "upfirdn2d_plugin"... Done.
Generator Parameters Buffers Output shape Datatype
--- --- --- --- ---
mapping.fc0 262656 - [4, 512] float32
mapping.fc1 262656 - [4, 512] float32
mapping - 512 [4, 18, 512] float32
synthesis.b4.conv1 2622465 32 [4, 512, 4, 4] float32
synthesis.b4.torgb 264195 - [4, 3, 4, 4] float32
synthesis.b4:0 8192 16 [4, 512, 4, 4] float32
synthesis.b4:1 - - [4, 512, 4, 4] float32
synthesis.b8.conv0 2622465 80 [4, 512, 8, 8] float32
synthesis.b8.conv1 2622465 80 [4, 512, 8, 8] float32
synthesis.b8.torgb 264195 - [4, 3, 8, 8] float32
synthesis.b8:0 - 16 [4, 512, 8, 8] float32
synthesis.b8:1 - - [4, 512, 8, 8] float32
synthesis.b16.conv0 2622465 272 [4, 512, 16, 16] float32
synthesis.b16.conv1 2622465 272 [4, 512, 16, 16] float32
synthesis.b16.torgb 264195 - [4, 3, 16, 16] float32
synthesis.b16:0 - 16 [4, 512, 16, 16] float32
synthesis.b16:1 - - [4, 512, 16, 16] float32
synthesis.b32.conv0 2622465 1040 [4, 512, 32, 32] float32
synthesis.b32.conv1 2622465 1040 [4, 512, 32, 32] float32
synthesis.b32.torgb 264195 - [4, 3, 32, 32] float32
synthesis.b32:0 - 16 [4, 512, 32, 32] float32
synthesis.b32:1 - - [4, 512, 32, 32] float32
synthesis.b64.conv0 2622465 4112 [4, 512, 64, 64] float32
synthesis.b64.conv1 2622465 4112 [4, 512, 64, 64] float32
synthesis.b64.torgb 264195 - [4, 3, 64, 64] float32
synthesis.b64:0 - 16 [4, 512, 64, 64] float32
synthesis.b64:1 - - [4, 512, 64, 64] float32
synthesis.b128.conv0 1442561 16400 [4, 256, 128, 128] float16
synthesis.b128.conv1 721409 16400 [4, 256, 128, 128] float16
synthesis.b128.torgb 132099 - [4, 3, 128, 128] float16
synthesis.b128:0 - 16 [4, 256, 128, 128] float16
synthesis.b128:1 - - [4, 256, 128, 128] float32
synthesis.b256.conv0 426369 65552 [4, 128, 256, 256] float16
synthesis.b256.conv1 213249 65552 [4, 128, 256, 256] float16
synthesis.b256.torgb 66051 - [4, 3, 256, 256] float16
synthesis.b256:0 - 16 [4, 128, 256, 256] float16
synthesis.b256:1 - - [4, 128, 256, 256] float32
synthesis.b512.conv0 139457 262160 [4, 64, 512, 512] float16
synthesis.b512.conv1 69761 262160 [4, 64, 512, 512] float16
synthesis.b512.torgb 33027 - [4, 3, 512, 512] float16
synthesis.b512:0 - 16 [4, 64, 512, 512] float16
synthesis.b512:1 - - [4, 64, 512, 512] float32
synthesis.b1024.conv0 51297 1048592 [4, 32, 1024, 1024] float16
synthesis.b1024.conv1 25665 1048592 [4, 32, 1024, 1024] float16
synthesis.b1024.torgb 16515 - [4, 3, 1024, 1024] float16
synthesis.b1024:0 - 16 [4, 32, 1024, 1024] float16
synthesis.b1024:1 - - [4, 32, 1024, 1024] float32
--- --- --- --- ---
Total 28794124 2797104 - -
Discriminator Parameters Buffers Output shape Datatype
--- --- --- --- ---
b1024.fromrgb 128 16 [4, 32, 1024, 1024] float16
b1024.skip 2048 16 [4, 64, 512, 512] float16
b1024.conv0 9248 16 [4, 32, 1024, 1024] float16
b1024.conv1 18496 16 [4, 64, 512, 512] float16
b1024 - 16 [4, 64, 512, 512] float16
b512.skip 8192 16 [4, 128, 256, 256] float16
b512.conv0 36928 16 [4, 64, 512, 512] float16
b512.conv1 73856 16 [4, 128, 256, 256] float16
b512 - 16 [4, 128, 256, 256] float16
b256.skip 32768 16 [4, 256, 128, 128] float16
b256.conv0 147584 16 [4, 128, 256, 256] float16
b256.conv1 295168 16 [4, 256, 128, 128] float16
b256 - 16 [4, 256, 128, 128] float16
b128.skip 131072 16 [4, 512, 64, 64] float16
b128.conv0 590080 16 [4, 256, 128, 128] float16
b128.conv1 1180160 16 [4, 512, 64, 64] float16
b128 - 16 [4, 512, 64, 64] float16
b64.skip 262144 16 [4, 512, 32, 32] float32
b64.conv0 2359808 16 [4, 512, 64, 64] float32
b64.conv1 2359808 16 [4, 512, 32, 32] float32
b64 - 16 [4, 512, 32, 32] float32
b32.skip 262144 16 [4, 512, 16, 16] float32
b32.conv0 2359808 16 [4, 512, 32, 32] float32
b32.conv1 2359808 16 [4, 512, 16, 16] float32
b32 - 16 [4, 512, 16, 16] float32
tick 0 kimg 0.0 time 47s sec/tick 6.8 sec/kimg 1709.21 maintenance 39.8 cpumem 3.69 gpumem 17.00 augment 0.000
b16 - 16 [4, 512, 8, 8] float32
b8.skip 262144 16 [4, 512, 4, 4] float32
b8.conv0 2359808 16 [4, 512, 8, 8] float32
b8.conv1 2359808 16 [4, 512, 4, 4] float32
b8 - 16 [4, 512, 4, 4] float32
b4.mbstd - - [4, 513, 4, 4] float32
b4.conv 2364416 16 [4, 512, 4, 4] float32
b4.fc 4194816 - [4, 512] float32
b4.out 513 - [4, 1] float32
--- --- --- --- ---
Total 29012513 544 - -
Setting up augmentation...
Distributing across 1 GPUs...
Setting up training phases...
Exporting sample images...
Initializing logs...
Training for 25000 kimg...
tick 0 kimg 0.0 time 47s sec/tick 6.9 sec/kimg 1733.61 maintenance 39.6 cpumem 3.69 gpumem 17.00 augment 0.000
Evaluating metrics...
b16 - 16 [4, 512, 8, 8] float32
b8.skip 262144 16 [4, 512, 4, 4] float32
b8.conv0 2359808 16 [4, 512, 8, 8] float32
b8.conv1 2359808 16 [4, 512, 4, 4] float32
b8 - 16 [4, 512, 4, 4] float32
b4.mbstd - - [4, 513, 4, 4] float32
b4.conv 2364416 16 [4, 512, 4, 4] float32
b4.fc 4194816 - [4, 512] float32
b4.out 513 - [4, 1] float32
--- --- --- --- ---
Total 29012513 544 - -
Setting up augmentation...
Distributing across 1 GPUs...
Setting up training phases...
Exporting sample images...
Initializing logs...
Training for 25000 kimg...
tick 0 kimg 0.0 time 47s sec/tick 6.9 sec/kimg 1733.61 maintenance 39.6 cpumem 3.69 gpumem 17.00 augment 0.000
Evaluating metrics...
b16.conv1 2359808 16 [4, 512, 8, 8] float32
b16 - 16 [4, 512, 8, 8] float32
b8.skip 262144 16 [4, 512, 4, 4] float32
b8.conv0 2359808 16 [4, 512, 8, 8] float32
b8.conv1 2359808 16 [4, 512, 4, 4] float32
b8 - 16 [4, 512, 4, 4] float32
b4.mbstd - - [4, 513, 4, 4] float32
b4.conv 2364416 16 [4, 512, 4, 4] float32
b4.fc 4194816 - [4, 512] float32
b4.out 513 - [4, 1] float32
--- --- --- --- ---
Total 29012513 544 - -
Setting up augmentation...
Distributing across 1 GPUs...
Setting up training phases...
Exporting sample images...
Initializing logs...
Training for 25000 kimg...
tick 0 kimg 0.0 time 47s sec/tick 6.9 sec/kimg 1733.61 maintenance 39.6 cpumem 3.69 gpumem 17.00 augment 0.000
Evaluating metrics...
2021-02-03 01:32:07
{"results": {"fid50k_full": 276.11341135899374}, "metric": "fid50k_full", "total_time": 890.9179244041443, "total_time_str": "14m 51s", "num_gpus": 1, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1612312310.2584891}
Evaluating metrics…
{"results": {"fid50k_full": 513.5465590711145}, "metric": "fid50k_full", "total_time": 893.2358613014221, "total_time_str": "14m 53s", "num_gpus": 1, "snapshot_pkl": "network-snapshot-000040.pkl", "timestamp": 1612320983.0430386}
python3 train.py --outdir=results/flickr3/ --data=data/foam-flickr/ --gpus=1 --resume=ffhq1024 --snap=10 tick 11 kimg 44.0 time 2h 53m 35s sec/tick 784.5 sec/kimg 196.13 maintenance 897.6 cpumem 3.93 gpumem 7.96 augment 0.209 tick 12 kimg 48.0 time 3h 06m 39s sec/tick 784.0 sec/kimg 196.00 maintenance 0.1 cpumem 3.93 gpumem 7.95 augment 0.196 tick 13 kimg 52.0 time 3h 19m 41s sec/tick 782.5 sec/kimg 195.63 maintenance 0.1 cpumem 3.94 gpumem 7.91 augment 0.216 tick 14 kimg 56.0 time 3h 32m 45s sec/tick 783.5 sec/kimg 195.88 maintenance 0.1 cpumem 3.94 gpumem 7.98 augment 0.255 tick 15 kimg 60.0 time 3h 45m 50s sec/tick 785.4 sec/kimg 196.34 maintenance 0.1 cpumem 3.94 gpumem 7.92 augment 0.288 tick 16 kimg 64.0 time 3h 58m 58s sec/tick 787.5 sec/kimg 196.88 maintenance 0.1 cpumem 3.94 gpumem 8.05 augment 0.299 tick 17 kimg 68.0 time 4h 12m 04s sec/tick 785.6 sec/kimg 196.39 maintenance 0.1 cpumem 3.95 gpumem 8.05 augment 0.336 tick 18 kimg 72.0 time 4h 25m 09s sec/tick 785.4 sec/kimg 196.35 maintenance 0.1 cpumem 3.96 gpumem 8.00 augment 0.376 tick 19 kimg 76.0 time 4h 38m 15s sec/tick 786.2 sec/kimg 196.55 maintenance 0.1 cpumem 3.96 gpumem 8.14 augment 0.416 tick 20 kimg 80.0 time 4h 51m 23s sec/tick 787.4 sec/kimg 196.85 maintenance 0.1 cpumem 3.97 gpumem 8.03 augment 0.455 Evaluating metrics... {"results": {"fid50k_full": 574.2049302708526}, "metric": "fid50k_full", "total_time": 891.4142208099365, "total_time_str": "14m 51s", "num_gpus": 1, "snapshot_pkl": "network-snapshot-000080.pkl", "timestamp": 1612329731.1134176} tick 21 kimg 84.0 time 5h 19m 26s sec/tick 787.9 sec/kimg 196.97 maintenance 895.2 cpumem 3.97 gpumem 8.05 augment 0.495 tick 22 kimg 88.0 time 5h 32m 35s sec/tick 789.4 sec/kimg 197.35 maintenance 0.1 cpumem 3.97 gpumem 8.16 augment 0.535 tick 23 kimg 92.0 time 5h 45m 46s sec/tick 790.2 sec/kimg 197.56 maintenance 0.1 cpumem 3.97 gpumem 8.10 augment 0.575 tick 24 kimg 96.0 time 5h 58m 58s sec/tick 792.1 sec/kimg 198.02 maintenance 0.1 cpumem 3.98 gpumem 8.18 augment 0.614 tick 25 kimg 100.0 time 6h 12m 11s sec/tick 793.0 sec/kimg 198.26 maintenance 0.1 cpumem 3.98 gpumem 8.30 augment 0.621 tick 26 kimg 104.0 time 6h 25m 26s sec/tick 795.3 sec/kimg 198.82 maintenance 0.1 cpumem 3.98 gpumem 8.28 augment 0.621 tick 27 kimg 108.0 time 6h 38m 41s sec/tick 793.9 sec/kimg 198.48 maintenance 0.1 cpumem 3.98 gpumem 8.12 augment 0.608 tick 28 kimg 112.0 time 6h 51m 55s sec/tick 794.3 sec/kimg 198.56 maintenance 0.1 cpumem 3.99 gpumem 8.05 augment 0.595 tick 29 kimg 116.0 time 7h 05m 08s sec/tick 793.2 sec/kimg 198.30 maintenance 0.1 cpumem 3.99 gpumem 8.11 augment 0.576 tick 30 kimg 120.0 time 7h 18m 22s sec/tick 793.6 sec/kimg 198.41 maintenance 0.1 cpumem 3.99 gpumem 8.28 augment 0.558 Evaluating metrics... {"results": {"fid50k_full": 363.36702330928847}, "metric": "fid50k_full", "total_time": 892.5261161327362, "total_time_str": "14m 53s", "num_gpus": 1, "snapshot_pkl": "network-snapshot-000120.pkl", "timestamp": 1612338569.0465086} tick 31 kimg 124.0 time 7h 46m 48s sec/tick 792.4 sec/kimg 198.09 maintenance 914.1 cpumem 3.97 gpumem 8.38 augment 0.535 tick 32 kimg 128.0 time 8h 00m 01s sec/tick 792.4 sec/kimg 198.10 maintenance 0.1 cpumem 3.97 gpumem 8.18 augment 0.502 tick 33 kimg 132.0 time 8h 13m 12s sec/tick 790.7 sec/kimg 197.66 maintenance 0.1 cpumem 3.97 gpumem 8.03 augment 0.466 tick 34 kimg 136.0 time 8h 26m 22s sec/tick 789.9 sec/kimg 197.48 maintenance 0.1 cpumem 3.98 gpumem 8.09 augment 0.433 tick 35 kimg 140.0 time 8h 39m 31s sec/tick 789.2 sec/kimg 197.29 maintenance 0.1 cpumem 3.98 gpumem 8.05 augment 0.417 tick 36 kimg 144.0 time 8h 52m 38s sec/tick 787.1 sec/kimg 196.77 maintenance 0.1 cpumem 3.98 gpumem 8.19 augment 0.456 tick 37 kimg 148.0 time 9h 05m 46s sec/tick 788.3 sec/kimg 197.06 maintenance 0.1 cpumem 3.98 gpumem 8.11 augment 0.496
DiffAugment for StyleGAN2
see also > https://github.com/mit-han-lab/data-efficient-gans/tree/master/DiffAugment-stylegan2
further training
see details in 20210126-1611697225-grapheme.org
using weights from transfer training of foam flickr images from ffhq1024
various tweaks after ~5000 kimg cycles
python3 train.py --outdir=results/foam-flickr/20210316 \
--data=data/1024x1024-foam-flickr \
--resume=results/foam-flickr/20210310/00001-1024x1024-foam-flickr-auto2-resumecustom/network-snapshot-005040.pkl \
--snap=20 \
--gpus=2 \
--batch=8 \
--gamma=2
python3 train.py --outdir=results/foam-flickr/20210316 \
--data=data/1024x1024-foam-flickr \
--resume=results/foam-flickr/20210310/00001-1024x1024-foam-flickr-auto2-resumecustom/network-snapshot-005040.pkl \
--snap=20 \
--gpus=2 \
--batch=8 \
--gamma=120
python3 generate.py \ --network=results/foam-flickr/20210316/00000-1024x1024-foam-flickr-auto2-gamma2-batch16-resumecustom/network-snapshot-000000.pkl \ --outdir=results/foam-flickr/20210316/generated/foam-000000 \ --seeds=1-10000