Day2Night | Image2Image Translation Research

Diploma

Graduation Work

Bachelor’s Degree

Lomonosov Moscow State University, Faculty of Computer Science, Graphics Lab

Usage

This repo is huge

System

• Python 3.6
• Ubuntu 18.04 LTS

  Requirements

  pip3 install -r requirements.txt
  pip3 install -r scripts/requirements.txt # for scripts 
  

  or use docker

---

Usage

Get project

Clone repository

# Full repo
git clone --depth 1 https://github.com/solesensei/day2night.git
# Mini repo (just code)
git clone --branch code --depth 1 https://github.com/solesensei/day2night.git

or get the code from drive (if not working)

wget --no-check-certificate -r "https://docs.google.com/uc?export=download&id=1mrj0vDzuFufpmxSW5SMIAn9XekegX4Hh" -O code.zip
unzip -o code.zip

Testing

Modify parametrs in test.sh

And run test

cd ~/prj/UNIT
bash ./scripts/test.sh
# or
python test_batch.py --device $gpu --config $config --input_folder $indir --output_folder $outdir --number $number --checkpoint $checkpoint --a2b $d2n --trainer UNIT --recon

Training

Modify parametrs in train.sh and configs.

Get data

I used two open datasets.

• Nexet 2017 provided by Nexar. See more.
• bdd100k 2018 under bdd100 licence from bdd100k.com. See more.

# BDD100k
kaggle datasets download -d solesensei/solesensei_bdd100k
# NEXET
kaggle datasets download -d solesensei/nexet-original

Prepare data

See DomainShifter and DataClassificator.

Start training

bash ./scripts/train.sh
# or
python train.py --device $gpu --config $config --trainer UNIT

Docker

Automated

Download bash script

wget https://raw.githubusercontent.com/solesensei/day2night/master/day2night/UNIT/scripts/day2night.sh -O ~/prj/day2night.sh

Then just run it (in repository you want to start)

cd ~/prj
bash day2night.sh

Manually

Get project

Get image

Pull Docker image

docker pull solesensei/day2night:pytorch_0.4.1 # CUDA 10 : Ubuntu 18.04 LTS
# or
docker pull solesensei/day2night:pytorch_0.4.1_cuda9 # CUDA 9 : Ubuntu 16.04 LTS

All tags:

• pytorch_0.4.1_cuda9 : CUDA 9, Ubuntu 16.04 LTS, Pytorch==0.4.1
• cyclegan - CUDA 9, Base, Pytorch>=1.0.1
• pytorch_0.4.1 - CUDA 10, Ubuntu 18.04 LTS, Pytorch==0.4.1
• pytorch_latest - CUDA 10, Ubuntu 18.04 LTS, Pytorch>=1.0.1

Run container

cd ~/prj
docker run -it -p 1111:1111 --name day2night --mount type=bind,source=$PWD,target=/mnt/w/prj -w /mnt/w/prj/UNIT --runtime nvidia -i -t solesensei/day2night:pytorch_0.4.1 # your tag here

Run Test

PT-models

• Google.Drive
• Minimal Source Code

UNIT vs. CycleGAN

RetinaNet	Day	Night	All
Real Data Images (NEXET)	0.8664	0.8406	0.8535
CycleGAN	0.8701 (+0.42%)	0.8571 (+1.96%)	0.8636 (+1.18%)
UNIT	0.8749 (+0.98%)	0.8512 (+1.26%)	0.8631 (+1.12%)

Faster R-CNN	Day	Night	All
Real Data Images (NEXET)	0.9015	0.8822	0.8919
CycleGAN	0.9087 (+0.79%)	0.8881 (+0.66%)	0.8984 (+0.72%)
UNIT	0.9066 (+0.56%)	0.8929 (+1.21%)	0.8998 (+0.88%)

VGG16 and Normalization

Results

UNIT

(top to bottom: input - reconstructed - translated)

Input - no VGG - VGG