sczhou /CodeFormer
[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer
★ 18,234⑂ 3,720👁 314⑃ 52● 379▤ 1
+6
Daily
18,234
stars
3,720
forks
314
watchers
52
prs
379
issues
1
releases
README
Towards Robust Blind Face Restoration with Codebook Lookup Transformer (NeurIPS 2022)
Paper | Project Page | Video
!Visitors
Shangchen Zhou, Kelvin C.K. Chan, Chongyi Li, Chen Change Loy
S-Lab, Nanyang Technological University
:star: If CodeFormer is helpful to your images or projects, please help star this repo. Thanks! :hugs:
Update
- 2023.07.20: Integrated to :panda_face: OpenXLab. Try out online demo!
- 2023.04.19: :whale: Training codes and config files are public available now.
- 2023.04.09: Add features of inpainting and colorization for cropped and aligned face images.
- 2023.02.10: Include dlib as a new face detector option, it produces more accurate face identity.
- 2022.10.05: Support video input --input_path [YOUR_VIDEO.mp4]. Try it to enhance your videos! :clapper:
- 2022.09.14: Integrated to :hugs: Hugging Face. Try out online demo!
- 2022.09.09: Integrated to :rocket: Replicate. Try out online demo!
- More
TODO
- [x] Add training code and config files
- [x] Add checkpoint and script for face inpainting
- [x] Add checkpoint and script for face colorization
- [x] ~~Add background image enhancement~~
:panda_face: Try Enhancing Old Photos / Fixing AI-arts
Face Restoration
Face Color Enhancement and Restoration
Face Inpainting
Dependencies and Installation
- Pytorch = 1.7.1
- CUDA = 10.1
- Other required packages in requirements.txt
git clone this repository
git clone https://github.com/sczhou/CodeFormer
cd CodeFormer
create new anaconda env
conda create -n codeformer python=3.8 -y
conda activate codeformer
install python dependencies
pip3 install -r requirements.txt
python basicsr/setup.py develop
conda install -c conda-forge dlib (only for face detection or cropping with dlib)
Quick Inference
Download Pre-trained Models:
Download the facelib and dlib pretrained models from [Releases | Google Drive | OneDrive] to the weights/facelib folder. You can manually download the pretrained models OR download by running the following command:
python scripts/download_pretrained_models.py facelib
python scripts/download_pretrained_models.py dlib (only for dlib face detector)
Download the CodeFormer pretrained models from [Releases | Google Drive | OneDrive] to the weights/CodeFormer folder. You can manually download the pretrained models OR download by running the following command:
python scripts/download_pretrained_models.py CodeFormer
Prepare Testing Data:
You can put the testing images in the inputs/TestWhole folder. If you would like to test on cropped and aligned faces, you can put them in the inputs/cropped_faces folder. You can get the cropped and aligned faces by running the following command:
you may need to install dlib via: conda install -c conda-forge dlib
python scripts/crop_align_face.py -i [input folder] -o [output folder]
Testing:
[Note] If you want to compare CodeFormer in your paper, please run the following command indicating --has_aligned (for croppeTop issues
#286Issues while loading the stage2 model18 comments · 2023-08-07#25Sharpness of the restored face14 comments · 2022-09-02#176Running on CPU now! Make sure your PyTorch version matches your CUDA12 comments · 2023-03-11#89Video method not working10 comments · 2022-12-30#44[Feature Suggestion] - Output Control10 comments · 2022-10-05#140av_interleaved_write_frame(): Broken pipe9 comments · 2023-02-08#102would the script be Apple M1 friendly?9 comments · 2023-01-15#336人脸会呈现许多暗色的斑点8 comments · 2023-12-21
Latest release
CodeFormer V0.1.0 Release v0.1.0 2022-08-10This release is mainly for storing pre-trained models, etc.