官方代码路径:
GitHub - CompVis/latent-diffusion: High-Resolution Image Synthesis with Latent Diffusion Models
官方参考论文路径:
[arXiv](https://arxiv.org/abs/2112.10752) | [BibTeX](#bibtex)
目前复现的:
使用预训练模型推理:
使用cpu进行autoencoder训练:
DDPM训练未完成:
硬件环境参考:
windows主机,单张显卡(暂时还没搞定gpu训练)
环境管理工具:
anaconda/mini conda均可
环境配置:
step01:
参考官方Readme.md中的内容,不过因为有墙,以及本地版本各种问题,所以我们需要修改一下yaml文件:
官方:
name: ldm
channels:- pytorch- defaults
dependencies:- python=3.8.5- pip=20.3- cudatoolkit=11.0- pytorch=1.7.0- torchvision=0.8.1- numpy=1.19.2- pip:- albumentations==0.4.3- opencv-python==4.1.2.30- pudb==2019.2- imageio==2.9.0- imageio-ffmpeg==0.4.2- pytorch-lightning==1.4.2- omegaconf==2.1.1- test-tube>=0.7.5- streamlit>=0.73.1- einops==0.3.0- torch-fidelity==0.3.0- transformers==4.3.1- -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers- -e git+https://github.com/openai/CLIP.git@main#egg=clip- -e .
修改后:
name: ldm
channels:# - pytorch- defaults
dependencies:- python=3.8.5- pip=20.3# - cudatoolkit=11.0# - pytorch=1.7.0# - torchvision=0.8.1- numpy=1.19.2- pip:- albumentations==0.4.3- opencv-python==4.1.2.30- pudb==2019.2- imageio==2.9.0- imageio-ffmpeg==0.4.2# - pytorch-lightning==1.4.2- omegaconf==2.1.1- test-tube>=0.7.5# - streamlit>=0.73.1- einops==0.3.0
然后在anaconda中执行:
conda env create -f environment.yaml
step02:
进入新建的ldm环境中,安装torch等相关库:
conda activate ldm
pip install torch==1.7.0 torchvision==0.8.1 torchaudio==0.7.0 torch-lightning==1.4.2
# 在更新一下torchmetrics
pip install torchmetrics==0.5
torch与torch-lighting以及相关库匹配官网可见:
Versioning Policy — PyTorch Lightning 2.4.0 documentation
|
|
|
|
| Python |
---|---|---|---|---|---|
2.4 | 2.4 | 2.4 | ≥2.1, ≤2.4 | ≥0.7.0 | ≥3.9, ≤3.12 |
2.3 | 2.3 | 2.3 | ≥2.0, ≤2.3 | ≥0.7.0 | ≥3.8, ≤3.11 |
2.2 | 2.2 | 2.2 | ≥1.13, ≤2.2 | ≥0.7.0 | ≥3.8, ≤3.11 |
2.1 | 2.1 | 2.1 | ≥1.12, ≤2.1 | ≥0.7.0 | ≥3.8, ≤3.11 |
2.0 | 2.0 | 2.0 (GA) | ≥1.11, ≤2.0 | ≥0.7.0 | ≥3.8, ≤3.10 |
1.9 | 1.9 | 1.9 (experimental) | ≥1.10, ≤1.13 | ≥0.7.0 | ≥3.7, ≤3.10 |
1.8** | 1.8 | n/a*** | ≥1.10, ≤1.13 | ≥0.7.0 | ≥3.7, ≤3.10 |
n/a | 1.7 | n/a*** | ≥1.9, ≤1.12 | ≥0.7.0 | ≥3.7, ≤3.10 |
n/a | 1.6 | n/a*** | ≥1.8, ≤1.11 | ≥0.4.1 | ≥3.7, ≤3.9 |
n/a | 1.5 | n/a*** | ≥1.7, ≤1.10 | ≥0.4.1 | ≥3.6, ≤3.9 |
n/a | 1.4 | n/a | ≥1.6, ≤1.9 | ≥0.4.0 | ≥3.6, ≤3.9 |
历史torch版本下载路径:
Previous PyTorch Versions | PyTorch
这样子,初步可以用来推理的环境就差不多了;
接下来是,预训练模型权重下载:
待续....