URSA/docs/evaluation.md

Evaluations

GenEval

1. Sample prompt images

python ./evaluations/geneval/sample.py \
--height 1024 --width 1024 \
--guidance_scale 7 --num_inference_steps 25 \
--ckpt /path/to/URSA-1.7B-IBQ1024 \
--prompt_size 4 --outdir ./samples/geneval/URSA-1.7B-IBQ1024

2. Evaluation

=./samples/geneval/URSA-1.7B-IBQ1024

Please refer GenEval evaluation guide.

DPG-Bench

1. Sample prompt images

python evaluations/dpgbench/sample.py \
--height 1024 --width 1024 \
--guidance_scale 7 --num_inference_steps 25 \
--ckpt ./checkpoints/URSA-1.7B-IBQ1024 \
--prompt_size 4 --outdir samples/dpgbench/URSA-1.7B-IBQ1024

2. Evaluation

=./samples/dpgbench/URSA-1.7B-IBQ1024

Please refer DPG-Bench evaluation guide.

VBench

1. Sample prompt videos

python evaluations/vbench/sample.py \
--num_frames 49 --height 320 --width 512 \
--guidance_scale 7 --num_inference_steps 50 --motion_score 9 \
--ckpt ./checkpoints/URSA-1.7B-FSQ320 \
--prompt_size 1 --outdir ./samples/vbench/URSA-1.7B-FSQ320

2. Evaluation

=./samples/vbench/URSA-1.7B-FSQ320

Please refer VBench evaluation guide.