Teacher sample for the prompt: a photo of three apples
Teacher “a photo of three apples”

One model, any number of loops

The reel steps through the SD3-medium teacher and then Dooop with K = 1, 2, 3, 4, 8 and 16 loops. Every frame starts from the same initial noise.

    Overview

    Looped transformers reuse one block of shared weights along the depth of the network. Recent looped diffusion transformers are trained from scratch, which repeats the full pretraining cost even though strong pretrained non-looped diffusion transformers already exist.

    Dooop distills a pretrained non-looped diffusion transformer into a looped student. The student keeps a few leading and trailing blocks of the teacher and replaces the middle stack by a small set of shared blocks that are looped. Every block of the student starts from a teacher block.

    We cast distillation as a fixed-point iteration toward the teacher: the output after every loop is trained toward the teacher, and each loop is one step of an ordinary differential equation in depth. As a result, one model runs any number of loops at test time without retraining, and the higher-order ODE solvers of diffusion sampling can be applied along depth. Because a looped model produces an output after every loop, we also build loop guidance between the outputs of different loop counts, which improves samples at little extra compute.

    • 0.756Bstudent parameters, about 37% of the teacher's
    • about 58%of the teacher's compute with two loops
    • 100.7 vs 100average over GenEval, ImageReward, PickScore and HPSv2.1 with two loops, teacher = 100
    • 105.8average after continued training on real data; GenEval 0.707 vs the teacher's 0.648
    • 88.2 → 99.2average of a four-step student with loop guidance, at 7.21 → 7.79 TFLOPs per image

    Any number of loops

    Pick a prompt and a loop count. The Dooop sample is shown next to the teacher sample from the same initial noise. Changing the prompt keeps the selected K.

    Loops (K)
    Teacher (SD3-medium, 2.03B)
    Dooop, K = 2

    Teacher vs. Dooop

    The Dooop sample is drawn on top of the teacher sample. Move the slider to fade between them. The prompt and K follow the selection above.

    Loops (K)

    Method

    Each loop moves the student a step toward the teacher. The target for the output after a loop mixes the previous loop's output and the teacher velocity, so every loop is pulled toward the teacher and the last loop reaches it. The loops are indexed by a continuous depth coordinate from 0 to 1, so every loop count covers the same depth interval and the loop count is chosen at inference. Training uses noisy data states and rollout states from a sampler that combines the teacher and the current student, so training covers the states the student meets when it samples.

    Training, step by step

    Training pipeline of Dooop: (a) the looped student initialized from the frozen teacher, (b) training states mixing noisy data states and rollout states, (c) the student forward pass with per-round targets.

      Inference with loop guidance, step by step

      Loop guidance at one sampling step: (a) strong and weak predictions of the student, (b) the loop guidance combination and the sampler step.

        Results

        Text-to-image generation with SD3-medium as the teacher

        Avg. (%) is the average over GenEval, ImageReward, PickScore and HPSv2.1 relative to the teacher (teacher = 100.0). Best student result of each part in bold.

        Text-to-image generation with SD3-medium as the teacher. Best student result of each part in bold.
        Model TFLOPs K GenEval ↑ Image­Reward ↑ Pick­Score ↑ HPSv2.1 ↑ Avg. (%) ↑
        Single obj. Two obj. Counting Colors Position Color attr. Overall
        SD3-medium (teacher)50.5–0.9940.7530.5060.8400.2850.5080.6480.90822.2140.278100.0
        Distill
        Dooop (ours)18.910.9880.7370.3220.8110.2480.4280.5890.66321.7900.25988.8
        Dooop (ours)29.420.9910.8380.4560.8560.2880.5350.6610.92422.2450.275100.7
        Dooop (ours)39.930.9840.8130.4660.8430.2700.5500.6540.93322.3070.276100.9
        Dooop (ours)50.540.9880.7950.5030.8300.2730.5650.6590.91922.3120.276100.7
        Dooop (ours)92.680.9840.8030.4750.8320.2450.5480.6480.88922.2780.27599.3
        Dooop (ours)177.0160.9910.7600.4380.8380.2500.5250.6330.85322.2290.27397.6
        + Real data (training continued on real images after distillation)
        Dooop (ours)18.910.9940.8110.3940.8460.3050.4950.6410.77021.8310.26794.5
        Dooop (ours)29.420.9970.8790.4970.8800.3530.6350.7071.02122.2820.281105.8
        Dooop (ours)39.930.9940.8890.5220.8860.3430.6080.7071.03922.3910.284106.6
        Dooop (ours)50.540.9970.8840.5470.8780.3430.6100.7101.03722.3900.283106.6

        Compute per image

        TFLOPs per image from the table above. The student with four loops costs the same per-image compute as the teacher.

        Teacher50.5
        Dooop K = 118.9
        Dooop K = 229.4
        Dooop K = 339.9
        Dooop K = 450.5
        Dooop K = 892.6
        Dooop K = 16177.0

        Loop guidance on a four-step student

        The student is the CDM-distilled four-step SD3 student, and the teacher is the four-step CDM model. The weak model is either a separate forward pass with k loops or the output after loop k of the same K = 4 forward pass. Best student result in bold.

        Loop guidance on the four-step SD3 student. Best student result in bold.
        Setting wl NFE TFLOPs GenEval ↑ Image­Reward ↑ Pick­Score ↑ HPSv2.1 ↑ Avg. ↑
        Teacher–47.210.6511.02622.8220.298100.0
        Student147.210.5920.73922.3810.27388.2
        Weak model: separate k-loop forward
        k = 1289.900.6390.94122.5680.29096.5
        k = 1389.900.6360.97522.4160.29597.5
        k = 23811.410.6270.90722.5400.28895.0
        k = 24811.410.6310.93522.4880.29196.1
        Weak model: loop-k output of the same forward
        k = 1247.790.6431.00222.5750.29698.7
        k = 1347.790.6331.02922.2480.29398.3
        k = 2247.790.6340.97122.6130.29497.4
        k = 2347.790.6431.02522.4540.29799.2

        Metrics during training

        Four line plots of GenEval, ImageReward, PickScore and HPSv2.1 for Dooop on SD3-medium with K = 2, 3 and 4 loops over training, with a dotted line for the teacher.
        GenEval, ImageReward, PickScore and HPSv2.1 of Dooop on SD3-medium with K = 2, 3, 4 loops during training. The dotted line marks the teacher.