Torch Hammer benchmark report

Sunspot — Sunspot 24-GPU fleet (2×12 Max 1550, stress) — fleet_sunspot_allgpus2
source: fleet_sunspot_allgpus2 GPU: Intel(R) Data Center GPU Max 1550 nodes: 2 GPUs: 24 benchmarks: 4
nodes
2
GPUs
24
benchmarks
4
outliers
0

(a) Batched GEMM (float32) [TFLOP/s]

fleet mean: 18.0 TFLOP/s | CV: 0.9% | median: 18 | p5/p95: 18/18
Fleet distribution of GPU mean performance (TFLOP/s)
0 2 4 GPU count 17.7–17.7 TFLOP/s: 2 GPUs 17.8–17.8 TFLOP/s: 4 GPUs 17.8–17.9 TFLOP/s: 4 GPUs 17.9–18.0 TFLOP/s: 2 GPUs 18.0–18.0 TFLOP/s: 5 GPUs 18.0–18.1 TFLOP/s: 1 GPUs 18.1–18.2 TFLOP/s: 2 GPUs 18.2–18.2 TFLOP/s: 4 GPUs 17.7 17.7 17.8 17.9 17.9 18.0 18.0 18.1 18.2 18.2 TFLOP/s mean
node GPU 0GPU 1GPU 2GPU 3GPU 4GPU 5GPU 6GPU 7GPU 8GPU 9GPU 10GPU 11 node mean vs fleet thrt
x1921c1s2b0n018181818181818181818181818+0.4%
x1921c1s3b0n018181818181818181818181818-0.4%

(b) Convolution (float32) [img/s]

fleet mean: 1,480,298.9 img/s | CV: 1.4% | median: 1,481,538 | p5/p95: 1,448,408/1,509,168
Fleet distribution of GPU mean performance (img/s)
0 1 2 3 4 GPU count 1.4M–1.5M img/s: 2 GPUs 1.5M–1.5M img/s: 4 GPUs 1.5M–1.5M img/s: 3 GPUs 1.5M–1.5M img/s: 4 GPUs 1.5M–1.5M img/s: 4 GPUs 1.5M–1.5M img/s: 2 GPUs 1.5M–1.5M img/s: 3 GPUs 1.5M–1.5M img/s: 1 GPUs 1.5M–1.5M img/s: 1 GPUs 1.5M 1.5M 1.5M 1.5M 1.5M 1.5M 1.5M 1.5M 1.5M 1.5M img/s mean
node GPU 0GPU 1GPU 2GPU 3GPU 4GPU 5GPU 6GPU 7GPU 8GPU 9GPU 10GPU 11 node mean vs fleet thrt
x1921c1s2b0n01,495,4541,500,1921,509,1681,523,1841,481,5381,468,5661,479,2001,485,2481,480,8941,478,5831,502,4411,498,6841,491,929+0.8%
x1921c1s3b0n01,448,4081,456,5921,466,7981,468,7551,488,3961,489,2801,446,5201,457,4621,454,2181,456,5961,488,4191,502,5801,468,669-0.8%

(c) Einsum Attention (float32) [TFLOP/s]

fleet mean: 13.1 TFLOP/s | CV: 7.6% | median: 13 | p5/p95: 12/14
Fleet distribution of GPU mean performance (TFLOP/s)
0 2 5 7 GPU count 12.0–12.3 TFLOP/s: 9 GPUs 12.3–12.5 TFLOP/s: 4 GPUs 12.5–12.8 TFLOP/s: 1 GPUs 13.7–14.0 TFLOP/s: 1 GPUs 14.0–14.2 TFLOP/s: 3 GPUs 14.2–14.5 TFLOP/s: 6 GPUs 12.1 12.4 12.6 12.9 13.1 13.4 13.6 13.9 14.1 14.3 TFLOP/s mean
node GPU 0GPU 1GPU 2GPU 3GPU 4GPU 5GPU 6GPU 7GPU 8GPU 9GPU 10GPU 11 node mean vs fleet thrt
x1921c1s2b0n014141414121214141414141414+6.1%
x1921c1s3b0n012121212131312121212121212-6.1%

(d) Memory Traffic (float32) [GB/s]

fleet mean: 25.7 GB/s | CV: 6.2% | median: 25 | p5/p95: 24/28
Fleet distribution of GPU mean performance (GB/s)
0 2 5 7 10 GPU count 23.9–24.3 GB/s: 4 GPUs 24.3–24.6 GB/s: 7 GPUs 24.6–25.0 GB/s: 2 GPUs 25.0–25.4 GB/s: 1 GPUs 27.3–27.6 GB/s: 10 GPUs 24.1 24.4 24.8 25.2 25.6 26.0 26.3 26.7 27.1 27.5 GB/s mean
node GPU 0GPU 1GPU 2GPU 3GPU 4GPU 5GPU 6GPU 7GPU 8GPU 9GPU 10GPU 11 node mean vs fleet thrt
x1921c1s2b0n028282827252427282827272827+5.0%
x1921c1s3b0n024242424252524242424252524-5.0%

Outliers

None detected