The speedup column in a result table compares gems against torch, not one build
against another, so an A/B needs the two runs paired by shape. batch_summary
gains an --ab mode that indexes both run.logs by operator, dtype and size
detail, computes before/after per pair, and prints the per-shape rows plus the
range. Only rows that succeeded on both sides with a positive latency count, so
an incomplete pairing cannot masquerade as a complete range. When either side
carries a BENCHMARK_DIRECT_NO_CUDAGRAPH marker the range is annotated, because
a fallback row was timed under a different regime than a captured one.
ab_rlc_test.sh drives the pairing for the FlagTree RemoveLayoutConversions
switch the way ab_fold_test.sh does for the FlagGems one: per operator, run with
the enhancement off while recording autotune configs, then replay those configs
with it on, so the two sides differ by that switch alone. Each operator
therefore leaves exactly two run.log files, and the summary of each pair is
printed as it completes.
Checked on an S5000 against gelu_backward over five shapes: both sides captured
with no fallback, five rows paired, range -0.43% to +0.09%.
Several upstream FlagGems tests gate themselves on torch.cuda.is_available()
even though the code under test runs on whatever accelerator FlagGems
dispatches to. On MThreads those guards skip the operator instead of exercising
it, so a working path reports as "not measured".
A guard is dropped only when it is pinned to one file and one exact skip
reason, only when FlagGems has dispatched to the MThreads vendor, and only
after the unskipped test was observed to pass on an S5000. The plugin walks the
node chain rather than the item's own markers, because a module-level pytestmark
does not appear on the item.
Two guards were probed on device and deliberately left in place, which is why
they are named in the docstring rather than silently absent.
test_grid_sample.py builds its inputs through a helper whose device argument
defaults to the literal "cuda": unskipped, all 90 collected cases fail in
aten::empty on a CUDA backend that is not present. test_cutlass_scaled_mm.py
gates on vLLM as well as CUDA, so removing its guard would raise on the missing
dependency instead of measuring anything.
The cudagraph plugin swapped do_bench for do_bench_cudagraph, but upstream's
do_bench_cudagraph names torch.cuda directly: torch.cuda.stream, Event,
synchronize, CUDAGraph, graph. FlagTree's do_bench already resolves the device
through driver.active.get_device_interface(), its cudagraph twin does not. On a
non-CUDA backend every capture therefore raised and every measurement silently
fell back to plain do_bench, so the plugin looked enabled while nothing was
being captured.
The plugin now resolves the interface itself and takes the stream, event,
synchronize and graph calls from it. CUDA still calls upstream's
do_bench_cudagraph verbatim, so its numbers do not move; other backends get an
otherwise step-for-step port. The scratch-RNG guard around capture is skipped
when the interface exposes no default_generators, which is the case on MThreads
and where there is no generator registration to protect against.
The replay timings are handed to _summarize_statistics as a plain list of
floats, matching do_bench and upstream's cudagraph path. This fork's helper runs
Python's statistics module over them, so wrapping them in a tensor makes "mean"
raise and makes a quantile request return tensors that later fail to serialize.
Verified on a Moore Threads S5000 through torch_musa, which supplies Stream,
Event, synchronize, graph and MUSAGraph. softmax and gelu_backward capture and
replay with no fallback marker. RNG operators cannot be captured there yet:
dropout aborts with "Cannot call MUSAGeneratorImpl::current_seed during MUSA
graph capture" and takes the documented BENCHMARK_DIRECT_NO_CUDAGRAPH fallback,
so a corpus mixing RNG and non-RNG operators mixes two timing regimes and the
marker is the thing to grep before comparing them.
The drivers here are shell scripts executed on Linux accelerator hosts, and
consumers record the SHA-256 of the files they ran. With Git's default autocrlf
on Windows a checkout rewrites those files with CRLF, which breaks the shell
drivers outright and changes every digest a receipt refers to.
Aharrypotter