feat: absorb the MThreads compat plugin and the A/B range report

Three pieces from the fork's feature/musa-benchmark-support branch that never
reached main, rebased onto the backend abstraction rather than copied verbatim.

_musa_benchmark_compat_plugin drops FlagGems benchmark skip guards written as
`torch.cuda.is_available()` where the code under test dispatches through
flag_gems.device. On MThreads those turn a working operator into "not measured":
test_fft.py skipped entirely, and unskipped on an S5000 it produces 25 rows. The
plugin pins each override to one file and one exact skip reason and only acts
when flag_gems has dispatched to the mthreads vendor; two guards were probed on
device and deliberately left in place, which is why they are named in its
docstring rather than silently absent.

It is wired through a new DeviceBackend.compat_plugins rather than a name test
in the planner, so the policy stays where every other backend difference lives.
Compat plugins are planned first because they rewrite collection-time markers
and nothing may read the item list before them; a test pins that position and
another pins that they do not leak to other backends.

batch_summary gains --ab: the speedup column compares gems against torch, not
one build against another, so an A/B needs the two run.logs paired by operator,
dtype and size detail. Only rows that succeeded on both sides with a positive
latency count, so an incomplete pairing cannot look like a complete range, and
the range is annotated when either side carries a BENCHMARK_DIRECT_NO_CUDAGRAPH
marker, since a fallback row was timed under a different regime.

ab_rlc_test.sh is the same caliber as ab_fold_test.sh with the comparison axis
moved to FLAGTREE_<VENDOR>_RLC_ENHANCE. Both are caliber examples, not
regression scripts: if the switch under test no longer exists the script still
runs every round and prints a full table showing no difference, so confirm the
axis is live before reusing either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
AharrypotterandClaude Opus 5 committed 2026-09-10 10:31:59 +08:00
1 parent c9f9b7885f
commit 29af59be9b
8 files changed
+258 -3

No files matched your search

+7
View File
@@ -90,6 +90,13 @@ class DeviceBackend:
def denies_capture(self, op: str) -> bool:
return bool(op) and op in self.capture_deny_ops
# Plugins that patch around upstream assumptions this backend violates, e.g.
# FlagGems benchmarks that gate themselves on torch.cuda.is_available() even
# though the operator under test runs on whatever device FlagGems dispatched
# to. Loaded first so collection-time overrides land before anything reads
# the item list. Each such plugin must scope itself to its own vendor.
compat_plugins: Tuple[str, ...] = ()
# Optional listing command; (argv, line_prefix). prefix="" means count
# non-empty lines. Must be a real vendor CLI; do not invent flags.
list_command: Optional[Tuple[Sequence[str], str]] = None
+5
View File
@@ -42,6 +42,11 @@ class MusaBackend(DeviceBackend):
# timing precision instead of its measurement. Revisit if a newer torch_musa
# changes the behaviour.
capture_deny_ops = ("unique2",)
# Several FlagGems benchmarks skip on `not torch.cuda.is_available()` while
# the code under test dispatches through flag_gems.device. On MThreads that
# turns a working operator into "not measured"; the plugin drops those
# guards, pinned to one file and one exact skip reason each.
compat_plugins = ("_musa_benchmark_compat_plugin",)
# Device count comes from torch.musa or MUSA_VISIBLE_DEVICES. Do not guess
# a vendor CLI here; confirm the real listing tool before adding it.
list_command = None