forked from zhoulin/zl_bench
feat: absorb the MThreads compat plugin and the A/B range report
Three pieces from the fork's feature/musa-benchmark-support branch that never reached main, rebased onto the backend abstraction rather than copied verbatim. _musa_benchmark_compat_plugin drops FlagGems benchmark skip guards written as `torch.cuda.is_available()` where the code under test dispatches through flag_gems.device. On MThreads those turn a working operator into "not measured": test_fft.py skipped entirely, and unskipped on an S5000 it produces 25 rows. The plugin pins each override to one file and one exact skip reason and only acts when flag_gems has dispatched to the mthreads vendor; two guards were probed on device and deliberately left in place, which is why they are named in its docstring rather than silently absent. It is wired through a new DeviceBackend.compat_plugins rather than a name test in the planner, so the policy stays where every other backend difference lives. Compat plugins are planned first because they rewrite collection-time markers and nothing may read the item list before them; a test pins that position and another pins that they do not leak to other backends. batch_summary gains --ab: the speedup column compares gems against torch, not one build against another, so an A/B needs the two run.logs paired by operator, dtype and size detail. Only rows that succeeded on both sides with a positive latency count, so an incomplete pairing cannot look like a complete range, and the range is annotated when either side carries a BENCHMARK_DIRECT_NO_CUDAGRAPH marker, since a fallback row was timed under a different regime. ab_rlc_test.sh is the same caliber as ab_fold_test.sh with the comparison axis moved to FLAGTREE_<VENDOR>_RLC_ENHANCE. Both are caliber examples, not regression scripts: if the switch under test no longer exists the script still runs every round and prints a full table showing no difference, so confirm the axis is live before reusing either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c9f9b7885f
commit
29af59be9b
8 files changed
+258
-3
No files matched your search
@@ -90,6 +90,13 @@ class DeviceBackend:
|
||||
|
||||
def denies_capture(self, op: str) -> bool:
|
||||
return bool(op) and op in self.capture_deny_ops
|
||||
|
||||
# Plugins that patch around upstream assumptions this backend violates, e.g.
|
||||
# FlagGems benchmarks that gate themselves on torch.cuda.is_available() even
|
||||
# though the operator under test runs on whatever device FlagGems dispatched
|
||||
# to. Loaded first so collection-time overrides land before anything reads
|
||||
# the item list. Each such plugin must scope itself to its own vendor.
|
||||
compat_plugins: Tuple[str, ...] = ()
|
||||
# Optional listing command; (argv, line_prefix). prefix="" means count
|
||||
# non-empty lines. Must be a real vendor CLI; do not invent flags.
|
||||
list_command: Optional[Tuple[Sequence[str], str]] = None
|
||||
|
||||
Reference in new issue
Block a user