forked from zhoulin/zl_bench
feat: absorb the MThreads compat plugin and the A/B range report
Three pieces from the fork's feature/musa-benchmark-support branch that never reached main, rebased onto the backend abstraction rather than copied verbatim. _musa_benchmark_compat_plugin drops FlagGems benchmark skip guards written as `torch.cuda.is_available()` where the code under test dispatches through flag_gems.device. On MThreads those turn a working operator into "not measured": test_fft.py skipped entirely, and unskipped on an S5000 it produces 25 rows. The plugin pins each override to one file and one exact skip reason and only acts when flag_gems has dispatched to the mthreads vendor; two guards were probed on device and deliberately left in place, which is why they are named in its docstring rather than silently absent. It is wired through a new DeviceBackend.compat_plugins rather than a name test in the planner, so the policy stays where every other backend difference lives. Compat plugins are planned first because they rewrite collection-time markers and nothing may read the item list before them; a test pins that position and another pins that they do not leak to other backends. batch_summary gains --ab: the speedup column compares gems against torch, not one build against another, so an A/B needs the two run.logs paired by operator, dtype and size detail. Only rows that succeeded on both sides with a positive latency count, so an incomplete pairing cannot look like a complete range, and the range is annotated when either side carries a BENCHMARK_DIRECT_NO_CUDAGRAPH marker, since a fallback row was timed under a different regime. ab_rlc_test.sh is the same caliber as ab_fold_test.sh with the comparison axis moved to FLAGTREE_<VENDOR>_RLC_ENHANCE. Both are caliber examples, not regression scripts: if the switch under test no longer exists the script still runs every round and prints a full table showing no difference, so confirm the axis is live before reusing either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c9f9b7885f
commit
29af59be9b
8 files changed
+258
-3
No files matched your search
@@ -173,6 +173,17 @@ class PluginPlanTests(unittest.TestCase):
|
||||
self.assertNotIn("_cudagraph_plugin", names)
|
||||
self.assertNotIn("_mm_cluster_fix_plugin", names)
|
||||
|
||||
def test_musa_loads_its_compat_plugin_first(self):
|
||||
names = self._plan("musa")
|
||||
self.assertIn("_musa_benchmark_compat_plugin", names)
|
||||
# It rewrites collection-time markers, so nothing may inspect the item
|
||||
# list before it runs.
|
||||
self.assertEqual(names[0], "_musa_benchmark_compat_plugin")
|
||||
|
||||
def test_compat_plugins_do_not_leak_to_other_backends(self):
|
||||
for name in ("cuda", "rocm", "ascend"):
|
||||
self.assertNotIn("_musa_benchmark_compat_plugin", self._plan(name), name)
|
||||
|
||||
def test_musa_same_core_without_cluster_fix(self):
|
||||
# _cudagraph_plugin is absent from the plan on every backend: the timing
|
||||
# plugin loads it, so its absence here says nothing about capture support.
|
||||
|
||||
Reference in new issue
Block a user