forked from zhoulin/zl_bench
fix: have the campaign worker ask zl_bench for its plugin list
The worker carried a hand-copied plugin array, taken from zl_bench before backends/ existed, and it had drifted twice over. It still named _device_guard_plugin, which is now a shim for _device_setup_plugin. More consequentially it carried no timing plugin at all, so every measurement this campaign ever sealed was timer=direct while zl_bench's own driver had been capturing -- a difference the harness never reported, because the missing plugin cannot announce itself. It now calls plugin_plan.py with ZL_BACKEND=musa and uses whatever comes back, so the caliber follows the backend's declared capabilities instead of a snapshot of them. The plan is preflighted for non-emptiness, since an empty array would silently run pytest with no plugins at all: no shape injection, no autotune record, no IR dump, and a table that still looks well-formed. Two effects on this campaign. _timing_plugin now enters the list, and since MUSA declares graph capture the worker will capture where the operator allows it; operators that cannot are marked per row, as on any other backend. And _parallel_warmup_plugin joins, inert unless PARALLEL_WARMUP_GPUS is set. The sealed 32-workload run therefore sits on the far side of a caliber change and is not comparable with anything produced after this commit. Re-running it is a campaign decision, not a harness one. The worker test asserted the literal "-p _musa_benchmark_compat_plugin"; it now asserts the delegation and that no -p is spelled out locally. What the plan holds for musa is covered where it belongs, in zl_bench's test_backends. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
75405cc555
commit
d7cb7efc94
2 files changed
+17
-12
No files matched your search
@@ -53,17 +53,16 @@ export FLAGGEMS_PERF_CURRENT_OP="$OP"
|
||||
export FLAGGEMS_PERF_COLOR=never
|
||||
export NO_COLOR=1
|
||||
|
||||
plugins=(
|
||||
-p _musa_benchmark_compat_plugin
|
||||
-p _device_guard_plugin
|
||||
-p _seed_plugin
|
||||
-p _shape_inject_plugin
|
||||
-p _shape_iter_inject_plugin
|
||||
-p _bespoke_shape_plugin
|
||||
-p _autotune_record_plugin
|
||||
-p _pretty_report_plugin
|
||||
-p _ir_meta_plugin
|
||||
)
|
||||
# The plugin list comes from zl_bench, not from here. Hand-copying it is how
|
||||
# this campaign drifted: the copy was taken before backends/ existed, so it
|
||||
# still named _device_guard_plugin (now _device_setup_plugin) and, more
|
||||
# importantly, carried no timing plugin at all -- which is why every sealed
|
||||
# measurement is timer=direct while zl_bench's own driver had been capturing.
|
||||
# Asking the planner means the caliber tracks whatever the backend declares.
|
||||
export ZL_BACKEND=musa
|
||||
mapfile -t plugins < <("$PYTHON_BIN" "$ZL_BENCH_DIR/plugin_plan.py" \
|
||||
--backend "$ZL_BACKEND" --pytest-args | tr ' ' '\n')
|
||||
preflight "plugin plan is non-empty" test "${#plugins[@]}" -gt 0
|
||||
|
||||
dtype_args=()
|
||||
for dtype in ${DTYPES:-float16}; do
|
||||
|
||||
@@ -66,7 +66,13 @@ class MusaRlcProductTests(unittest.TestCase):
|
||||
self.assertIn(
|
||||
'FLAGGEMS_PERF_TTGIR_DUMP_DIR="$OUT_DIR/ttgir"', worker
|
||||
)
|
||||
self.assertIn("-p _musa_benchmark_compat_plugin", worker)
|
||||
# The plugin list is asked for, not written here: a hand-copied list is
|
||||
# what let this worker keep _device_guard_plugin and no timing plugin
|
||||
# long after zl_bench had moved on. What the plan contains for musa --
|
||||
# the compat plugin included -- is asserted in zl_bench's test_backends.
|
||||
self.assertIn("plugin_plan.py", worker)
|
||||
self.assertIn('--backend "$ZL_BACKEND"', worker)
|
||||
self.assertNotIn("-p _", worker)
|
||||
self.assertIn(
|
||||
'PYTHONPATH="$worker_dir:$ZL_BENCH_DIR:$FLAGGEMS_DIR/src:$FLAGTREE_SITE"',
|
||||
worker,
|
||||
|
||||
Reference in new issue
Block a user