forked from zhoulin/zl_bench
feat: expose the upstream benchmark level as BENCH_LEVEL
run_pytest.sh hardcoded --level core, so the only shapes ever reachable were each operator's core_shapes entry. FlagGems merges a subclass's set_more_shapes() on top of that at --level comprehensive, and for BLAS that addition is where the large GEMMs live: mm goes from 5 shapes to 29, taking in four large-K tuples plus models_shapes/*.yaml, with N/K up to 152064. The comprehensive path also yields each shape a second time (transposed, for BLAS), so the case count doubles. None of that was reachable without editing the driver. BENCH_LEVEL defaults to core, so existing runs are unchanged, and rejects anything other than core/comprehensive rather than passing it through to pytest. The chosen level is echoed in the run header, written into the archived shapes.yaml, and recorded in run_identity.json, because an A/B whose two sides used different levels is not comparable and is otherwise invisible afterwards. Sweep cost scales with the shape count, so comprehensive on a BLAS operator wants PARALLEL_WARMUP_GPUS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c8ea586d0f
commit
92dae82c7f
3 files changed
+21
-6
No files matched your search
@@ -70,6 +70,7 @@ def main(argv=None) -> int:
|
||||
p.add_argument("--device", default=os.environ.get("ZL_DEVICE", ""))
|
||||
p.add_argument("--test-file", default=os.environ.get("TEST_FILE", ""))
|
||||
p.add_argument("--shape-mode", default=os.environ.get("SHAPE_MODE", ""))
|
||||
p.add_argument("--bench-level", default=os.environ.get("BENCH_LEVEL", "core"))
|
||||
args = p.parse_args(argv)
|
||||
init_identity(
|
||||
args.out_dir,
|
||||
@@ -81,6 +82,7 @@ def main(argv=None) -> int:
|
||||
device=args.device,
|
||||
test_file=args.test_file,
|
||||
shape_mode=args.shape_mode,
|
||||
bench_level=args.bench_level,
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
Reference in new issue
Block a user