forked from zhoulin/zl_bench
bench: let the verdict govern the report, and put the caliber in the table
A pair judged incomparable was still followed by a speedup range on the next line. A NO_TRITON pair's range says nothing about the switch — nothing reached the compiler — and a NO_IR_DIFF pair's range is the machine's noise floor for that operator, which is useful but is not a result. Printing either unlabelled produces exactly the thing this tooling exists to prevent: a table that looks like a finding. NO_ROWS and NO_TRITON now print no range; NO_IR_DIFF prints one and says what it is — how far it sits from zero is the confidence floor for a single measurement of that operator. screen.tsv gains the seven caliber columns, read back through `batch_summary.py --caliber`, so CALIBER_FIELDS stays the single definition and the screening table does not restate it. Two screen.tsv files taken at different --level are otherwise indistinguishable side by side, and those columns are the only thing that would stop the comparison.
This commit is contained in:
1 parent
a11863399a
commit
2e456bdc74
3 files changed
+38
-7
No files matched your search
@@ -172,6 +172,20 @@ SWITCH=FLAGTREE_MUSA_RLC_ENHANCE ROUNDS=6 OPS="topk:topk" bash ab_repeat.sh
|
||||
同一份 `_ir_fingerprint.sh` 也被 `ab_repeat.sh` 用于第 1 轮后的对比轴有效性检查,
|
||||
两处判据不会漂开。
|
||||
|
||||
判定还管住报告:`NO_ROWS` 与 `NO_TRITON` 不出 speedup 区间(前者没有可比的测量,
|
||||
后者与被测开关无关);`NO_IR_DIFF` 照出,但明说那是**本机对该算子的噪声底**,不是
|
||||
开关的收益——它离零多远,就是该算子单次测量的可信下限。判定刚说完这一对不可比,
|
||||
下一行就给个 `+0.3%`,正是这套工具一路在防的"看起来像结论的表"。
|
||||
|
||||
`screen.tsv` 末七列是口径(`backend device timer bench_level shape_mode
|
||||
tuner_cache direct_repeat`),由 `python batch_summary.py --caliber <run_dir>` 取——
|
||||
`CALIBER_FIELDS` 只有那一处定义,筛选表不重列一遍。两份不同 `--level` 的
|
||||
`screen.tsv` 放一起比时,这几列是唯一能拦住的东西。
|
||||
|
||||
每对有 `CELL_TIMEOUT`(默认 2400 秒)兜底,超时打 `!!! CELL_TIMEOUT` 并按耗时问题
|
||||
报告,不混进 dtype 那条解释里。`elapsed_before/after` 也进表:两步流程的全部依据
|
||||
就是"一对 0.8 分钟对六轮 13~44 分钟"这个比例,那这一步该记下自己那一半。
|
||||
|
||||
## A/B 对比测试标准流程
|
||||
|
||||
```bash
|
||||
|
||||
Reference in new issue
Block a user