bench: let the verdict govern the report, and put the caliber in the table

A pair judged incomparable was still followed by a speedup range on the next
line. A NO_TRITON pair's range says nothing about the switch — nothing reached
the compiler — and a NO_IR_DIFF pair's range is the machine's noise floor for
that operator, which is useful but is not a result. Printing either unlabelled
produces exactly the thing this tooling exists to prevent: a table that looks
like a finding. NO_ROWS and NO_TRITON now print no range; NO_IR_DIFF prints one
and says what it is — how far it sits from zero is the confidence floor for a
single measurement of that operator.

screen.tsv gains the seven caliber columns, read back through
`batch_summary.py --caliber`, so CALIBER_FIELDS stays the single definition and
the screening table does not restate it. Two screen.tsv files taken at
different --level are otherwise indistinguishable side by side, and those
columns are the only thing that would stop the comparison.
This commit is contained in:
Aharrypotter committed 2026-09-10 17:10:59 +08:00
1 parent a11863399a
commit 2e456bdc74
3 files changed
+38 -7

No files matched your search

+17 -7
View File
@@ -59,7 +59,7 @@ ROOT="runs/ab_rlc_$(date +%Y%m%d-%H%M%S)"
mkdir -p "$ROOT"
LOG="$ROOT/summary.log"
TSV="$ROOT/screen.tsv"
printf 'op op_file dtypes rows_before rows_after kernels_before kernels_after cvt_before cvt_after elapsed_before elapsed_after verdict\n' > "$TSV"
printf 'op op_file dtypes rows_before rows_after kernels_before kernels_after cvt_before cvt_after elapsed_before elapsed_after verdict backend device timer bench_level shape_mode tuner_cache direct_repeat\n' > "$TSV"
count_cvt() { find "$1/ttgir" -name '*.ttgir' -type f -exec cat {} + 2>/dev/null | grep -c convert_layout || true; }
count_kernels() { find "$1/ttgir" -name '*.ttgir' -type f 2>/dev/null | wc -l; }
@@ -114,15 +114,25 @@ count_rows() { grep -c '^SUCCESS' "$1/run.log" 2>/dev/null || true; }
verdict=IR_CHANGED
note="生成代码有变(cvt ${cb}→${ca},指纹 ${fb:0:8}/${fa:0:8}):值得排六轮 A/B"
fi
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
# 口径从 batch_summary.py 取,那里是 CALIBER_FIELDS 的唯一定义。两份不同
# --level 的 screen.tsv 放一起比时,这几列是唯一能拦住的东西。
cal=$(python batch_summary.py --caliber "$BEFORE" 2>/dev/null)
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
"$op" "$op_file" "${dtypes:-<all>}" "${rb:-0}" "${ra:-0}" \
"$kb" "$ka" "${cb:-0}" "${ca:-0}" "$el_before" "$el_after" "$verdict" >> "$TSV"
"$kb" "$ka" "${cb:-0}" "${ca:-0}" "$el_before" "$el_after" "$verdict" \
"$cal" >> "$TSV"
echo ">>> $op: $verdict — $note"
if [[ "$verdict" == NO_ROWS ]]; then
echo "!!! $op 没有可对比的测量,不出 speedup 区间"
continue
fi
# 判定说这一对不可比,就别在下一行给出一个看起来像结论的区间
case "$verdict" in
NO_ROWS)
echo "!!! $op 没有可对比的测量,不出 speedup 区间"; continue ;;
NO_TRITON)
echo "!!! $op 没有经过 Triton,speedup 区间与被测开关无关,不出"; continue ;;
NO_IR_DIFF)
echo ">>> $op 两侧生成代码相同:下面这个区间是本机对该算子的噪声底,"
echo ">>> 不是开关的收益。它离零多远,就是该算子单次测量的可信下限。" ;;
esac
echo "########## $op | summary ##########"
python batch_summary.py --ab "$BEFORE/run.log" "$AFTER/run.log" --label "$op" \
|| echo "!!! summary failed: $op"