Default USE_FLAGTUNE to 0 and add fold_scale A/B script with trace shapes

Plain autotune is the common case (fast, same-config A/B); FlagTune's
exhaustive first-run search is now opt-in via USE_FLAGTUNE=1.
ab_fold_test.sh automates the FLAGGEMS_MXFP4_FOLDSCALE=0/1 record-replay
comparison over the four DeepSeek-V4-Flash trace shape sets, which are
now tracked under runs/shape/.
This commit is contained in:
2026-07-20 16:50:59 +00:00
parent 26b071c6e1
commit 00902e6c67
12 changed files with 502 additions and 6 deletions
+36
View File
@@ -0,0 +1,36 @@
#!/bin/bash
# FOLD 开关 A/B 验证脚本(4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
# 口径:USE_FLAGTUNE=0FOLD=0(基线)先跑并 record autotune configFOLD=1 用
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time,见实施记录 §5)。
# 全部输出汇总到 runs/ab_fold_<时间戳>.log;各轮 runs/<op>_<时间戳>/ 产物照常存档。
# 用法:bash ab_fold_test.sh
set -uo pipefail
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# 终端颜色:下方 tee 管道会让 run_pytest.sh 侧检测不到 tty,在这里先判断并下传
# (与 run_pytest.sh 自身的做法一致);总 log 最后统一去色。
if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then
export FLAGGEMS_PERF_COLOR=always
fi
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
{
for y in runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
echo "########## $y | FOLD=0 baseline (record) ##########"
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|| { echo "!!! FAILED fold=0: $y"; continue; }
REC=$(ls -dt runs/fused_marlin_moe_mxfp4_* | head -1)
echo "########## $y | FOLD=1 (replay $REC) ##########"
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=1 REPLAY_FROM="$REC" bash run_pytest.sh \
|| echo "!!! FAILED fold=1: $y"
done
echo "########## ALL DONE ##########"
} 2>&1 | tee "$LOG"
# 总 log 去掉 ANSI 转义,保证 grep/diff 面对纯文本(终端输出保留颜色)
sed -i -E $'s/\x1b\\[[0-9;]*[A-Za-z]//g' "$LOG"
echo ">>> 总 log: $(pwd)/$LOG"
echo ">>> 快速对账: grep -E '^#####|FAILED' $LOG"