Default USE_FLAGTUNE to 0 and add fold_scale A/B script with trace shapes
Plain autotune is the common case (fast, same-config A/B); FlagTune's exhaustive first-run search is now opt-in via USE_FLAGTUNE=1. ab_fold_test.sh automates the FLAGGEMS_MXFP4_FOLDSCALE=0/1 record-replay comparison over the four DeepSeek-V4-Flash trace shape sets, which are now tracked under runs/shape/.
This commit is contained in:
Executable
+36
@@ -0,0 +1,36 @@
|
||||
#!/bin/bash
|
||||
# FOLD 开关 A/B 验证脚本(4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
|
||||
# 口径:USE_FLAGTUNE=0;FOLD=0(基线)先跑并 record autotune config,FOLD=1 用
|
||||
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time,见实施记录 §5)。
|
||||
# 全部输出汇总到 runs/ab_fold_<时间戳>.log;各轮 runs/<op>_<时间戳>/ 产物照常存档。
|
||||
# 用法:bash ab_fold_test.sh
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# 终端颜色:下方 tee 管道会让 run_pytest.sh 侧检测不到 tty,在这里先判断并下传
|
||||
# (与 run_pytest.sh 自身的做法一致);总 log 最后统一去色。
|
||||
if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then
|
||||
export FLAGGEMS_PERF_COLOR=always
|
||||
fi
|
||||
|
||||
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
|
||||
|
||||
{
|
||||
for y in runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
|
||||
echo "########## $y | FOLD=0 baseline (record) ##########"
|
||||
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|
||||
|| { echo "!!! FAILED fold=0: $y"; continue; }
|
||||
REC=$(ls -dt runs/fused_marlin_moe_mxfp4_* | head -1)
|
||||
echo "########## $y | FOLD=1 (replay $REC) ##########"
|
||||
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=1 REPLAY_FROM="$REC" bash run_pytest.sh \
|
||||
|| echo "!!! FAILED fold=1: $y"
|
||||
done
|
||||
echo "########## ALL DONE ##########"
|
||||
} 2>&1 | tee "$LOG"
|
||||
|
||||
# 总 log 去掉 ANSI 转义,保证 grep/diff 面对纯文本(终端输出保留颜色)
|
||||
sed -i -E $'s/\x1b\\[[0-9;]*[A-Za-z]//g' "$LOG"
|
||||
|
||||
echo ">>> 总 log: $(pwd)/$LOG"
|
||||
echo ">>> 快速对账: grep -E '^#####|FAILED' $LOG"
|
||||
Reference in New Issue
Block a user