Move trace shape sets to top-level shape/ and frame ab_fold_test.sh as A/B reference example
This commit is contained in:
+1
-2
@@ -1,3 +1,2 @@
|
||||
runs/*
|
||||
!runs/shape/
|
||||
__pycache__/
|
||||
runs/
|
||||
|
||||
@@ -85,16 +85,16 @@ A/B 两侧的 `USE_FLAGTUNE` 必须取同值:该开关决定调优空间,两
|
||||
|
||||
replay 的兜底行为:B 侧遇到记录中没有的 key、或记录的 config 在新编译器下编译失败时,自动回退到现场 autotune 并在 `run.log` 打 `AUTOTUNE_REPLAY_FALLBACK reason=...` 标记——出现该标记的测量点不再满足"同 config"前提,解读时注意。另外 replay 模式的 run 目录不产生 `autotune_records/`,后续 run 的 `REPLAY_FROM` 应始终指向最初 record 的那次 A 侧目录,不要链式指向 replay 产物。
|
||||
|
||||
### ab_fold_test.sh:FOLD 开关 A/B(现成示例)
|
||||
### ab_fold_test.sh:A/B 测试参考示例
|
||||
|
||||
`ab_fold_test.sh` 把上述流程封装成 FlagGems `FLAGGEMS_MXFP4_FOLDSCALE`(fold_scale 优化)开关的 A/B 验证:对 4 个 DeepSeek-V4-Flash 真实 trace shape 集(`runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml`,随仓库提供)各跑一对 FOLD=0(基线,record)→ FOLD=1(replay 同 config),共 8 轮:
|
||||
`ab_fold_test.sh` 是上述 A/B 流程的现成参考:以 FlagGems `FLAGGEMS_MXFP4_FOLDSCALE`(fold_scale 优化)开关为对比对象,对 4 个 DeepSeek-V4-Flash 真实 trace shape 集(`shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml`,随仓库提供)各跑一对 FOLD=0(基线,record)→ FOLD=1(replay 同 config),共 8 轮:
|
||||
|
||||
```bash
|
||||
bash ab_fold_test.sh
|
||||
# 汇总 log:runs/ab_fold_<时间戳>.log(已去色);快速对账:grep -E '^#####|FAILED' <log>
|
||||
```
|
||||
|
||||
对比其它开关 / 算子时可直接改脚本内的循环变量与环境变量,A/B 口径(同 config replay、cudagraph 计时)无需改动。
|
||||
要对比其它开关 / 编译器改动 / 算子,套用该脚本改循环变量与环境变量即可,A/B 口径(同 config replay、cudagraph 计时)无需改动。
|
||||
|
||||
### cudagraph 计时的 warmup、回退与精度
|
||||
|
||||
|
||||
+5
-3
@@ -1,7 +1,9 @@
|
||||
#!/bin/bash
|
||||
# FOLD 开关 A/B 验证脚本(4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
|
||||
# A/B 测试参考示例:FLAGGEMS_MXFP4_FOLDSCALE(fold_scale)开关验证
|
||||
# (4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
|
||||
# 口径:USE_FLAGTUNE=0;FOLD=0(基线)先跑并 record autotune config,FOLD=1 用
|
||||
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time,见实施记录 §5)。
|
||||
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time)。
|
||||
# 对比其它开关/算子时,改循环变量与环境变量即可复用同一口径。
|
||||
# 全部输出汇总到 runs/ab_fold_<时间戳>.log;各轮 runs/<op>_<时间戳>/ 产物照常存档。
|
||||
# 用法:bash ab_fold_test.sh
|
||||
set -uo pipefail
|
||||
@@ -17,7 +19,7 @@ fi
|
||||
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
|
||||
|
||||
{
|
||||
for y in runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
|
||||
for y in shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
|
||||
echo "########## $y | FOLD=0 baseline (record) ##########"
|
||||
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|
||||
|| { echo "!!! FAILED fold=0: $y"; continue; }
|
||||
|
||||
Reference in New Issue
Block a user