Move trace shape sets to top-level shape/ and frame ab_fold_test.sh as A/B reference example
This commit is contained in:
1 parent
00902e6c67
commit
c8a42b66ee
11 files changed
+9
-8
No files matched your search
+1
-2
@@ -1,3 +1,2 @@
|
|||||||
runs/*
|
|
||||||
!runs/shape/
|
|
||||||
__pycache__/
|
__pycache__/
|
||||||
|
runs/
|
||||||
@@ -85,16 +85,16 @@ A/B 两侧的 `USE_FLAGTUNE` 必须取同值:该开关决定调优空间,两
|
|||||||
|
|
||||||
replay 的兜底行为:B 侧遇到记录中没有的 key、或记录的 config 在新编译器下编译失败时,自动回退到现场 autotune 并在 `run.log` 打 `AUTOTUNE_REPLAY_FALLBACK reason=...` 标记——出现该标记的测量点不再满足"同 config"前提,解读时注意。另外 replay 模式的 run 目录不产生 `autotune_records/`,后续 run 的 `REPLAY_FROM` 应始终指向最初 record 的那次 A 侧目录,不要链式指向 replay 产物。
|
replay 的兜底行为:B 侧遇到记录中没有的 key、或记录的 config 在新编译器下编译失败时,自动回退到现场 autotune 并在 `run.log` 打 `AUTOTUNE_REPLAY_FALLBACK reason=...` 标记——出现该标记的测量点不再满足"同 config"前提,解读时注意。另外 replay 模式的 run 目录不产生 `autotune_records/`,后续 run 的 `REPLAY_FROM` 应始终指向最初 record 的那次 A 侧目录,不要链式指向 replay 产物。
|
||||||
|
|
||||||
### ab_fold_test.sh:FOLD 开关 A/B(现成示例)
|
### ab_fold_test.sh:A/B 测试参考示例
|
||||||
|
|
||||||
`ab_fold_test.sh` 把上述流程封装成 FlagGems `FLAGGEMS_MXFP4_FOLDSCALE`(fold_scale 优化)开关的 A/B 验证:对 4 个 DeepSeek-V4-Flash 真实 trace shape 集(`runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml`,随仓库提供)各跑一对 FOLD=0(基线,record)→ FOLD=1(replay 同 config),共 8 轮:
|
`ab_fold_test.sh` 是上述 A/B 流程的现成参考:以 FlagGems `FLAGGEMS_MXFP4_FOLDSCALE`(fold_scale 优化)开关为对比对象,对 4 个 DeepSeek-V4-Flash 真实 trace shape 集(`shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml`,随仓库提供)各跑一对 FOLD=0(基线,record)→ FOLD=1(replay 同 config),共 8 轮:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash ab_fold_test.sh
|
bash ab_fold_test.sh
|
||||||
# 汇总 log:runs/ab_fold_<时间戳>.log(已去色);快速对账:grep -E '^#####|FAILED' <log>
|
# 汇总 log:runs/ab_fold_<时间戳>.log(已去色);快速对账:grep -E '^#####|FAILED' <log>
|
||||||
```
|
```
|
||||||
|
|
||||||
对比其它开关 / 算子时可直接改脚本内的循环变量与环境变量,A/B 口径(同 config replay、cudagraph 计时)无需改动。
|
要对比其它开关 / 编译器改动 / 算子,套用该脚本改循环变量与环境变量即可,A/B 口径(同 config replay、cudagraph 计时)无需改动。
|
||||||
|
|
||||||
### cudagraph 计时的 warmup、回退与精度
|
### cudagraph 计时的 warmup、回退与精度
|
||||||
|
|
||||||
|
|||||||
+5
-3
@@ -1,7 +1,9 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# FOLD 开关 A/B 验证脚本(4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
|
# A/B 测试参考示例:FLAGGEMS_MXFP4_FOLDSCALE(fold_scale)开关验证
|
||||||
|
# (4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮)。
|
||||||
# 口径:USE_FLAGTUNE=0;FOLD=0(基线)先跑并 record autotune config,FOLD=1 用
|
# 口径:USE_FLAGTUNE=0;FOLD=0(基线)先跑并 record autotune config,FOLD=1 用
|
||||||
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time,见实施记录 §5)。
|
# REPLAY_FROM 回放同一套 config(两侧同配置比 wall-time)。
|
||||||
|
# 对比其它开关/算子时,改循环变量与环境变量即可复用同一口径。
|
||||||
# 全部输出汇总到 runs/ab_fold_<时间戳>.log;各轮 runs/<op>_<时间戳>/ 产物照常存档。
|
# 全部输出汇总到 runs/ab_fold_<时间戳>.log;各轮 runs/<op>_<时间戳>/ 产物照常存档。
|
||||||
# 用法:bash ab_fold_test.sh
|
# 用法:bash ab_fold_test.sh
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
@@ -17,7 +19,7 @@ fi
|
|||||||
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
|
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
|
||||||
|
|
||||||
{
|
{
|
||||||
for y in runs/shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
|
for y in shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
|
||||||
echo "########## $y | FOLD=0 baseline (record) ##########"
|
echo "########## $y | FOLD=0 baseline (record) ##########"
|
||||||
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|
||||||
|| { echo "!!! FAILED fold=0: $y"; continue; }
|
|| { echo "!!! FAILED fold=0: $y"; continue; }
|
||||||
|
|||||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Reference in new issue
Block a user