zhoulin/zl_bench
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: zhoulin/zl_bench:26b071c6e119ce11f1418a064d4ebf599133aab7
Branches Tags
zhoulin/zl_bench:main
...
compare: zhoulin/zl_bench:c8a42b66ee4533354dcffdf57db3c0c8024360c3
Branches Tags
zhoulin/zl_bench:main

2 Commits
26b071c6e1 ... c8a42b66ee

Author SHA1 Message Date
zhoulin c8a42b66ee Move trace shape sets to top-level shape/ and frame ab_fold_test.sh as A/B reference example 2026-07-20 16:53:14 +00:00
zhoulin 00902e6c67 Default USE_FLAGTUNE to 0 and add fold_scale A/B script with trace shapes 
Plain autotune is the common case (fast, same-config A/B); FlagTune's
exhaustive first-run search is now opt-in via USE_FLAGTUNE=1.
ab_fold_test.sh automates the FLAGGEMS_MXFP4_FOLDSCALE=0/1 record-replay
comparison over the four DeepSeek-V4-Flash trace shape sets, which are
now tracked under runs/shape/.
 2026-07-20 16:50:59 +00:00
12 changed files with 503 additions and 6 deletions
Expand all files Collapse all files
.gitignore
+1 -1
View File
@@ -1,2 +1,2 @@
runs/
__pycache__/
runs/
README.md
+14 -3
View File
@@ -28,7 +28,7 @@ softmax:
跑之前先 `nvidia-smi` 确认目标卡空闲——共享机器上别的任务会把 baseline 和被测两边一起等比拖慢，出一份看似自洽实则作废的数据。多卡机器上用 `CUDA_VISIBLE_DEVICES=<空闲卡号>` 明确选卡。
脚本内固定了 `--level core --mode kernel`；`USE_FLAGTUNE` 默认 `1`（FlagTune 扩展调优空间，见"为什么需要 replay"一节），可用环境变量覆盖（`USE_FLAGTUNE=0 bash run_pytest.sh` 走普通 autotune，跳过全量搜索，快速验证时用）；需要改动其它口径时直接编辑 `run_pytest.sh` 中的 pytest 命令行。
脚本内固定了 `--level core --mode kernel`；`USE_FLAGTUNE` 默认 `0`（普通 autotune，快速出结果），需要 FlagTune 扩展调优空间时用 `USE_FLAGTUNE=1 bash run_pytest.sh`（首跑全量搜索、慢，见"为什么需要 replay"一节）；需要改动其它口径时直接编辑 `run_pytest.sh` 中的 pytest 命令行。
### 环境变量一览
@@ -39,7 +39,7 @@ softmax:
| `SHAPE_FILE` | 空（用脚本内置 yaml） | shape yaml 路径 |
| `FLAGGEMS_DIR` | `/workspace/FlagGems-dev` | FlagGems 仓库路径 |
| `REPLAY_FROM` | 空（record 模式） | 指向某次历史 run 目录，replay 其 autotune 选择（见下） |
| `USE_FLAGTUNE` | `1` | `1` 用 FlagTune 扩展调优空间（首跑全量搜索、慢）；`0` 走普通 autotune，跳过搜索、快速验证 |
| `USE_FLAGTUNE` | `0` | `0` 走普通 autotune（跳过搜索、快速验证）；`1` 用 FlagTune 扩展调优空间（首跑全量搜索、慢） |
| `FLAGGEMS_PERF_COLOR` | 空（按 tty 自动判断） | `always`/`never` 强制开/关终端颜色；`run.log` 始终为去色纯文本 |
## 输出目录结构
@@ -85,6 +85,17 @@ A/B 两侧的 `USE_FLAGTUNE` 必须取同值：该开关决定调优空间，两
replay 的兜底行为：B 侧遇到记录中没有的 key、或记录的 config 在新编译器下编译失败时，自动回退到现场 autotune 并在 `run.log` 打 `AUTOTUNE_REPLAY_FALLBACK reason=...` 标记——出现该标记的测量点不再满足"同 config"前提，解读时注意。另外 replay 模式的 run 目录不产生 `autotune_records/`，后续 run 的 `REPLAY_FROM` 应始终指向最初 record 的那次 A 侧目录，不要链式指向 replay 产物。
### ab_fold_test.sh：A/B 测试参考示例
`ab_fold_test.sh` 是上述 A/B 流程的现成参考：以 FlagGems `FLAGGEMS_MXFP4_FOLDSCALE`（fold_scale 优化）开关为对比对象，对 4 个 DeepSeek-V4-Flash 真实 trace shape 集（`shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml`，随仓库提供）各跑一对 FOLD=0（基线，record）→ FOLD=1（replay 同 config），共 8 轮：
```bash
bash ab_fold_test.sh
# 汇总 log：runs/ab_fold_<时间戳>.log（已去色）；快速对账：grep -E '^#####|FAILED' <log>
```
要对比其它开关 / 编译器改动 / 算子，套用该脚本改循环变量与环境变量即可，A/B 口径（同 config replay、cudagraph 计时）无需改动。
### cudagraph 计时的 warmup、回退与精度
**capture 前的显式 warmup**：`do_bench_cudagraph` 自带的内部预热只有 5 次迭代，对首跑要触发 autotune 编译（尤其含 FlagTune 扩展空间）、libtuner 选择、lazy JIT 的算子远远不够——这些一次性开销若漏进被捕获的 graph 或第一个计时迭代，测出的 latency 会 run-to-run 抖动（M=1 多 kernel 路径最明显，实测首轮可低到稳态的 ~1/3）。`_cudagraph_plugin` 因此在 capture 前显式预热：先跑一次并丢弃（吸收 autotune/JIT 编译），再按调用方传入的 warmup 时间预算（`Config.warm_up`）循环稳态预热，然后才 capture。预热次数按稳态单次耗时换算，并夹在 5–200 次之间——亚毫秒 kernel 的实际预热时长因此低于名义预算（1000ms），实测足够；若换新算子仍见首轮抖动，优先调大 `_warmup_before_capture` 里的次数上限。这样首轮即稳态、多轮一致（实测同一 M=1 shape 两轮 speedup 差 <0.1%）。
@@ -134,7 +145,7 @@ libtuner 的持久缓存何时失效：FlagGems kernel 源码改动、tune_confi
**Q: 第一次跑某算子特别慢？**
两个来源：（1）默认 `USE_FLAGTUNE=1`，首跑要在 FlagTune 扩展空间做全量搜索（fused_marlin_moe_mxfp4 单进程可达数分钟甚至十几分钟，期间 GPU 满载、`run.log` 停在测试名不动属正常，不是卡死）；（2）libtuner 算子首跑的全量 sweep（mm 约 50 分钟）。winner 持久化到 `~/.flaggems/config_cache/` 后同 shape 秒级命中（但该缓存会因源码/开关/GPU 变化失效，失效后又需重搜）。快速验证（不追求"各自最优 config"、只看功能/稳定性）时直接 `USE_FLAGTUNE=0 bash run_pytest.sh` 走普通 autotune，单 shape 通常 10 秒级出结果。
两个来源：（1）开了 `USE_FLAGTUNE=1` 时，首跑要在 FlagTune 扩展空间做全量搜索（fused_marlin_moe_mxfp4 单进程可达数分钟甚至十几分钟，期间 GPU 满载、`run.log` 停在测试名不动属正常，不是卡死）；（2）libtuner 算子首跑的全量 sweep（mm 约 50 分钟）。winner 持久化到 `~/.flaggems/config_cache/` 后同 shape 秒级命中（但该缓存会因源码/开关/GPU 变化失效，失效后又需重搜）。默认 `USE_FLAGTUNE=0` 走普通 autotune，单 shape 通常 10 秒级出结果。
**Q: `--warmup/--iter` 要设吗？**
ab_fold_test.sh
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/bin/bash
# A/B 测试参考示例：FLAGGEMS_MXFP4_FOLDSCALE（fold_scale）开关验证
# （4 个 DeepSeek-V4-Flash trace yaml × FOLD=0/1 共 8 轮）。
# 口径：USE_FLAGTUNE=0；FOLD=0（基线）先跑并 record autotune config，FOLD=1 用
# REPLAY_FROM 回放同一套 config（两侧同配置比 wall-time）。
# 对比其它开关/算子时，改循环变量与环境变量即可复用同一口径。
# 全部输出汇总到 runs/ab_fold_<时间戳>.log；各轮 runs/<op>_<时间戳>/ 产物照常存档。
# 用法：bash ab_fold_test.sh
set -uo pipefail
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# 终端颜色：下方 tee 管道会让 run_pytest.sh 侧检测不到 tty，在这里先判断并下传
# （与 run_pytest.sh 自身的做法一致）；总 log 最后统一去色。
if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then
export FLAGGEMS_PERF_COLOR=always
fi
LOG="runs/ab_fold_$(date +%Y%m%d-%H%M%S).log"
{
for y in shape/DeepSeek-V4-Flash-p{1024,4096,32768,65536}d1024.yaml; do
echo "########## $y | FOLD=0 baseline (record) ##########"
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=0 bash run_pytest.sh \
|| { echo "!!! FAILED fold=0: $y"; continue; }
REC=$(ls -dt runs/fused_marlin_moe_mxfp4_* | head -1)
echo "########## $y | FOLD=1 (replay $REC) ##########"
SHAPE_FILE="$y" USE_FLAGTUNE=0 FLAGGEMS_MXFP4_FOLDSCALE=1 REPLAY_FROM="$REC" bash run_pytest.sh \
|| echo "!!! FAILED fold=1: $y"
done
echo "########## ALL DONE ##########"
} 2>&1 | tee "$LOG"
# 总 log 去掉 ANSI 转义，保证 grep/diff 面对纯文本（终端输出保留颜色）
sed -i -E $'s/\x1b\\[[0-9;]*[A-Za-z]//g' "$LOG"
echo ">>> 总 log: $(pwd)/$LOG"
echo ">>> 快速对账: grep -E '^#####|FAILED' $LOG"
run_pytest.sh
+2 -2
View File
@@ -46,8 +46,8 @@ fused_marlin_moe_mxfp4:
shape_desc: "num_tokens, num_experts, hidden_size, intermediate_size, topk"
YAML
# 调优空间：1=FlagTune 扩展空间（首跑全量搜索、慢）；0=普通 autotune（快速验证）
USE_FLAGTUNE="${USE_FLAGTUNE:-1}"
# 调优空间：0=普通 autotune（默认，快速验证）；1=FlagTune 扩展空间（首跑全量搜索、慢）
USE_FLAGTUNE="${USE_FLAGTUNE:-0}"
# autotune record/replay（见 _autotune_record_plugin.py）：
# - 空：record 模式，本次选中的 config 记录到 $OUT_DIR/autotune_records/<op>.json；
shape/DeepSeek-V4-Flash-p1024d1024.txt
+55
View File
@@ -0,0 +1,55 @@
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 1376
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [496, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [512, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [1028, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 215
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [3805, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 215
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [1, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [104, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [112, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [120, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [128, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [136, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [144, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [152, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [160, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [168, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [176, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [184, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [192, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [200, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [208, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [216, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [224, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [232, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [24, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [240, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [248, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [256, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [272, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [288, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [304, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [32, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [320, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [336, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [352, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [368, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [4, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [40, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [400, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [416, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [432, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [448, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [464, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [48, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [480, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [56, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [64, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [72, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [8, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [80, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [88, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [96, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2048, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 43
shape/DeepSeek-V4-Flash-p1024d1024.yaml
+59
View File
@@ -0,0 +1,59 @@
fused_marlin_moe_mxfp4:
# 来源: DeepSeek-V4-Flash-p1024d1024.txt（55 个去重 shape，按 num_tokens 升序；行尾注释为 trace 中的调用次数）
shapes:
- [1, 256, 4096, 256, 6] # count: 172
- [2, 256, 4096, 256, 6] # count: 172
- [4, 256, 4096, 256, 6] # count: 172
- [8, 256, 4096, 256, 6] # count: 172
- [16, 256, 4096, 256, 6] # count: 172
- [24, 256, 4096, 256, 6] # count: 172
- [32, 256, 4096, 256, 6] # count: 172
- [40, 256, 4096, 256, 6] # count: 172
- [48, 256, 4096, 256, 6] # count: 172
- [56, 256, 4096, 256, 6] # count: 172
- [64, 256, 4096, 256, 6] # count: 172
- [72, 256, 4096, 256, 6] # count: 172
- [80, 256, 4096, 256, 6] # count: 172
- [88, 256, 4096, 256, 6] # count: 172
- [96, 256, 4096, 256, 6] # count: 172
- [104, 256, 4096, 256, 6] # count: 172
- [112, 256, 4096, 256, 6] # count: 172
- [120, 256, 4096, 256, 6] # count: 172
- [128, 256, 4096, 256, 6] # count: 172
- [136, 256, 4096, 256, 6] # count: 172
- [144, 256, 4096, 256, 6] # count: 172
- [152, 256, 4096, 256, 6] # count: 172
- [160, 256, 4096, 256, 6] # count: 172
- [168, 256, 4096, 256, 6] # count: 172
- [176, 256, 4096, 256, 6] # count: 172
- [184, 256, 4096, 256, 6] # count: 172
- [192, 256, 4096, 256, 6] # count: 172
- [200, 256, 4096, 256, 6] # count: 172
- [208, 256, 4096, 256, 6] # count: 172
- [216, 256, 4096, 256, 6] # count: 172
- [224, 256, 4096, 256, 6] # count: 172
- [232, 256, 4096, 256, 6] # count: 172
- [240, 256, 4096, 256, 6] # count: 172
- [248, 256, 4096, 256, 6] # count: 172
- [256, 256, 4096, 256, 6] # count: 172
- [272, 256, 4096, 256, 6] # count: 172
- [288, 256, 4096, 256, 6] # count: 172
- [304, 256, 4096, 256, 6] # count: 172
- [320, 256, 4096, 256, 6] # count: 172
- [336, 256, 4096, 256, 6] # count: 172
- [352, 256, 4096, 256, 6] # count: 172
- [368, 256, 4096, 256, 6] # count: 172
- [384, 256, 4096, 256, 6] # count: 172
- [400, 256, 4096, 256, 6] # count: 172
- [416, 256, 4096, 256, 6] # count: 172
- [432, 256, 4096, 256, 6] # count: 172
- [448, 256, 4096, 256, 6] # count: 172
- [464, 256, 4096, 256, 6] # count: 172
- [480, 256, 4096, 256, 6] # count: 172
- [496, 256, 4096, 256, 6] # count: 344
- [512, 256, 4096, 256, 6] # count: 344
- [1028, 256, 4096, 256, 6] # count: 215
- [2048, 256, 4096, 256, 6] # count: 43
- [3805, 256, 4096, 256, 6] # count: 215
- [16384, 256, 4096, 256, 6] # count: 1376
shape_desc: "num_tokens, num_experts, hidden_size, intermediate_size, topk"
shape/DeepSeek-V4-Flash-p32768d1024.txt
+53
View File
@@ -0,0 +1,53 @@
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 946
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [496, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [512, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [1, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [104, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [112, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [120, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [128, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [136, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [144, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [152, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [160, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [168, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [176, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [184, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [192, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [200, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [208, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [216, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [224, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [232, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [24, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [240, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [248, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [256, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [272, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [288, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [304, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [32, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [320, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [336, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [352, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [368, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [4, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [40, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [400, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [416, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [432, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [448, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [464, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [48, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [480, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [56, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [64, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [72, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [8, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [80, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [88, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [96, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2048, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 43
shape/DeepSeek-V4-Flash-p32768d1024.yaml
+57
View File
@@ -0,0 +1,57 @@
fused_marlin_moe_mxfp4:
# 来源: DeepSeek-V4-Flash-p32768d1024.txt（53 个去重 shape，按 num_tokens 升序；行尾注释为 trace 中的调用次数）
shapes:
- [1, 256, 4096, 256, 6] # count: 172
- [2, 256, 4096, 256, 6] # count: 172
- [4, 256, 4096, 256, 6] # count: 172
- [8, 256, 4096, 256, 6] # count: 172
- [16, 256, 4096, 256, 6] # count: 172
- [24, 256, 4096, 256, 6] # count: 172
- [32, 256, 4096, 256, 6] # count: 172
- [40, 256, 4096, 256, 6] # count: 172
- [48, 256, 4096, 256, 6] # count: 172
- [56, 256, 4096, 256, 6] # count: 172
- [64, 256, 4096, 256, 6] # count: 172
- [72, 256, 4096, 256, 6] # count: 172
- [80, 256, 4096, 256, 6] # count: 172
- [88, 256, 4096, 256, 6] # count: 172
- [96, 256, 4096, 256, 6] # count: 172
- [104, 256, 4096, 256, 6] # count: 172
- [112, 256, 4096, 256, 6] # count: 172
- [120, 256, 4096, 256, 6] # count: 172
- [128, 256, 4096, 256, 6] # count: 172
- [136, 256, 4096, 256, 6] # count: 172
- [144, 256, 4096, 256, 6] # count: 172
- [152, 256, 4096, 256, 6] # count: 172
- [160, 256, 4096, 256, 6] # count: 172
- [168, 256, 4096, 256, 6] # count: 172
- [176, 256, 4096, 256, 6] # count: 172
- [184, 256, 4096, 256, 6] # count: 172
- [192, 256, 4096, 256, 6] # count: 172
- [200, 256, 4096, 256, 6] # count: 172
- [208, 256, 4096, 256, 6] # count: 172
- [216, 256, 4096, 256, 6] # count: 172
- [224, 256, 4096, 256, 6] # count: 172
- [232, 256, 4096, 256, 6] # count: 172
- [240, 256, 4096, 256, 6] # count: 172
- [248, 256, 4096, 256, 6] # count: 172
- [256, 256, 4096, 256, 6] # count: 172
- [272, 256, 4096, 256, 6] # count: 172
- [288, 256, 4096, 256, 6] # count: 172
- [304, 256, 4096, 256, 6] # count: 172
- [320, 256, 4096, 256, 6] # count: 172
- [336, 256, 4096, 256, 6] # count: 172
- [352, 256, 4096, 256, 6] # count: 172
- [368, 256, 4096, 256, 6] # count: 172
- [384, 256, 4096, 256, 6] # count: 172
- [400, 256, 4096, 256, 6] # count: 172
- [416, 256, 4096, 256, 6] # count: 172
- [432, 256, 4096, 256, 6] # count: 172
- [448, 256, 4096, 256, 6] # count: 172
- [464, 256, 4096, 256, 6] # count: 172
- [480, 256, 4096, 256, 6] # count: 172
- [496, 256, 4096, 256, 6] # count: 344
- [512, 256, 4096, 256, 6] # count: 344
- [2048, 256, 4096, 256, 6] # count: 43
- [16384, 256, 4096, 256, 6] # count: 946
shape_desc: "num_tokens, num_experts, hidden_size, intermediate_size, topk"
shape/DeepSeek-V4-Flash-p4096d1024.txt
+55
View File
@@ -0,0 +1,55 @@
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 3311
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [496, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [512, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [13021, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 215
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [4100, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 215
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [1, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [104, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [112, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [120, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [128, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [136, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [144, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [152, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [160, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [168, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [176, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [184, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [192, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [200, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [208, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [216, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [224, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [232, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [24, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [240, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [248, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [256, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [272, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [288, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [304, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [32, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [320, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [336, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [352, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [368, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [4, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [40, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [400, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [416, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [432, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [448, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [464, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [48, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [480, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [56, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [64, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [72, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [8, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [80, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [88, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [96, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2048, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 43
shape/DeepSeek-V4-Flash-p4096d1024.yaml
+59
View File
@@ -0,0 +1,59 @@
fused_marlin_moe_mxfp4:
# 来源: DeepSeek-V4-Flash-p4096d1024.txt（55 个去重 shape，按 num_tokens 升序；行尾注释为 trace 中的调用次数）
shapes:
- [1, 256, 4096, 256, 6] # count: 172
- [2, 256, 4096, 256, 6] # count: 172
- [4, 256, 4096, 256, 6] # count: 172
- [8, 256, 4096, 256, 6] # count: 172
- [16, 256, 4096, 256, 6] # count: 172
- [24, 256, 4096, 256, 6] # count: 172
- [32, 256, 4096, 256, 6] # count: 172
- [40, 256, 4096, 256, 6] # count: 172
- [48, 256, 4096, 256, 6] # count: 172
- [56, 256, 4096, 256, 6] # count: 172
- [64, 256, 4096, 256, 6] # count: 172
- [72, 256, 4096, 256, 6] # count: 172
- [80, 256, 4096, 256, 6] # count: 172
- [88, 256, 4096, 256, 6] # count: 172
- [96, 256, 4096, 256, 6] # count: 172
- [104, 256, 4096, 256, 6] # count: 172
- [112, 256, 4096, 256, 6] # count: 172
- [120, 256, 4096, 256, 6] # count: 172
- [128, 256, 4096, 256, 6] # count: 172
- [136, 256, 4096, 256, 6] # count: 172
- [144, 256, 4096, 256, 6] # count: 172
- [152, 256, 4096, 256, 6] # count: 172
- [160, 256, 4096, 256, 6] # count: 172
- [168, 256, 4096, 256, 6] # count: 172
- [176, 256, 4096, 256, 6] # count: 172
- [184, 256, 4096, 256, 6] # count: 172
- [192, 256, 4096, 256, 6] # count: 172
- [200, 256, 4096, 256, 6] # count: 172
- [208, 256, 4096, 256, 6] # count: 172
- [216, 256, 4096, 256, 6] # count: 172
- [224, 256, 4096, 256, 6] # count: 172
- [232, 256, 4096, 256, 6] # count: 172
- [240, 256, 4096, 256, 6] # count: 172
- [248, 256, 4096, 256, 6] # count: 172
- [256, 256, 4096, 256, 6] # count: 172
- [272, 256, 4096, 256, 6] # count: 172
- [288, 256, 4096, 256, 6] # count: 172
- [304, 256, 4096, 256, 6] # count: 172
- [320, 256, 4096, 256, 6] # count: 172
- [336, 256, 4096, 256, 6] # count: 172
- [352, 256, 4096, 256, 6] # count: 172
- [368, 256, 4096, 256, 6] # count: 172
- [384, 256, 4096, 256, 6] # count: 172
- [400, 256, 4096, 256, 6] # count: 172
- [416, 256, 4096, 256, 6] # count: 172
- [432, 256, 4096, 256, 6] # count: 172
- [448, 256, 4096, 256, 6] # count: 172
- [464, 256, 4096, 256, 6] # count: 172
- [480, 256, 4096, 256, 6] # count: 172
- [496, 256, 4096, 256, 6] # count: 344
- [512, 256, 4096, 256, 6] # count: 344
- [2048, 256, 4096, 256, 6] # count: 43
- [4100, 256, 4096, 256, 6] # count: 215
- [13021, 256, 4096, 256, 6] # count: 215
- [16384, 256, 4096, 256, 6] # count: 3311
shape_desc: "num_tokens, num_experts, hidden_size, intermediate_size, topk"
shape/DeepSeek-V4-Flash-p65536d1024.txt
+53
View File
@@ -0,0 +1,53 @@
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 946
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [496, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [512, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 344
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [1, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [104, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [112, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [120, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [128, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [136, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [144, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [152, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [16, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [160, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [168, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [176, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [184, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [192, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [200, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [208, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [216, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [224, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [232, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [24, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [240, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [248, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [256, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [272, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [288, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [304, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [32, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [320, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [336, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [352, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [368, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [384, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [4, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [40, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [400, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [416, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [432, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [448, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [464, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [48, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [480, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [56, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [64, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [72, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [8, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [80, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [88, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [96, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 172
flag_gems.ops.fused_marlin_moe.fused_marlin_moe, [shape info]: [2048, 256, 4096, 256, 6](num_tokens, num_experts, hidden_size, intermediate_size, topk), [count]: 43
shape/DeepSeek-V4-Flash-p65536d1024.yaml
+57
View File
@@ -0,0 +1,57 @@
fused_marlin_moe_mxfp4:
# 来源: DeepSeek-V4-Flash-p65536d1024.txt（53 个去重 shape，按 num_tokens 升序；行尾注释为 trace 中的调用次数）
shapes:
- [1, 256, 4096, 256, 6] # count: 172
- [2, 256, 4096, 256, 6] # count: 172
- [4, 256, 4096, 256, 6] # count: 172
- [8, 256, 4096, 256, 6] # count: 172
- [16, 256, 4096, 256, 6] # count: 172
- [24, 256, 4096, 256, 6] # count: 172
- [32, 256, 4096, 256, 6] # count: 172
- [40, 256, 4096, 256, 6] # count: 172
- [48, 256, 4096, 256, 6] # count: 172
- [56, 256, 4096, 256, 6] # count: 172
- [64, 256, 4096, 256, 6] # count: 172
- [72, 256, 4096, 256, 6] # count: 172
- [80, 256, 4096, 256, 6] # count: 172
- [88, 256, 4096, 256, 6] # count: 172
- [96, 256, 4096, 256, 6] # count: 172
- [104, 256, 4096, 256, 6] # count: 172
- [112, 256, 4096, 256, 6] # count: 172
- [120, 256, 4096, 256, 6] # count: 172
- [128, 256, 4096, 256, 6] # count: 172
- [136, 256, 4096, 256, 6] # count: 172
- [144, 256, 4096, 256, 6] # count: 172
- [152, 256, 4096, 256, 6] # count: 172
- [160, 256, 4096, 256, 6] # count: 172
- [168, 256, 4096, 256, 6] # count: 172
- [176, 256, 4096, 256, 6] # count: 172
- [184, 256, 4096, 256, 6] # count: 172
- [192, 256, 4096, 256, 6] # count: 172
- [200, 256, 4096, 256, 6] # count: 172
- [208, 256, 4096, 256, 6] # count: 172
- [216, 256, 4096, 256, 6] # count: 172
- [224, 256, 4096, 256, 6] # count: 172
- [232, 256, 4096, 256, 6] # count: 172
- [240, 256, 4096, 256, 6] # count: 172
- [248, 256, 4096, 256, 6] # count: 172
- [256, 256, 4096, 256, 6] # count: 172
- [272, 256, 4096, 256, 6] # count: 172
- [288, 256, 4096, 256, 6] # count: 172
- [304, 256, 4096, 256, 6] # count: 172
- [320, 256, 4096, 256, 6] # count: 172
- [336, 256, 4096, 256, 6] # count: 172
- [352, 256, 4096, 256, 6] # count: 172
- [368, 256, 4096, 256, 6] # count: 172
- [384, 256, 4096, 256, 6] # count: 172
- [400, 256, 4096, 256, 6] # count: 172
- [416, 256, 4096, 256, 6] # count: 172
- [432, 256, 4096, 256, 6] # count: 172
- [448, 256, 4096, 256, 6] # count: 172
- [464, 256, 4096, 256, 6] # count: 172
- [480, 256, 4096, 256, 6] # count: 172
- [496, 256, 4096, 256, 6] # count: 344
- [512, 256, 4096, 256, 6] # count: 344
- [2048, 256, 4096, 256, 6] # count: 43
- [16384, 256, 4096, 256, 6] # count: 946
shape_desc: "num_tokens, num_experts, hidden_size, intermediate_size, topk"