forked from zhoulin/zl_bench
bench: report when a foreign vendor's FlagGems implementation gets loaded
FlagGems keeps per-vendor implementations under
flag_gems/runtime/backend/_<vendor>/ops/, and benchmarks commonly gate on
whether one imports:
try:
from flag_gems.runtime.backend._nvidia.hopper.ops.mm import router_gemm
ROUTER_GEMM_AVAILABLE = True
except Exception:
ROUTER_GEMM_AVAILABLE = False
An import only proves the .py is on disk. It touches no device, so it succeeds
on every backend and the gate never fires. test_router_gemm.py therefore ran
the NVIDIA Hopper kernel on an S5000: its autotune and heuristics are keyed to
nvidia, supply no BLOCK_M/N/K/SPLIT_K on musa, and the run died deep inside the
JIT with "dynamic_func() missing 4 required positional arguments" — an error
that points nowhere near the cause. Meanwhile _mthreads/ops/mm.py holds a
working router_gemm the benchmark never imports. Gating on vendor_name first,
the way the Cambricon branch in the same file already does, makes it run: 10
shapes, 10 kernels, cvt 10 -> 2.
The plugin now scans sys.modules at session end and prints
FOREIGN_VENDOR_MODULE with the vendors it found. It reports rather than
refuses: loading another vendor's module is not proof of a problem, but when a
result looks wrong it is the first line to check. Vendor roots are compared
segment-wise, not by prefix, so a vendor whose name extends ours is not
mistaken for our own.
README gains the ordered checklist for an operator that will not run, with the
real instance behind each entry.
This commit is contained in:
1 parent
2e456bdc74
commit
4ac2a9acd8
3 files changed
+140
No files matched your search
@@ -186,6 +186,58 @@ tuner_cache direct_repeat`),由 `python batch_summary.py --caliber <run_dir>
|
||||
报告,不混进 dtype 那条解释里。`elapsed_before/after` 也进表:两步流程的全部依据
|
||||
就是"一对 0.8 分钟对六轮 13~44 分钟"这个比例,那这一步该记下自己那一半。
|
||||
|
||||
## 算子跑不起来时的排查清单
|
||||
|
||||
`NO_ROWS` / `NO_TRITON` 说的是"这一格没测出东西",不是"开关无作用"。按下面的顺序查,
|
||||
每一条都踩过:
|
||||
|
||||
| # | 症状 | 查什么 | 实例 |
|
||||
|---|---|---|---|
|
||||
| 1 | `PREFLIGHT_FAILED check=interpreter imports triton + flag_gems` | 用的是哪个 python。triton 通常来自 PYTHONPATH 指向的编译器构建 | 一次噪声标定因此废掉 12 格 + 160 次负载 run |
|
||||
| 2 | `PREFLIGHT_FAILED check=benchmark file exists` | `OPS` 元素是 `OP:OP_FILE`,写反就找不到文件 | `test_perf_sort` 在 `test_sort.py` 里,要写 `perf_sort:sort` |
|
||||
| 3 | `ValueError: Given dtype(s) '...' can't be supported by this op` | 该算子只吃特定 dtype,别全局写死 `DTYPES` | `fft` 只吃 complex64,被一刀切的 fp16 判成"跑不出行",而它是 cvt 消除量最大的算子 |
|
||||
| 4 | `dynamic_func() missing ... BLOCK_M, BLOCK_N ...` 之类 JIT 深处的报错 | **benchmark 是不是导入了别家厂商的实现**(见下) | `router_gemm` 拿 NVIDIA Hopper 的 kernel 跑在 MUSA 上 |
|
||||
| 5 | 跑出行但 `kernels=0` | 该算子在本后端没走 Triton,走的是 aten 或厂商库 | — |
|
||||
|
||||
### 第 4 条:import 成功不等于这个后端支持
|
||||
|
||||
FlagGems 的厂商实现放在 `flag_gems/runtime/backend/_<vendor>/ops/`。benchmark 里常见
|
||||
这种可用性判据:
|
||||
|
||||
```python
|
||||
try:
|
||||
from flag_gems.runtime.backend._nvidia.hopper.ops.mm import router_gemm
|
||||
ROUTER_GEMM_AVAILABLE = True # ← import 成功就置 True
|
||||
except Exception:
|
||||
ROUTER_GEMM_AVAILABLE = False
|
||||
```
|
||||
|
||||
**`import` 只证明那个 `.py` 在磁盘上,它不碰设备,所以在任何后端上都成功。** gate 于是
|
||||
形同虚设:MUSA 上照样置 True,benchmark 拿 NVIDIA Hopper 的 kernel 跑在摩尔线程卡上,
|
||||
最后死在 JIT 深处的 `dynamic_func() missing BLOCK_M, BLOCK_N, BLOCK_K, SPLIT_K`——
|
||||
**那个报错完全指不向真正的原因**。而 `_mthreads/ops/mm.py` 下明明有一份能用的实现,
|
||||
benchmark 从头到尾没导入过。
|
||||
|
||||
正确写法是先判 vendor(同一个文件里 Cambricon 那支就是对的):
|
||||
|
||||
```python
|
||||
if base.vendor_name == "nvidia":
|
||||
try:
|
||||
from flag_gems.runtime.backend._nvidia.hopper.ops.mm import router_gemm
|
||||
ROUTER_GEMM_AVAILABLE = True
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
`_device_setup_plugin` 会在 session 结束时扫 `sys.modules`,载入过别家厂商实现时打:
|
||||
|
||||
```
|
||||
FOREIGN_VENDOR_MODULE active=mthreads loaded=nvidia
|
||||
```
|
||||
|
||||
**它不拦,只报**——载入别家模块本身不一定有问题(有些是共享工具),但一个算子结果可疑
|
||||
时,这行是第一个该看的地方。
|
||||
|
||||
## A/B 对比测试标准流程
|
||||
|
||||
```bash
|
||||
|
||||
Reference in new issue
Block a user