forked from zhoulin/zl_bench
bench: report when a foreign vendor's FlagGems implementation gets loaded
FlagGems keeps per-vendor implementations under
flag_gems/runtime/backend/_<vendor>/ops/, and benchmarks commonly gate on
whether one imports:
try:
from flag_gems.runtime.backend._nvidia.hopper.ops.mm import router_gemm
ROUTER_GEMM_AVAILABLE = True
except Exception:
ROUTER_GEMM_AVAILABLE = False
An import only proves the .py is on disk. It touches no device, so it succeeds
on every backend and the gate never fires. test_router_gemm.py therefore ran
the NVIDIA Hopper kernel on an S5000: its autotune and heuristics are keyed to
nvidia, supply no BLOCK_M/N/K/SPLIT_K on musa, and the run died deep inside the
JIT with "dynamic_func() missing 4 required positional arguments" — an error
that points nowhere near the cause. Meanwhile _mthreads/ops/mm.py holds a
working router_gemm the benchmark never imports. Gating on vendor_name first,
the way the Cambricon branch in the same file already does, makes it run: 10
shapes, 10 kernels, cvt 10 -> 2.
The plugin now scans sys.modules at session end and prints
FOREIGN_VENDOR_MODULE with the vendors it found. It reports rather than
refuses: loading another vendor's module is not proof of a problem, but when a
result looks wrong it is the first line to check. Vendor roots are compared
segment-wise, not by prefix, so a vendor whose name extends ours is not
mistaken for our own.
README gains the ordered checklist for an operator that will not run, with the
real instance behind each entry.
This commit is contained in:
1 parent
2e456bdc74
commit
4ac2a9acd8
3 files changed
+140
No files matched your search
@@ -67,3 +67,46 @@ def pytest_configure(config):
|
||||
|
||||
|
||||
_setup_at_import()
|
||||
|
||||
# FlagGems 的厂商实现按 flag_gems.runtime.backend._<vendor>.ops.* 组织。benchmark
|
||||
# 常写成 `try: import <某厂商实现>; AVAILABLE = True`——但 import 只证明那个 .py 在
|
||||
# 磁盘上,它不碰设备,所以在任何后端上都会成功。于是 gate 形同虚设,测出来的是别家
|
||||
# 的 kernel 跑在本机上。router_gemm 就是这样:test_router_gemm.py 无条件导入
|
||||
# _nvidia.hopper 版,在 MUSA 上 gate 照过,最后死在 JIT 深处的
|
||||
# "dynamic_func() missing BLOCK_M, BLOCK_N, BLOCK_K, SPLIT_K"——那个报错完全指不
|
||||
# 向真正的原因,而 _mthreads 下明明有一份能用的实现。
|
||||
#
|
||||
# 这里不拦,只报:跑完看一眼载入过谁家的实现。
|
||||
_VENDOR_MOD_PREFIX = "flag_gems.runtime.backend._"
|
||||
|
||||
|
||||
def foreign_vendor_modules(active_vendor: str, modules=None) -> list:
|
||||
"""本次进程里载入过的、不属于当前厂商的 FlagGems 后端模块。"""
|
||||
if modules is None:
|
||||
modules = sys.modules
|
||||
out = set()
|
||||
for name in list(modules):
|
||||
if not name.startswith(_VENDOR_MOD_PREFIX):
|
||||
continue
|
||||
# 只保留厂商根,_nvidia.hopper.ops.mm 与 _nvidia.ops.mm 算同一条。
|
||||
# 按段比而不是按前缀:前缀会让 _<vendor>_something 被误认成自家。
|
||||
root = name[len(_VENDOR_MOD_PREFIX):].split(".")[0]
|
||||
if root != active_vendor:
|
||||
out.add(root)
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def pytest_sessionfinish(session, exitstatus):
|
||||
from _term_style import tag
|
||||
try:
|
||||
vendor = get_backend().flaggems_vendor
|
||||
except Exception:
|
||||
return
|
||||
foreign = foreign_vendor_modules(vendor)
|
||||
if not foreign:
|
||||
return
|
||||
print(f"{tag('[device-setup-plugin]')} FOREIGN_VENDOR_MODULE active={vendor} "
|
||||
f"loaded={','.join(foreign)}", file=sys.stderr, flush=True)
|
||||
print(f"{tag('[device-setup-plugin]')} 本次载入了别家厂商的 FlagGems 实现。"
|
||||
"如果这个算子的结果可疑,先查 benchmark 的可用性判据是不是只做了 import "
|
||||
"而没判 vendor_name", file=sys.stderr, flush=True)
|
||||
Reference in new issue
Block a user