forked from zhoulin/zl_bench
Aharrypotter
8bbea472bb
A frozen 32-workload campaign that measures how FlagTree's RemoveLayoutConversions pass behaves on a Moore Threads S5000, through FlagGems public dispatch and this repository's existing plugins. The manifest is the contract: source commits, the single physical device, the three screen cells off/mask0/candidate, the native-event timer, and per-workload dtypes, timeouts and canary membership. prepare validates it against ops/inventory_main.csv and freezes an immutable task matrix; the runner executes the matrix serially, and the worker drives one operator cell through pytest with _ir_meta_plugin, _autotune_record_plugin and the rest. Three properties are worth calling out because they decide whether a verdict means anything. Convert topology is scoped to the measured operator. A benchmark call builds its inputs once and then runs the operator many times under the timer, so counting ttg.convert_layout over every executed kernel credits the candidate with converts removed from input construction that no timer can observe. Each dumped TTGIR file is joined to its launch count from index.tsv and only variants launched more than measured_launch_floor contribute; the unscoped total and the excluded files stay in the receipt, and a cell that cannot be joined to its index is classified X rather than trusted. Correctness is scoped to the variant a workload measures. Running a whole test file lets an unrelated variant decide the verdict, so a workload may pin node ids or a -k selector, never both, and any narrowing must state its reason. Targets are resolved before the campaign starts. The inventory can lag the pinned FlagGems commit, and such a task previously failed only inside the worker's preflight, which exited silently and left an empty log. The runner now resolves every benchmark and correctness target against the real tree, a workload may name a target the inventory does not know about if it says why, and each preflight check names itself on failure.