3 Commits

Author SHA1 Message Date
zhoulin da22885645 Add multi-op batch screening layer; fix cudagraph fallback regressions
run_batch.py drives run_pytest.sh per operator across GPUs (stable-hash
sharding, per-op subprocess isolation, process-group timeouts, retry with
deterministic-failure cutoff, two-level dtype fallback, .complete resume,
per-op REPLAY_FROM). batch_summary.py aggregates run.log tables into
summary.csv. ops/ holds the curated assets: dual-repo inventories rebuilt
via AST scan + pytest collect verification, shape sets migrated from the
old regression harness and merged with upstream core_shapes class-name
keys (upstream's set_shapes falls back op_name -> MRO class name ->
1-D DEFAULT_SHAPES, so replacing the shape file without class keys
crashes the BLAS family), and a dismiss list where all 76 entries carry
verified reasons. Validated end to end: 1036-op full screen with zero
failures.

Also fix two cudagraph plugin regressions: newer torch appends "enable
device-side assertions" to every CUDA error, so the loose fatal-error
marker disabled the documented do_bench fallback entirely; and an aborted
graph capture can leave the default CUDA RNG generator stuck in capturing
state, poisoning every later torch.randn - captures now run under a
throwaway RNG state. run_pytest.sh gains an optional DTYPES passthrough.
2026-08-12 19:04:09 +00:00
zhoulin 26b071c6e1 Stabilize first-run cudagraph timing and colorize terminal output
- _cudagraph_plugin: warm up autotune/JIT explicitly before graph capture
  (internal 5-iter warmup is too short), tag fallback markers with the
  failing phase; first-run latency no longer jitters run-to-run.
- _device_guard_plugin (new): set GEMS_VENDOR via torch probe before
  importing flag_gems, avoiding its timeout-less nvidia-smi subprocess
  probe that can hang import in fork-broken environments.
- _pretty_report_plugin (new) + _term_style (new): fold inputs identical
  across all result rows into a legend line, color status/plugin
  tags/markers on the live terminal; run.log is ANSI-stripped and keeps
  upstream SUCCESS/column wording for grep compatibility.
- run_pytest.sh: make USE_FLAGTUNE overridable, group all knobs into a
  config section with Chinese comments, add start/end banners.
- README: document the warmup semantics, new plugins/env vars, and the
  A/B rule that both sides must use the same USE_FLAGTUNE.
2026-07-19 19:21:13 +00:00
zhoulin 8ace0d790c Add FlagGems single-op perf benchmark harness
pytest plugin set + driver script for compiler A/B perf comparison:
- reproducible runs: fixed seed, yaml-driven shapes, autotune record/replay
- per-shape ttgir dump of actually-used variants with readable naming
- cudagraph-based timing with documented fallback semantics

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 03:52:07 +00:00