forked from zhoulin/zl_bench
bench: partition operators across cards by measured cost, and rescore in place
Two pieces the parallel campaign needs. partition.py assigns screened operators to cards longest-first. run_batch.py's existing hash partition ignores weight: on this batch it puts the critical path at 120 minutes and leaves a card idle, where greedy gets 70. The floor is set by the single heaviest operator — topk alone is 70.6 minutes against an ideal share of 45.4 — so partitioning cannot beat it, only keep the other cards from waiting. Weights come from screen.tsv's elapsed_before + elapsed_after, which is one pair, i.e. 2 cells; N rounds is 2N cells, so the estimate is N times the weight. Getting that wrong reported topk as 11.8 minutes. RESCORE=<root> re-judges an existing screening root against the current fingerprint and rewrites its screen.tsv, keeping the original alongside. The criterion has already changed twice — index.tsv used to be hashed, and the switch's own ttg.rlc-* annotations had to be stripped — and the second change moved 24 operators from 23 IR_CHANGED to 16. Rescoring the artifacts already on disk takes seconds; re-running that screen takes 79 minutes.
This commit is contained in:
1 parent
c6a7f834cb
commit
03fdcbab28
3 files changed
+259
No files matched your search
@@ -39,6 +39,51 @@ set -uo pipefail
|
||||
cd "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "./_ir_fingerprint.sh"
|
||||
|
||||
# RESCORE=<已有的 root>:判据改了之后,用当前的 ir_fingerprint 对盘上已有的 ttgir
|
||||
# 重新打分并改写该 root 的 screen.tsv,不重跑任何测量。
|
||||
#
|
||||
# 需要它是因为判据确实改过:_ir_fingerprint.sh 起初把 index.tsv 也算进哈希,后来又
|
||||
# 发现要抹掉开关自己盖的 ttg.rlc-* 属性。同一批 24 个算子,结论从 23 个 IR_CHANGED
|
||||
# 变成 16 个。重打分是秒级的,重跑那批筛选要 79 分钟。
|
||||
#
|
||||
# 原表存为 screen.tsv.before-rescore,判定有变化的算子逐个打出来。
|
||||
RESCORE="${RESCORE:-}"
|
||||
if [[ -n "$RESCORE" ]]; then
|
||||
[[ -d "$RESCORE" ]] || { echo "!!! 目录不存在: $RESCORE" >&2; exit 2; }
|
||||
TSV="$RESCORE/screen.tsv"
|
||||
[[ -f "$TSV" ]] || { echo "!!! 没有 screen.tsv: $TSV" >&2; exit 2; }
|
||||
cp "$TSV" "$TSV.before-rescore"
|
||||
|
||||
hdr=$(head -1 "$TSV.before-rescore")
|
||||
# 列名 -> 序号,不写死列序
|
||||
idx() { echo "$hdr" | tr '\t' '\n' | grep -nxF "$1" | cut -d: -f1; }
|
||||
c_op=$(idx op); c_rb=$(idx rows_before); c_ra=$(idx rows_after); c_v=$(idx verdict)
|
||||
[[ -n "$c_op" && -n "$c_v" ]] || { echo "!!! screen.tsv 缺 op/verdict 列" >&2; exit 2; }
|
||||
|
||||
printf '%s\n' "$hdr" > "$TSV"
|
||||
changed=0
|
||||
while IFS= read -r line; do
|
||||
op=$(printf '%s' "$line" | cut -f"$c_op")
|
||||
old=$(printf '%s' "$line" | cut -f"$c_v")
|
||||
rb=$(printf '%s' "$line" | cut -f"${c_rb:-0}"); ra=$(printf '%s' "$line" | cut -f"${c_ra:-0}")
|
||||
fa=$(ir_fingerprint "$RESCORE/${op}_before") || fa=""
|
||||
fb=$(ir_fingerprint "$RESCORE/${op}_after") || fb=""
|
||||
if [[ "${rb:-0}" == 0 || "${ra:-0}" == 0 ]]; then v=NO_ROWS
|
||||
elif [[ -z "$fa" || -z "$fb" ]]; then v=NO_TRITON
|
||||
elif [[ "$fa" == "$fb" ]]; then v=NO_IR_DIFF
|
||||
else v=IR_CHANGED; fi
|
||||
printf '%s\n' "$line" | awk -v FS='\t' -v OFS='\t' -v i="$c_v" -v v="$v" '{$i=v; print}' >> "$TSV"
|
||||
if [[ "$old" != "$v" ]]; then
|
||||
printf ' %-22s %-12s -> %s\n' "$op" "$old" "$v" >&2
|
||||
changed=$((changed+1))
|
||||
fi
|
||||
done < <(tail -n +2 "$TSV.before-rescore")
|
||||
|
||||
echo ">>> 重打分完成,$changed 个算子的判定变了;原表存为 $TSV.before-rescore"
|
||||
awk -F'\t' 'NR>1{c[$NF]++} END{for (k in c) printf(" %-12s %d\n", k, c[k])}' "$TSV"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
VENDOR="${VENDOR:-MUSA}"
|
||||
ENHANCE_VAR="FLAGTREE_${VENDOR}_RLC_ENHANCE"
|
||||
|
||||
|
||||
Reference in new issue
Block a user