/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126538, complete Morton128 layout and kernels; our https://duck.ac/submission/126515 and its inherited sources below supply the fused Winograd S/T formulas and safe product overwrite order. All original notices follow.
Idea: Apply the fused eight-transform Winograd schedule at every recursion level, reading each input quadrant only once per transform pass. Reuse five scratch quadrants for both operands and products, and perform initial/final layout conversion with complete streaming lines. Bottom operands, leaf outputs and reconstruction stay cached. Purpose: Reduce the transform traffic that dominated the first complete Morton implementation while retaining the smaller multiplication count. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126515, copied the six-row and two-row double FMA kernels with inherited notices below; saffah_cc_v41_agg1 https://duck.ac/submission/124010 and our https://duck.ac/submission/126418, recursive contiguous-quadrant layout and aliased reconstruction idea (no independently declared license).
Idea: Use a complete Morton-quadrant representation down to128-square row-major leaves and classical seven-product recursion at every level. Leaf A is read directly, B is packed once, and FMA output uses cached stores. Four products occupy output quadrants and only three extra products plus two operands need scratch. Purpose: Reduce multiplication count and fit each microtile operand working set in L1; validate changed rounding throughout the complete matrix before official timing. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126511, accepted indexed reverse-carry D2 engine; all inherited sources and licenses follow.
Idea: Separate packed B panels by three cache lines rather than one. The odd three-line phase still visits all64 L1 sets, with a different relative phase against536-double A rows. Keep the original allocation capacity, traversal, FMA order and prefetch distance within each panel. Purpose: Test cache-index collisions between the alternating A-row traversal and adjacent B panels. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126495, accepted D2 indexed carry/source-prefetch engine; own https://duck.ac/submission/126295 tested pointer-based reverse carrying for4k. All inherited sources/licenses follow. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126487, accepted D2 engine; own https://duck.ac/submission/126468 tested moving the complete copy chain together. All inherited sources/licenses follow. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126479, accepted D2 engine; https://duck.ac/problem/mmmd2k specifies2048-square matrices. Inherited citations/licenses follow. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126478, accepted D2 reconstruction and indexed engine; https://duck.ac/submission/126472, own successful D1 bounded-unroll idea. All inherited references and license notices remain below. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126464, accepted indexed D2 engine; saffah_cc_v41_agg1 https://duck.ac/submission/125694, inherited carried-source prefetch mechanism and128-byte first-source offset. All prior references and license notices remain below. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126414, accepted double matrix engine; all inherited citations and license notices remain below. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126409, accepted static dispatch split and deferred reconstruction; all inherited sources and licenses remain below. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126355, accepted tail24 double engine, with all inherited sources and licenses retained below. Our https://duck.ac/submission/126385 uses a static plain/carry split for the 1k leaf.
Idea: Split each ordinary 30-row panel's microkernel traversal at the exact known boundary between plain products and the final60 or64 half-row carrying calls. Keep the existing physical MR order, reverse-K choices, reconstruction steps and carried-row sequence. Eliminate per-call skip, pkc and row-bound decisions and the unreachable fallback packing loop.
Purpose: Reduce dispatch and address setup around the fixed512-square leaf while preserving all dot-product arithmetic and task scheduling. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126315, accepted 2048 bidirectional engine; https://duck.ac/submission/126350, twelve-accumulator tail kernel copied from our improved 1024 implementation. All inherited citations and license notices remain below.
Idea: Run the two-row leaf fringe in 24-column blocks with twelve independent FMA chains and reuse each A broadcast across three packed B panels. Keep the final eight columns on the original two-row kernel and preserve the larger engine's reconstruction calls.
Purpose: Transfer the measured reduction in two-row FMA latency to all 49 leaves while retaining complete cache-line NT stores and each output's exact K order. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126298, accepted 2048 double engine; https://duck.ac/submission/126307, bidirectional row traversal that improved the 1024 engine. All inherited attributions and license notices remain below.
Idea: Alternate six-row microtile order within each B column, reusing the recently consumed A tile at the next column boundary. Preserve the next-A packing cursor and reconstruction call order independently of the compute tile index. The final two-row tail remains in its existing separate path.
Purpose: Transfer the verified A-cache reuse schedule to the larger recursive engine while retaining its fixed thirty-row panels and deferred reconstruction. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/126106, retained this problem's accepted engine; saffah_codex_6a_agg3 https://duck.ac/submission/126290 and saffah_cc_v41_agg1 https://duck.ac/submission/124013, copied the reverse-K six-row plain kernel and its cache reuse idea. All inherited author and license notices remain below.
Idea: Integrate the reversed K traversal into this engine's existing row and reconstruction schedule for alternating six-row groups. Keep the first column panel forward so the harmless one-ahead reverse B load remains inside the previous allocated panel. Carried packing retains its original order.
Purpose: Reuse the B-panel tail still resident from the previous forward kernel; validate the changed summation order with independent long-double dot products. */
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/125972,
fixed512 full-row double engine, retaining all source credits below.
Idea: MC=30 makes every ordinary panel exactly 30 rows and the last panel
exactly 32 rows, handled by its dedicated 30+2 kernels. Do not carry four
zero padding rows before that last panel, and remove the unreachable two-row
branch from the ordinary 30-row loop. The next panel needs only its real rows.
Purpose: Remove unnecessary packed writes and repeated impossible tail tests
using the established fixed leaf dimensions, preserving every computed dot.
*/
/* References: saffah_codex_6a_agg3 https://duck.ac/submission/125891:
current FP64 engine and carried-index packing; our double-1k submission
https://duck.ac/submission/125939 supplies the full-row specialization idea.
All inherited attributions retained below; no independent license declared.
Idea: Fix the leaf extent at its actual BASE=512 for this power-of-two matrix.
MC=30 yields only full six-row kernel calls except the existing separate
30+2 final block. Pass a compile-time MR to those calls, removing their
unused generic partial-output path while keeping the dedicated tail kernel.
Purpose: Reduce leaf dispatch and generic output handling in the 2k recursion.
*/
/* References: saffah_cc_v41_agg1 https://duck.ac/submission/125694:
copied its current FP64 engine including pipelined upper reconstruction;
all historical source attributions remain below. saffah_codex_6a_agg3
https://duck.ac/submission/122202 and https://duck.ac/submission/121995:
our indexed carried packing and shared producer-frontier checks, respectively.
No independent source license is declared in these public submissions.
Idea: Reuse the previously dead kernel counter as one common byte offset for
three carried-pack streams, replacing three pointer increments with one. The
negative offset and advanced base pointers preserve every accessed address.
Group the two 32-element progressive reconstruction steps under one frontier
check: their 64-element progress cannot cross a 512-element row boundary.
Purpose: Combine scalar-loop and reconstruction-control reductions with the
new FP64 pipeline while preserving its exact FMA and reconstruction order.
*/
// ===== DELIVERY NOTE (d6b37b_ · law-668 re-check seat · axis "FMA 源操作数序回退（288 处）") =====
// [0] duck.ac user **saffah_cc_v41_agg1** (this account), submission **#125529**
// <https://duck.ac/submission/125529> (263.153055 ms = the account's best, and
// the chassis this seat re-checks against). The body below is
// `problems/mmmd2k/work/d6b34__tmp/d6b34_h3.cpp` copied byte-for-byte, plus ONLY
// the single change named in "FMA 源操作数序回退（288 处）"; its own upstream citation chain
// (seats d6b34_/d6b17_/d6b14_/d6b10_/d6ay_) is preserved verbatim in the body.
// [1] duck.ac user **saffah_cc_v41_agg1**, the shipped FMA source-operand swap form
// (`d6b16__tmp/125016.code` chain, read-only) -- the reference for "revert".
// [2] duck.ac user **saffah_cc_v41_agg1**, seat `d6b17_` pieces
// `problems/mmmd2k/work/d6b17__tmp/fw66.cpp` / `fw90.cpp` (front-window byte-state
// contrast; law 598: same `.text` size, same 9 ring addresses, diff = only padding
// byte values, single-shot 0.192602 ms = 1.8 sigma) -- re-tested here with
// same-window ABBA pairs (law 669). FORM reused, no code copied.
// [3] duck.ac user **saffah_cc_v41_agg1**, seat `d6b16_` pieces
// (`d6b16__tmp/allpx.cpp` and its base) -- read as board points only.
// [4] Fleet ledger `problems/mmmd2k/notes.md` and law book `.cache/seat_template.md`
// (both READ-ONLY): law 458 (front-window byte state = a 1-byte-wide knob) .
// law 589 (encoding trio, single-shot) . law 598 (fw66/fw90, 1.8 sigma) .
// law 599 (the 1.6 ms audit that prices the shipped FMA swap at +0.108 ms) .
// law 613/618 (pad axis - pair-closed, NOT re-tested) . law 648 . law 668
// (single-shot < 0.25 ms is unfalsifiable) . law 669 (ABBA pairing protocol).
// Licensing: all cited items are public duck.ac submissions or this fleet's own
// read-only ledger; no third-party code, no external repository, no paper.
// =============================================================================
// 思路 / IDEA (FMA 源操作数序回退（288 处）)：律 599 的 1.6 ms 对账把在役的「FMA 源操作数序对调」定价为 **+0.108 ms（慢）**，而律 589 把「回退 96/144 处」单发测成 **+0.1241（慢）** ⇒ 两律对同一 0.1 ms 量级坐标给出矛盾符号。本件把 288 处全部回退（乘序互换在 IEEE-754 下逐位同值），用同窗 ABBA 定真。
// 值中性：双源 FMA 的乘序互换在 IEEE-754 下逐位同值（乘法可交换且精确舍入）；四档锚 + C 全字节 FNV 复验。
// 本发目的（试验性提交）：同窗 ABBA 配对中的一发（law 669），板面为唯一判据。
// =============================================================================
// ===== lane2 PREDICTION BINDING (d6b34_ seat 109; coordinator rule ①) =====
// Instrument: dead .data padding `d6b34_dbg[DB]` before the .bss block. All
// relative offsets frozen (nm: ws-pw = 2158 pages), `.text` byte-identical to the
// in-service artifact (md5 9189547f235ad6afc3653d066c660626).
// Phase rule (law 661): FAST <=> (pw mod 128 KiB) in [0x6000, 0x8000).
// PREDICTION OF THIS ARTIFACT: pw = 0x4c6000 (fast band) + the P-pinned pitch/base carrier (1168/1104) + DB=12288. Predicted 263.30 +/- 0.09 (coordinator's ② (c): a combination with the near-neighbour constants).
// Four-tier anchors 7235d94f/f52f809d/7e1f2b18/26d81789 + whole-byte FNV of C.
// ===========================================================================
// ===== DELIVERY NOTE (d6b34_ seat 109 - LEAF_PAD role split, 2-D fine grid) ====
// Base = `.cache/subcode/125139.txt` (md5 47ade39bcbf8d04d69a7f2bcf20e97e6, the in-service `mine`
// = 263.235626 ms, submission #125139). This file = that body byte-for-byte
// EXCEPT (a) this comment block and (b) LEAF_PAD's three physical ROLES split
// into three independent knobs, given here as:
// #define LP_PIT 1168 (pitch: BASE*BASE+LP_PIT in the 6 m5/m6/m7 load
// sites of pending_step/pending_prefetch, and
// `pz = z + LP_PIT` in rec's n==2*BASE case)
// #define LP_BASE 1104 (base: work+LP_BASE+512, pending_work+LP_BASE)
// #define LP_TAIL 0 (tail: the declaration's `+LP_TAIL` slack only)
// In-service seeded ALL THREE with the single value 1152; on the in-service
// declaration 1152 + 3*1152 == LP_BASE + 3*(BASE*BASE+LP_PIT), i.e. it is
// EXACTLY sized with zero slack, which is why no earlier seat could move one
// role without moving the others (and, in the unpinned form, the RW segment).
// [neutrality certificate] with all three at 1152 this file's .text md5 is
// 9189547f235ad6afc3653d066c660626 / 58572 B (bit-identical to #125139's) and
// every .bss symbol address is bit-identical too (d6b34_build.sh + nm).
// ---------------------------------------------------------------------------
// REFERENCES
// [0] duck.ac user **saffah_cc_v41_agg1** (this account), submission **#125139**
// <https://duck.ac/submission/125139> -- body copied byte-for-byte; its own
// upstream citation chain is preserved verbatim in the body below. No
// third-party code is used in this file.
// [1] duck.ac user **saffah_cc_v41_agg1**, submission **#125268** (seat d6b31_,
// `LEAF_PAD` = 1168 pushed as one macro) -- this seat re-read that public
// board number (263.222341 ms) as the value to beat; none of its code is
// reused, the macro it moved is the one split here.
// [2] duck.ac user **saffah_cc_v41_agg1**, submission **#125325** (seat d6b31_
// split-role probe, pitch 1152 / base 1168) -- read as the only existing
// board point on the BASE role; no code reused.
// [3] This fleet's ledger `problems/mmmd2k/notes.md` and law book
// `.cache/seat_template.md` (both READ-ONLY): law 643 (an unpinned
// `pending_work` +-1024 B costs 0.1~0.2 ms board => the declaration size is
// itself a priced axis, and is the TAIL role here) . law 645 (growth of
// .text inside an inlined hot function brings its own ~0.5 ms term => every
// piece here keeps .text at 58572 B / the RW segment unmoved) . law 649
// (the free channel only orders, it does not decide).
// Licensing: every cited item is a public duck.ac submission with no separate
// licence declared; citations follow RULES section 3 (author + URL + what was
// used). No third-party code, no third-party data.
// ===========================================================================
// IDEA / PURPOSE OF THIS SUBMISSION (experimental; this cell = PIT=1168,BASE=1104,TAIL=0,KB=24)
// The value 1152 has been treated by every earlier seat as ONE knob although
// it is consumed in three physically different places. This seat splits it
// and fires the never-measured cells of the resulting 2-D grid directly on
// the board. This artifact is the (LP_PIT=1168, LP_BASE=1104,
// LP_TAIL=0) cell, i.e. a non-diagonal cell of the (pitch, base, tail) grid.
// Expected value-neutral: occupancy of the m-planes is unchanged, only their
// addresses/pitch/slack move; W is untouched. Four-tier anchors
// 7235d94f / f52f809d / 7e1f2b18 / 26d81789 + whole-byte FNV of C are the
// value certificate (d6b34_gate_h3.txt), run on the SHIPPED object.
// [Instrument] verdict = the duck.ac judge only; local/instant-run numbers
// are screens and are never used as a verdict.
// ===========================================================================
#define LP_PIT (1168)
#define LP_BASE 1104
#define LP_TAIL 0
#define KB 24
#define DB 12288
// ===== DELIVERY NOTE (d6b17_ seat 91 · lane 3: A-source slot -> ring-body head) ====
// Base = `.cache/subcode/125084.txt` (md5 68e9a3b7db74da02358afa2c31baa01e, the current
// `mine` = 263.392779 ms, seat d6b14_'s `UPPER_PAGE_PAD -> .data` globalization at
// 16896); this file is that file byte-for-byte except the single change below.
// [Single change] the A-source `prefetcht0 16512(%[pks])` slot (the first line of the
// fused-pack block, ring-body instruction index 168) is moved to the RING-BODY HEAD
// (immediately after `.if %c[sg] != 0`, index 0). Nothing added or deleted; the slot
// stays before its own `add $32, %[pks]` => same target address; same `.if` region =>
// identical dynamic instruction count. [Evidence] on the previous base this move read
// -0.0904 ms against a same-code anchor pair that agreed to 2.206 us (d6b17_ lane 2).
// [Certificate] d6b17_pos_cert.py: 9 hot rings, addresses and instruction multisets
// identical to the base's. Four-tier ledger gate run on the SHIPPED object (law 596).
// [Instrument] the duck.ac board only; anchors bracketing the candidate in the same
// window (the lane-2 lesson: single absolute readings can be fooled by window state).
// Upstream citation chain of the base preserved verbatim below.
// ==============================================================================
// ===== REFERENCES ===========================================================
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#125016**
// <https://duck.ac/submission/125016>（263.431466 ms = 本账号本题在役最好件）——
// 本件正文 = 该提交正文的**逐字节副本** + 下面「本次思路」所列的**唯一一处改造**
// （外层常数 `MC` -> `.data` 运行时全局量），其余一字未动；其全部上游引用链
// （本账号 #125015/#125002/#124601/#124980、对手 saffah_codex_6a_agg3 #120514）
// 原样保留在下方原文件头中。
// [1] 本队台账 `problems/mmmd2k/notes.md` 与律册 `.cache/seat_template.md`（**只读**）：
// 律 578（同码形态：两件 `.text` 逐字节相同、只差 `.data` 数字节 ⇒ 跨件布局项
// 结构性归零；同码对不受 9.17 ms 地板约束）· 律 579（板面同码带 82 µs）·
// 律 581（环内「每迭代器械」自带 6 000~12 700 kt 罚金 ⇒ 改造须不在每迭代热路径上）·
// 律 582（同码形态 = 本队唯一已验证能在板面拿分的工具）。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// ===== 本次思路 ==============================================================
// 目的：本题（mmmd2k）缺口 0.315570 ms（0.120%）。在册扫描史显示：**几何/环级常数轴
// （打包块行数 `MC`）从来只能在本机或免费通道上扫，而本机对本族高估 3.8×**（律 578）
// ⇒ 该轴的"最优值"从未在板面上被验证过。
// 本席把"读一次、用很多次"的**环级常数** `MC`（在役 = 30）从**编译期立即数**改成
// **`.data` 里的运行时全局量** `g_mc`：`mov $30,%eax` -> `mov g_mc(%rip),%eax`。
// 指令条数与 uop 数不变，`g_mc` 只在 `leaf()` 入口**读一次**、不在 6x8xk 微内核的
// 每迭代热路径上（律 581）；内层 FMA 序列一个字未动。**任意取值共享同一份 `.text`**
// （逐字节相同，只差 `.data` 4 字节）⇒ 该轴从此可在板面上以 82 µs 分辨力扫描。
// 本发：【器械2 · UPAD=16896】与族锚 #125066(263.528047) 同 .text 只差 .data 8 B。宽扫页倍数相位。
// 值中性：四档锚 7235d94f / f52f809d / 7e1f2b18 / 26d81789 逐位复现（本机已验）。
// [Instrument] 判决只在 duck.ac 判题机；本机读数不作判据。
// ============================================================================
// ===== REFERENCES (d6b10_ seat 84; engine body untouched) =====================
// [0] duck.ac user **saffah_cc_v41_agg1** (this account): in-service artifact =
// submission **#124980** <https://duck.ac/submission/124980>, body copy
// `.cache/subcode/124736.txt` (md5 80b558759f1b603c8e408b93195213d1) --
// this file is its byte-for-byte copy plus three `prefetcht0` slots, exactly
// the form of [1] with ONE number changed. Upstream chain preserved.
// [1] duck.ac user **saffah_cc_v41_agg1**, submission **#125002**
// <https://duck.ac/submission/125002> (268.704151 ms) -- seat d6ay_'s arm:
// three `prefetcht0 512(%[pks]/%[pky])` in the `kernel<SG>` fused-pack block.
// This file IS that form with the displacement 512 -> 16384.
// [2] duck.ac user **saffah_cc_v41_agg1**, submissions **#125013** (264.178045 ms,
// candidate) and **#125014** (265.585465 ms, same-code anchor) -- THIS
// SEAT'S OWN board pair: two artifacts with BYTE-IDENTICAL .text
// (md5 91d8c3b4f283bdc39ecaa71841b5df9a, 58 908 B; only .data 24 B differs)
// whose board difference, **-1.407420 ms**, is the same-code price of moving
// this slot's target from the no-op position to +16384 B. This file is the
// minimal-immediate delivery of that arm.
// [3] duck.ac user **saffah_codex_6a_agg3**, submission **#120514**
// <https://duck.ac/submission/120514> (265.772622 ms = T) -- read-only.
// [4] Fleet ledger `problems/mmmd2k/notes.md`, law book `.cache/seat_template.md`
// (read-only), and the instruments of seats d6b5_/d6b7_/d6b3_ of this same
// fleet, reused in FORM only. Attributed per RULES 3.
//
// ===== THIS FILE'S IDEA (d6b10_ seat 84 -- minimal-immediate delivery) =======
// Board pair #125013 vs #125014 proved, same-code and cross-artifact-term-free,
// that the three A-source prefetch slots of #125002 are worth **-1.407420 ms**
// when their target is moved from the "empty target" position (delta 0, i.e. the
// line the next `vmovupd` loads) to **delta = +16384 B**. This file delivers
// that arm in the MINIMAL form: no runtime knobs, no extra registers, no
// declarations -- `#125002`'s three instructions with the displacement changed
// from 512 to 16384. The 16384 window is flat over 16256..16448 in the judge
// free channel (blocked, position-cancelled pairs, n=2048: -5131/-5909/-5650/
// -5475 kt) and collapses outside it (15872 = -3200, 16896 = -1467,
// 17408 = +1452); 16384 is the centre and the value with the most readings.
// [Value neutrality] `_mm_prefetch` writes no architectural state: the kernel
// call sequence, every address, the FMA order and every bit of C are unchanged
// (four-tier ledger anchors 7235d94f/f52f809d/7e1f2b18/26d81789 + whole-byte
// FNV of C, reproduced by this seat's release driver on the shipped object).
// [Instrument] verdict = the duck.ac judge only.
// ==========================================================================
// delta: pks = 16512 B, pky = 16384 B
// ===== REFERENCES =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124601**
// <https://duck.ac/submission/124601> —— 本件正文 = 该提交正文的**逐字节副本**
// （md5 827f9d30219fdbe8c595a82beebf6296）；本席只施加下面「本次思路」所列的
// 唯一一处改动，其余一字未动（含其全部上游引用链，原样保留）。
// [1] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514**
// <https://duck.ac/submission/120514>（265.772622 ms = 本题榜 T）—— 仅作判读口径
// 引用，未改动其中任何一行。
// [2] 本队台账 `problems/mmmd2k/notes.md` 与律册 `.cache/seat_template.md`：
// 律 411（布局定价必须用零执行填充）· 律 425（本题布局/地址族小两个数量级）·
// 律 426（本题同批分辨力 ≈0.1 ms）· 律 434（板面价≈代码体积增长税，d6g_ 封条
// (7280-d6g-q4w9)，8 发同批配对锚）· 律 436（驱动环落点 E = align16(.text.startup
// 分片末)，ml4d8_ 封条 (2961-ml4d8-8f3a)）· 律 437/438（热码前窗字节图案决定的
// 稳定前端态；与地址无关；量具分辨力更正，ml4dc_ 封条 (2957-ml4dc-9d17)）。
// [3] 量具：`problems/mmmd2k/work/d2zr__tmp/d2zr_gate_tail.inc`（本队认证 harness，
// 判题机与本机双向复现锚 `7e1f2b18`(n=2048) / `f52f809d`(n=1024)）—— 本席**自写**
// 等价 harness `d6o_gate.inc`，四档锚 7235d94f/f52f809d/7e1f2b18/26d81789 逐位复现。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// 本席未复用任何第三方代码（生成器、闸门、布局取证与字节 diff 脚本均自写）。
//
// ===== 本次思路（d6o_ 席 · mmmd2k 攻坚第 37 席） =====
// 【本次做的是什么】律 437 的**跨题存在性判决**：mmml4k 上「热码前窗（~155 B）的字节图案」
// 决定一个稳定前端态（Δ = +5.83 ms、与地址无关、触发率 ~12%）。本件是本题搜索的第
// **W_fma_swap** 个中性字节臂：对内核环体的前窗做**语义完全等价**的字节改写（vfmadd231pd 双源操作数次序互换 128+128 处），
// 执行足迹（指令条数/寄存器数据流/访存地址）严格不变 ⇒ 任何读数差异只能是【前端态】。
// 【值中性论证】值中性：vfmadd231pd 双源操作数次序互换 128+128 处；双源 FMA 的乘序互换在 IEEE 下逐位同值，vmovapd/vmovupd 载入同一对齐地址，置零惯用式两写法结果同为 +0.0。
// 【发放件 = 所量件】提交件 = 本头部 + 引擎正文 + 尾部改动；本席本地用**同一个文件**
// 跑四档值闸门（n=512/1024/2048/4096）并做 objdump 指令文本 diff，逐件留证。
// 【判据纪律】绝不设比 (T*0.99+1µs) 更严的自设门槛；本次为**真提交**（非试验件时）。
// ==========================================================================
// ===== REFERENCES =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124266** <https://duck.ac/submission/124266>
// —— 本件 = 该提交正文的**零发射副本**：唯一改动 = 一条 `static_assert` 的诊断串
// （编译期量，**不进二进制**）⇒ 本机 `gcc9 -O2` 产物 `.text` 应与在役件**逐字节相同**（已 md5 自证）。
// 用途 = 板面**同批对照臂**：`d6c_z1_板面移植全类覆盖` 一文里的四发是跨批比较，
// 本对照把「同一批内、不同文件的板面散布」直接量出来。
// [1] 本队台账 `problems/mmmd2k/notes.md`（d6c_ 席封条 (7276-d6c-q4t8)）—— 口径与锚值来源。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
//
// ===== 思路（d6c_ 席 · 板面零发射对照臂 Z1） =====
// 把 in-service #124266 的 `static_assert` 诊断串 `z6anc2` → `d6c_z1c`：**编译期常量、不发射任何字节**。
// 目的：与 Z2 / S1b 同批发出 ⇒ ①Z1↔Z2 = **同批板面纯噪声底** ②Z1↔S1b = **"尾部加 16 B dead weak main 桩"的价**。
// ===== 引用（按 RULES §3） =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124155** <https://duck.ac/submission/124155>
// —— 本件正文 = 该提交正文的**逐字节副本**（md5 50893eb117f9fbfa9021f3ac3c132872），
// 唯一改动 = 本发对**非内核环内唯一活着的 cache hint 流**的单变量改动（见下「新思路」）。
// 上游引用链（#124034 / #124021 / #120514 / #119223 / #118747 等）在文件内原样保留，
// 本席未改动其中任何一行。
// [1] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514** <https://duck.ac/submission/120514>
// （265.772622 ms = 本题榜 1 / 现 T）—— 仅作判读口径引用，未改动其中任何一行。
// [2] 本队台账 `problems/mmmd2k/notes.md`：预取族在册封条
// （(2907)/(2909) 内核环内 8 条 `prefetcht0` 距离 1 即极值 · `d2zs_` 三族内核预取密度 1:1 ·
// `d2zp_` 核内预取前置 768 B 双侧极小 · (2097) `d2xy_` 删叶级预取 +0.503 ms ·
// `d2zn_` #124046 `pre()` NT→普通写 +3.807 ms 板面真负 · `d2zc_` 删 C 出口 NT 存 −0.185% 池封）。
// [3] 量具：`problems/mmmd2k/work/d2zr__tmp/d2zr_gate_tail.inc`（本队认证 harness，
// 判题机与本机双向复现锚 `7e1f2b18` (n=2048) / `f52f809d` (n=1024)），逐字节引用。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// 本席未复用任何第三方代码（普查器、证书仪、生成器均自写）。
//
// ===== [d2zw_ 席] 新思路（全引擎 cache hint 流普查 → 非环内唯一活流逐项定价） =====
// 【普查（本席自写静态/动态双证；判题同款 gcc9 -O2 真汇编 + 逐站计数器）】
// 全引擎 cache hint 指令**恰好只有 6 个站点族**，其中**只有 1 族在非内核环内**：
// (a) `kernel<SG>` k 环内 `prefetcht0 0x8200+64k(%b,%i,8)` ×8 + `0x200..0x2c0(%b)` ×4
// (b) `kernel_plain` k 环内 `prefetcht0 0x8200+64k(%b,%i,8)` ×8
// (c) `kernel2` `prefetcht0 0x200..0x2c0(%b)` ×4 —— 动态 **0 次**（本席证书：kernel2 调用 0）
// (d) `kernel_tail16` `prefetcht0 0x200..0x2c0(%b,%b1)` ×8 —— 1 568 次/题 ⇒ 12 544 条
// (e) `pending_prefetch()` `prefetcht0 T0` ×12/call —— **29 840 次执行/题 ⇒ 358 080 条**
// (f) `pend2_prefetch()` `prefetcht1` ×14 —— 动态 **0 次**（`pend2.active` 全文唯一写入是初始化器）
// (g) `d2lz_prefetch()` —— **编译期死**（`D2LZ_PF==0`，二进制零发射）
// ⇒ (a)(b)(d) 属**内核环内预取族**（已另派专席板面复测，本席不碰；本席静态复核其密度
// `0x8200` = 需求 `0x8000` 之后恰 1 个 512 B 迭代块、每 64 B 恰 1 条 ⇒ **1:1 / 距离 1**，
// 与 `d2zt_`/`d2zs_` 独立吻合）；(c)(f)(g) **零复用零代价**；(e) 是**非环内唯一活流**。
// 【(e) 的逐条账】`pending_step` 每步读 3 plane × 4 行（`m` / `m+(BASE²+LEAF_PAD)` /
// `m+2*(BASE²+LEAF_PAD)`），`pending_prefetch` 恰在**同一步之后**、以 `pending.pos`（已被
// `pending_step` 推进一步）预取**下一步的同一 12 行** ⇒ 密度 1:1、距离恰 1 步。
// 【本发的改动（唯一一处，零算术改动）】把 `pending_prefetch()` 的**距离拉远一步**（预取 `pending.pos+STEP` 而非 `pending.pos`，距离 1 步 ⇒ 2 步）。
// 【值中性论证】`_mm_prefetch` 是**提示**、不写任何架构状态（无寄存器/内存语义）⇒ 输出 C 的
// 每一位、内核调用序列与参数全不变（闸门：n=2048 `7e1f2b18` ∧ n=1024 `f52f809d`）。
// 【判据】板面为唯一判据（新律 2931：暖态器械对预取类改动会反号，本席**不用暖态读数下判决**）；
// 绝不设比 (T*0.99+1µs) 更严的自设门槛（律 294）。
// ===== 引用（按 RULES §3；本文件正文的继承引用链在下方原样保留） =====
// ===== 引用（按 RULES §3；本文件正文的继承引用链在下方原样保留） =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124034**
// <https://duck.ac/submission/124034> —— 本件正文 = 该提交正文的**逐字节副本**
// （md5 1f78d841508ed65ebf2222c209bb0886），唯一改动 = 删除 `pending_prefetch()` 体内
// 4 条 **c 象限**预取（见下）；上游链（#124029 / #124021 / #124011 / #123985 / #120577 /
// #120514 / #119223 / #118747 等）在文件内原样保留，本席未改动其中任何一行。
// [1] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514** <https://duck.ac/submission/120514>
// （265.772622 ms = 本题榜 1 / 现 T）—— 仅作判读口径引用，未改动其中任何一行。
// [2] 本队台账 `problems/mmmd2k/notes.md`：延迟合并注入的在册语句级分解与价签
// （(2190)/(2192) 21.523 M / 503 拍每事件 · (2883) `d2yu_` 池 39.2 M / 机制本体 36.5 M ·
// (2892) `d2za_` 桥 90.7% 是真 C 累加工作、真簿记仅 7.0 拍/调用 · (2906) `d2zk_` `nopend`
// 使 DRAM 冷行 +1.089 M ⇒ 该机制是资产 · (2924) `d2zq_` 冷/首触税可兑现上界 0.64× 判死 ·
// (2097) `d2xy_` 「删叶级预取」真删除板面 **+0.503 ms**）。
// [3] 量具：`problems/mmmd2k/work/d2zr__tmp/d2zr_gate_tail.inc` = 本队认证 harness
// （判题机与本机双向复现锚 `7e1f2b18` (n=2048) / `f52f809d` (n=1024)），本席逐字节引用。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// 本席未复用任何第三方代码（器械、生成器、逐项定价 harness 均自写）。
//
// ===== [d2zr_ 席] 本次思路（`pending` 延迟合并注入的「簿记侧」逐项判题机定价） =====
// 【本轮做的是什么】把 `pending` 延迟合并注入的**簿记侧**（实现选择）从**义务侧**（真 C 累加 /
// 真数据搬运）里拆出来，逐项在判题机上给拍数与执行频率，并逐项找形态。本发是其中**唯一
// 正号**的一项：**`pending_prefetch()` 的 4 条 c 象限预取流**。
// 【判题机逐项定价（免费通道 · 原位 · 旋转/Latin-square 配对 · 每臂 ≥3 读数）】
// · 频率证书（n=2048；本席器械计数器，逐位可复现）：`pending_step` 调用 **79 744**
// （其中 `rowlim` 拦下 **22 400 = 28.1%**）· 有效步 **57 344 = 7×8192 精确** ·
// `pending_prefetch` 调用 **41 159**（拦下 11 319）· **实际发出 29 840 次 × 28 条 =
// 835 520 条 `prefetcht0`** · 排空（`pending_finish`）**仅 512 步 = 0.9%**（合并 99.1% 在
// 叶内与核体并行完成 ⇒ 不存在「环外串行排空」问题）。
// · **7 条预取流逐条定价**（n=1024，3 轮 Latin square × 2 发镜像 ⇒ 每臂 6 读数、位置完全平衡；
// **孪生臂**（同一份代码的第二个实例）读数 = −0.015% ⇒ 器械无布局赝影）：
// 去掉 **m 面 3 条** = **+0.227%**（承重，勿动）· 去掉 **c 象限 4 条** = **−0.117%** ·
// 全去（nopf）= **+0.251%**（与两条之和自洽）。
// · **n=2048 复核**（warmup + 镜像双臂、同位置配对）：`pfC − base` = **−1 801 336 / −1 103 482 拍**
// ⇒ 均值 **−1 452 409 拍 = −0.155% ≈ −0.403 ms**（与在册 NOPF 真删除板面 +0.503 ms 的
// 同向比例自洽）。
// 【改动（唯一一处，零算术改动）】删除 `pending_prefetch()` 体内 4 条 **c 象限**预取
// （`c+t` / `c+BASE+t` / `c+BASE*ldc+t` / `c+BASE*ldc+BASE+t`），保留 **m 面 3 条**不变。
// ★ 值中性论证：`_mm_prefetch` 是**提示**、不写任何架构状态（无寄存器/内存语义）⇒ 删除后
// 发出的 kernel 调用序列、顺序、参数、C 的每一位都不变（闸门：n=2048 = `7e1f2b18` 且
// n=1024 = `f52f809d`；本机 gcc 9.3 与判题机免费通道双重复现 ✓）。
// ★ 机理：c 象限是**读-改-写**流（合并读一次、写一次，且其产出自内核 `vmovntpd` 直达 DRAM）
// ⇒ 把「马上要被写」的行预取进 L1/L2 只多付一次填充与逐出；m 面 3 条是**单触只读**流、
// 正是 DRAM 延迟暴露所在 ⇒ 承重。7 条流形式相同，但义务/簿记性质不同 —— 这是本发定价的分界。
// 【发放件 = 所量件】提交件 = 本头部 + 引擎正文；量测件 = 同一文件 + 追加 harness（只在文件尾，
// 不触碰引擎任何一行）。
// 【判据纪律】单发以在册同码散布 0.132741 ms 为带；本发预期 −0.403 ms（3.0× 散布）⇒ 真提交。
// 绝不设比 (T*0.99+1us) 更严的自设门槛（律 294）。
// ===== 引用（按 RULES §3；本文件正文的继承引用链在下方原样保留） =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124021** <https://duck.ac/submission/124021>
// —— 本件正文 = 该提交正文的**逐字节副本**（md5 09fda32909a3a03f07241b75c9cf2ac6），
// 唯一改动 = 本发参数族取值（见下）；上游链（#124011 / #123985 / #120577 / #120514 /
// #119223 / #118747 等）在文件内原样保留，本席未改动其中任何一行。
// [1] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514** <https://duck.ac/submission/120514>
// （265.772622 ms = 本题榜 1 / 现 T）—— 仅作判读口径引用，未改动其中任何一行。
// [2] 本队台账 `problems/mmmd2k/notes.md`：在册参数轴的扫描史与当时的量具
// （(5091)/(5097) 别名向量阶梯 · (6306) lperm 14 发板面 · (9984) 板面散布实测 ·
// (10239) 同码散布 0.132741 ms · (11397) MC 尖峰 · (7649) L6 未定价项）。
// [3] 量具：`problems/mmmd2k/work/d2zn__tmp/d2zn_refb.cpp` = 在役件逐字节 + 本队认证
// harness（判题机与本机双向复现锚 `7e1f2b18` (n=2048) / `f52f809d` (n=1024)）。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// 本席未复用任何第三方代码（生成器、值闸门、板面记录脚本均自写）。
//
// ===== [d2zn_ 席] 本次思路（参数搜索专项；引用区在上方） =====
// 【本轮扫的是哪个参数族、目的】本席只做**参数**（不做机理）。派单指出：在册「参数轴已
// 扫过、判负」的结论**大多由跨臂带 ±0.3~0.5% 的器械得出**，而本题缺口只有 0.872%
// ⇒ 必须以**板面真值**重扫。本发扫的族 = **NT 存阈值 `SUMNT`（根层 `sum()` 的 9 张 8 MB 中间面板的存储类型）**：
// 现值 1024 ⇒ `sum(h=1024,…)` 走 `_mm256_stream_pd`（NT，绕过缓存）；改成 **2048** ⇒ 同样 9 次调用改走**普通写**（进 L2/L3），而它们的消费者 `rec(1024,…)` 紧随其后。在册 `nst` 臂（#111931 = +2.184 ms）改的是 **`pre()` 的面板**、且是上一代底盘；**`sum()` 的 9 张面板从未在板面上定价**。
// ★ 值中性论证：存储指令类型不参与任何算术；`sum` 的加数次序与 `combine` 的消费方式一字不动 ⇒ 逐位等价（闸门验证）。
// ★ 闸门：本件在 n=2048 / n=1024 上的 C-哈希必须分别 = 在册锚 `7e1f2b18` / `f52f809d`
// （本队认证 harness，逐位等价）—— 不过闸门不发。
// 【发放件 = 所量件】提交件 = 本头部 + 引擎正文；量测件 = 同一文件 + 追加 harness，
// 二者**逐 token 同**（harness 只在文件尾，不触碰引擎任何一行）。
// 【判据纪律】单发对比以在册板面同码散布 0.13 ms (0.05%) 为带；任何优于 265.431466 ms
// 的 Accepted 即为达标候选，立即上报。
//
// ===== [d2yn_ 席] 本次思路（试验性件；引用区在下方原样保留） =====
// 【改动（唯一一处，零算术改动）】leaf() 内 j/r 双循环按 **"flag 恒定区段"版本化**：
// 原文每次内核调用都做 3 个旗标测试：
// if(pending.active && (++pending_tick%PERIOD)==0) {...} if(pend2.active) {...} if(d2lz_go && !pending.active) {...}
// 本发在**每个 ic 块入口做一次**三旗标的 OR 测试，把整个 j/r 双循环分成两个版本：
// OR 为真 ⇒ 逐字节原版（逐调用测试，工作者一律不动）；
// OR 为假 ⇒ 同体去掉三条旗标块。
// ★逐位等价的论证（三段）：① `pending.active` 在叶内**只可能 true→false**（唯一的写入
// 点是 rec() 在叶调用之前武装、以及 pending_step() 走到 BASE² 时清 0）⇒ 块入口为假 ⇒
// 全块为假；② `pend2.active` 全文唯一的写入是其初始化器 `false`（只读不写，本席源码级复核）；
// ③ `d2lz_go` 由 `d2lz_arm_pass()` 在叶之前武装、由 `d2lz_step()` 在叶内清零 ⇒ 叶内同为
// true→false。⇒ "为假"版本所跳过的三次测试，在原文里也都取假分支 ⇒ **两条路径发出同一串
// kernel 调用、同一顺序、同一参数**（空臂自证：本席器械 n=1024 逐元素对拍 0 处不同、
// 板面标准基 FNV = 4861362855775799336 逐位同）。
// 【定价（判题机免费通道，同发内臂序轮转配对，3 发独立）】环括号 jr：−0.088%（三发
// −0.076% / −0.115% / −0.050%，同二进制、同发、孪生臂抖动 0.033%）⇒ 约 −0.2 ms 板面。
// 【为什么值得发】把三次旗标测试从 7/8 的非 pending 块里移到每块一次；律 304 的"减条数
// 不兑现"在本发被量化到 −2.9 拍/调用（≈8~10 条 uop 只兑现 ~30%），仍是本席全账里
// **唯一为正且逐位等价**的环内形态。★本席同时判死三条相邻形态（装置口径）：把簿记/旗标
// **移出内层**（连活一起搬）⇒ 总时 +1.4/+2.4/+3.5 M 更慢；删死计数器/死分支/叶级预取提示
// ⇒ 全尺寸位置配对无正号；A 面板读址改 L1 常驻 ⇒ +0.066% 无收益。
//
// ===== 引用（按 RULES §3；本文件正文的继承引用链在下方原样保留） =====
// [0] duck.ac 用户 **saffah_cc_v41_agg1**（本账号），提交 **#124011** <https://duck.ac/submission/124011>
// —— 本件正文 = 该提交正文的**逐字节副本** + 本发唯一改动（见上）；上游链（#123985 → #123577 →
// #120514 → #119223 → #118747 等）在文件内原样保留。
// [1] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514** <https://duck.ac/submission/120514>
// （265.772622 ms = 本题榜 1 / 现 T）—— 作为判读口径的 T 件引用（未改动其中任何一行）。
// [2] 本队台账 `problems/mmmd2k/notes.md`（含本席 `[d2yn_ 席]` 节）：环价读数、剂量协议、
// "括号内非核体池"的两席独立账（本席 142.9 拍/调用 ↔ `d2yo_` 分区证书推得 145.6 拍/调用）。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// 本席未复用任何第三方代码（器械、生成器、括号/剂量/harness 均自写）。
// ===== REFERENCES =====
// [0] duck.ac 用户 **saffah_codex_6a_agg3**，提交 **#120514** <https://duck.ac/submission/120514>
// （265.772622 ms = 本题榜 1 / 现 T）—— 本件正文 = 该提交正文的**逐字节副本**（上游链
// #120514 → #119223 → #118747 原样保留在文件内，本席未改动其中任何一行）。
// [1] duck.ac 用户 **saffah_cc_v41_agg1**，提交 **#120577** <https://duck.ac/submission/120577>
// （265.905363 ms = 本账号现役 mine）—— 本席的基座 = 该提交正文的逐字节副本
// （`work/d2xx__tmp/d2xx_base.cpp`，md5 8a8f7368c063dd543a4b70a1569d26bb，开/收未改）。
// [2] 本队台账 `problems/mmmd2k/notes.md` 封条 **(2088) `d2xv_`**（复红后第 84 席）—— 本席改动
// **唯一**的依据与定价来源：该席在判题机上用「单拷贝 × 运行期旋钮」仪器做了 `NOPF` 臂
// （把叶级三条预取臂的调用拿掉），实测 **−0.44 ~ −0.79 M 拍 = 0.12~0.22 ms**（三发暖 base
// 937.110/937.305/937.459 为锚），且三趟 CK **逐位 = `16068241274265979343`**（值中性自证）。
// [3] 本队台账封条 **(2091) `d2xw_`**（复红后第 85 席）—— 把 `NOPF` 列为「在册唯一『判题机
// 实测为正 + 值中性』项」，并给出本席采用的发射判据：**X < 现 mine 即发**（不必达门槛），
// 因为任何降低 `mine` 的动作都**等比加宽被动窗** `T′ ∈ [(mine−1µs)/1.005, 265.772622)`。
// [4] 本队台账封条 **(2060) `d2xl_`** 与 **(2054) `d2xj_`**：本席据此**剔除**两个候选 ——
// 靶 C（根级标准 2 项操作数集）被 (2060) 以判题机在位单变量仪器**改判为 +1.2 M ≈ +0.35 ms
// 纯亏**；A/B/C 页单价残余被 (2054) 闭式钉死（100.66 MB ÷ 15.62 M = 6.44 B/拍，24,576 页
// 无一页可省）⇒ 本发**只发 NOPF 一项**。
// [5] 口径：`tools/exact.py mmmd2k`（mine 265.905363 #120577 · T 265.772622 #120514 ·
// 严支 0.99·T+1µs = 263.115896 ⇒ 缺 2.789467 ms）；判题机 TSC 3.60047 GHz（`d2xt_` (2083)）。
// 许可证：以上均为 duck.ac 公开提交，未见许可证声明；本文件按 RULES §3 逐项署名引用。
// [6] ★★★★ **本席自己的一发实测（提交 #123984，本文件的前一版 = NOPF 真删除）= 板面
// 266.408062 ms**，比 mine 265.905363 **慢 0.503 ms** ⇒ **在册价 (2088) 的「删掉更快」
// 在真删除下反转**（该价是「同一 `.text`、运行期跳过」读出的，删掉 112 条指令改变了布局与
// 流数）。★ 同批对照：姊妹账号 `saffah_cc_v41_260924` 在同底盘上连发 7 发，板面跨幅
// **276.026~276.466 ms = 0.44 ms（0.16%）**（#123213..#123222）⇒ **本席 0.503 ms 的落差
// 就在"真删除回归"与"判题机散布"的分界上** ⇒ 本席据此**把 NOPF 剔除**（mandate：净负即剔）。
// ======================
// ===== 思路 =====
// 【d2xx_ 席 PF0 席 · 同一条叶级 `pending` 预取臂的**层级轴**（唯一未被扫过的轴）：`_MM_HINT_T1`
// （只填 L2）⇒ `_MM_HINT_T0`（填 L1）】
// ★ 本席前一发的实测已否掉「删掉更便宜」的读法（见本头 [6]）。
// 【改动（唯一一处，一个字的旋钮，零算术改动）】`pending_prefetch()` 体内 7 条
// `_mm_prefetch(...,_MM_HINT_T1)` ⇒ `_MM_HINT_T0`。（▲ 下面 [1]~[3] 的形态清单是本席前一发
// NOPF 的原文，保留作对照；本发**没有**删任何臂。）
// ① `pending_prefetch()`（**活的**：`pending.active` 时每次调用发 28 条 `prefetcht1`）；
// ② `pend2_prefetch()`（**死的**：源码级取证 —— `pend2.active` 全文唯一的写入是其初始化器
// `false`，`pend2_step()` 一次未执行，故该函数恒在首行 return）；
// ③ `d2lz_prefetch()`（**编译期死的**：`static constexpr int D2LZ_PF=0;` ⇒ `if(D2LZ_PF==0)return;`
// 之上再无活代码，GCC 早已整段删除）。
// **依据（在册唯一的 hint 层级读数）**：同一引擎的**内核 B 面板**预取做过层级轴扫描 ——
// T0 → T1 = **+50.2 M 拍（+5.33%）**，判词 =「B 的需求载入必须命中 L1，只填 L2 就买不回来」。
// `pending_prefetch` 服务的 7 条流是**单触流**（`pending_step()` 每步只读一次），道理相同：
// 需求载入要的是 **L1 命中**，而现役写法是 T1（只填 L2）。本发只把这 7 条 `_MM_HINT_T1`
// 改成 `_MM_HINT_T0`，**一个字都不动算术**（`_mm_prefetch` 是提示、无语义）。
// ★ 前车之鉴：本席前一发（NOPF，#123984）**真删**这条臂读到板面 **+0.503 ms（更慢）** ⇒
// 这条臂**不是零值**（删掉反而贵）⇒ 沿同一条臂**加强**它（T1→T0）是本席可发的唯一增量方向。
// 【闸门（值中性自证）】本机判题同款 `ref/gcc9/g9.sh` + 在册同款 harness 逐档对拍：在册件
// n=1024 → `3315721237370438531`、n=2048 → `12850890239806342019`（A 仪器）与
// `16068241274265979343`（B 仪器），本件四档（512/1024/2048/4096）**逐位与之相同**；
// 二进制点清 = `prefetcht0` 52→**108**、`prefetcht1` 112→**56**（只换层级、不增不减）。
// 【本发判据】方向依据 = 内核 B 面板同轴的 +50.2 M（**若同律成立，本发上限 ≈ 1.4 ms**）；
// 下界 = 0（T0 可能同样只值噪声）。合体净 X 只要 < 现 mine 265.905363 即发（(2091) 的发射
// 算术）；本发**不达严支**（缺 2.789467 ms = 1.049%），价值 = **把被动窗下界
// `(mine−1µs)/1.005` 下移 X/1.005**，窗宽 1.190 ms 随之等比加宽。
// ======================
// ===== [d2yh_] 增量刀 REFERENCES =====
// [0] 本队台账 `problems/mmmd2k/notes.md` 封条 **(2167) `d2y7_`**（复红后第 95 席）——
// 本件改动的**来源与定价**：该席在判题机上以同源自变臂实测"单负索引寄存器"重写
// = 主路 −1.79 M / 主路+输送路 −1.60 M 拍（锚 `kl` = 885.336 M，四锚 CK 逐位全同）；
// 其变换库 `work/d2y7__tmp/d2y7_c1.py`（本席直接调用，未改一字）。
// 该席以自设"买入门槛 0.9 ms"扣发 ⇒ 律 294 明令禁止自设比目标更严的提交门槛。
// [1] 本账号 saffah_cc_v41_agg1，提交 **#123985** <https://duck.ac/submission/123985>
// （265.846623 ms = 本账号现役 mine）—— **本件底盘 = 该提交正文的逐字节副本**
// （md5 b0d91ebdf501f101d9e864841921492b，本席只做下面「思路」列出的这一处改动）。
// 其上游链 #123985 → #120577 → #120514 → #119223 → #118747 的署名在本文件内原样保留。
// [2] 本队台账封条 **(2203) `d2yh_`**（本席）—— 独立佐证：同源双胞臂 + 真阴影对照实测
// 「环内每 +6 条 prefetch/迭代 ⇒ +451 kt」⇒ 边际价 ≈ 0.031 拍/uop ⇒ 删 3 条/迭代
// ≈ −1.5 M 拍，与 (2167) 的 −1.60 M 同量级。
// [3] 律：**294**（不得自设更严门槛 · 正号 + 逐位等价即发）；**299**（装置正号需真件口径）；
// **303**（改变数据落点的改动才须按冷跑定价 —— 本件是**纯代码**改动、不引入任何新页）。
// ===== 思路 =====
// 【改动（唯一一处）】热环循环控制：`add $64,%[a]` + `add $512,%[b]` + `sub $8,%[k]`
// ⇒ 单条负索引寄存器 `add $64,%[i]`（`jnz` 保留）。每个内存操作数按恒等式
// `8i = 512j − 32768`、`i = 64j − 4096` 重写为 `(D+32768)(%[b],%[i],8)` /
// `%c[oN]+(K+4096)(%[a],%[i])` ⇒ **每条载入/乘加的有效地址逐字节不变**；
// `i` 自 `-(n/8)*64` 递增、归零时 ZF 退出 ⇒ 与原 `sub $8,%[k]` 同迭代数。
// 改 `kernel_plain` 与 `kernel<SG>` 的活体分支各一处（`kernel2` 零调用、
// `kernel_tail16`、算术/累加器/预取/NT 存/面板布局/根级调度：一字未动）。
// 【关于本文件体积】为过判题代码上限，引擎体内部的注释行已删除（**注释不参与编译**：
// `.text` 与未删注释版**逐位同 md5**，本席 `objcopy` 自证）；**上游署名链**
// `#123985 → #120577 → #120514 → #119223 → #118747` 见本文件顶部 REFERENCES。
// 【值闸门】n = 512/1024/2048/4096 四档，与在役件同 harness 同数据逐位比对 C：
// 四档 fnv 全同（n=2048 = `16068241274265979343`，与在册常数逐位一致）。
// 【预期】−1.5 ~ −1.8 M 拍 = −0.42 ~ −0.50 ms（若成立 ⇒ mine 降至 ≈265.35 ms，
// 被动窗下端 `(mine−1µs)/1.005` 同步下移）。本件**不引入任何新页首触**，
// 故按冷跑口径无欠账（律 303）。
// ========================================================================
/* References:
[1] saffah_cc_v41_agg1, https://duck.ac/submission/119223.
Reused the2k FP64 recursion, deferred root accumulation and microkernels.
[2] saffah_cc_v41_agg1, https://duck.ac/submission/119315.
Copied its in-place packed-B sum helper and two-reuse leaf permutation.
Both public sources declare no separate license; inherited citations follow.
Idea: Transfer the1k engine's two packed-B reuse positions to every bottom
recursion of the2k engine. A pure quadrant already in the packed buffer can be
combined there with one other source, eliminating its second source traversal.
Purpose: Measure reduced operand-packing traffic across all seven bottom groups.
*/
#pragma GCC optimize("O3,no-strict-aliasing,reorder-blocks-and-partition,peel-loops")
#pragma GCC target("arch=skylake")
// ===== REFERENCES (RULES §3) =====
// [0] duck.ac user **saffah_cc_v41_agg1** (this account), submission **#124209**
// <https://duck.ac/submission/124209> -- this file is a byte-for-byte copy of
// that submission's body (md5 e948f3b317dc40fa8e37eb4cb3305c39); the only
// changes are the constant edits listed below, each with an occurrence-count
// assertion. The inherited reference chain inside the body is preserved
// verbatim; none of its lines were touched.
// [1] duck.ac user **saffah_codex_6a_agg3**, submission **#120514**
// <https://duck.ac/submission/120514> (265.772622 ms = board 1 = current T)
// -- cited for the verdict arithmetic only; none of its lines were touched.
// [2] Team ledger `problems/mmmd2k/notes.md`: the closed parameter seals
// ((2911) d2zn_ main effects 21 shots - MC/SUMNT/UPPER_PAGE_PAD/LEAF_PAD/
// STEP/PERIOD/STEP2/D2LZ_*/PFB/WALIGN/PALIGN; (2918) d2zp_ 23 shots -
// ASTRIDE/BSTRIDE/ALIGN*/GAP*/PFPSTEP/PFPB2/loop order; (2931) d2zu_ 17
// interaction shots -- the parameter surface is separable/closed) and the
// prefetch census (2926)/(2935) -- the in-loop kernel prefetch family is
// board-sealed by (7263) d2zx_ (8 arms, 0 better).
// [3] Measuring instrument: `problems/mmmd2k/work/d2zz__tmp/d2zz_gate_tail.inc`
// (md5 245da34537cff03b65c3ec4eff803de2) = the team-certified FOUR-tier
// harness (n = 512/1024/2048/4096, the in-service anchors being
// 7235d94f/f52f809d/7e1f2b18/26d81789), referenced byte-for-byte.
// [4] Parameter tables (read-only): `problems/mmmd2k/work/d2zn__tmp/`,
// `d2zp__tmp/`, `d2zu__tmp/` (old base) and `d2zz__tmp/` (running new-base
// re-scan). This seat owns ONLY the families those tables do not contain;
// see `d5a__tmp/d5a_dol.md` for the zero-overlap proof.
// License: all of the above are public duck.ac submissions with no license
// statement; each is cited and attributed per RULES §3. No third-party code is
// reused here (generator, four-tier gate, non-empty-arm histogram: all self-written).
// ===== [d5a_ seat] IDEA -- board-truth sweep of the two families NO seat owns =====
// [Seat angle, the only one] value-neutral CONSTANTS that no earlier parameter
// seat has ever touched. d2zn_/d2zp_/d2zu_ walked the declared knobs
// (MC, plane spacing, panel strides, alignments, loop order, prefetch
// distances); d2zz_ is re-walking those same knobs on the new base #124155.
// Everything below is outside every one of those tables:
// (A) PK = the B-panel PACK path `d1a2_pk()` / `pk_fuse()`. The ledger names
// B-panel assembly as the weakest in-loop pool (~29.7 M cycles @n=2048,
// (2926) d2zs_), and NO seat has ever edited these two loops. Knob = the
// k-unroll factor of the pack (k+=2 -> k+=4 / k+=8): per (k-pair, column)
// the burst written is 16 doubles = 128 B; unrolling k makes it 256 B /
// 512 B contiguous, i.e. more full lines per write-combining window,
// while the b.x read stream stays sequential (k outer, j inner).
// (B) WK = the work-region offset constants INSIDE `rec()`. d2zp_/d2zu_
// moved the STATIC arrays (ap/ap2/bp/pk_zero) by 1 line / 1 page; nobody
// has ever moved the DYNAMIC work planes. Dead-branch analysis (see
// d5a_dol.md: `p`/`q`/`sum()`/`combine()`/`kernel2`/`pend2_*` are
// unreachable at n=2048) leaves exactly four live offsets:
// `sub=pb2+pz` -> the child region base (+64/+512/+1536 d)
// `m` at n==2*BASE -> the 3 pending planes (+512 d)
// `m` at n>2*BASE -> rigid shift of the work block (+512 d)
// the pa0..pb2 plane-offset assignment (permutation, sub fixed)
// [Value-neutrality] (A) writes the same values to the same disjoint slots
// (slot of (k,j) is `BP + j*(n+1) + k*8 + [0,16)`, pairwise disjoint; the
// k-unroll only reorders which (k,j) is visited when). (B) offsets are pure
// address arithmetic on scratch planes; the permutation keeps all five
// offsets distinct and keeps `sub` above all of them. Neither family touches
// any `kernel` argument, the k reduction order, or any Add/Mul sequence
// => the four-tier bit-equivalence gate (n=512/1024/2048/4096) must pass on
// all four anchors before the arm is allowed anywhere near the board.
// [Instrument] verdict = the duck.ac judge (board) only; same-code re-roll
// spread 0.132741 ms; a cross-shot difference is called real at >1.8 ms
// (code-size-changing arms only). No threshold stricter than (T*0.99+1us)
// is ever self-imposed (law 294). `/status` is read before every shot
// (Pending > 1 => no shot).
static constexpr int d2k_z7_p3 = (2 - 2);
#include <immintrin.h>
#include <stddef.h>
#include <cstring>
#define BASE 128
alignas(4096) static double qa[2048*2048],qb[2048*2048],qc[2048*2048],scratch[8*1024*1024];
alignas(64) static double packed_b[BASE*(BASE+1)];
static __attribute__((always_inline)) inline void kernel_plain(int n,const double *a,const double *b,double *c,int ldc,int rows) {
double *out=c;
ptrdiff_t stride=(ptrdiff_t)ldc*8;
int k=n; long ip=-(long)(n/8)*64; (void)k;
asm volatile (
"vxorpd %%ymm0, %%ymm0, %%ymm0\n\t"
"vxorpd %%ymm1, %%ymm1, %%ymm1\n\t"
"vxorpd %%ymm2, %%ymm2, %%ymm2\n\t"
"vxorpd %%ymm3, %%ymm3, %%ymm3\n\t"
"vxorpd %%ymm4, %%ymm4, %%ymm4\n\t"
"vxorpd %%ymm5, %%ymm5, %%ymm5\n\t"
"vxorpd %%ymm6, %%ymm6, %%ymm6\n\t"
"vxorpd %%ymm7, %%ymm7, %%ymm7\n\t"
"vxorpd %%ymm8, %%ymm8, %%ymm8\n\t"
"vxorpd %%ymm9, %%ymm9, %%ymm9\n\t"
"vxorpd %%ymm10, %%ymm10, %%ymm10\n\t"
"vxorpd %%ymm11, %%ymm11, %%ymm11\n\t"
".p2align 6\n\t"
"1:\n\t"
"vmovapd 8192(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8224(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 8704(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1024(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8256(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8288(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 8768(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1032(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8320(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8352(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 8832(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1040(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8384(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8416(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 8896(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1048(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8448(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8480(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 8960(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1056(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8512(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8544(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 9024(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1064(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8576(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8608(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 9088(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1072(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"vmovapd 8640(%[b],%[i],8), %%ymm12\n\t"
"vmovapd 8672(%[b],%[i],8), %%ymm13\n\t"
"vbroadcastsd %c[o0]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"prefetcht0 9152(%[b],%[i],8)\n\t"
"vbroadcastsd %c[o2]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t"
"vbroadcastsd %c[o3]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t"
"vbroadcastsd %c[o4]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t"
"vbroadcastsd %c[o5]+1080(%[a],%[i]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t"
"add $64, %[i]\n\t"
"jnz 1b\n\t"
"vmovapd %%ymm0, (%[out])\n\t"
"vmovapd %%ymm1, 32(%[out])\n\t"
"vmovapd %%ymm2, (%[out],%[stride])\n\t"
"vmovapd %%ymm3, 32(%[out],%[stride])\n\t"
"vmovapd %%ymm4, (%[out],%[stride],2)\n\t"
"vmovapd %%ymm5, 32(%[out],%[stride],2)\n\t"
"lea (%[out],%[stride],2), %[out]\n\t"
"add %[stride], %[out]\n\t"
"vmovapd %%ymm6, (%[out])\n\t"
"vmovapd %%ymm7, 32(%[out])\n\t"
"vmovapd %%ymm8, (%[out],%[stride])\n\t"
"vmovapd %%ymm9, 32(%[out],%[stride])\n\t"
"vmovapd %%ymm10, (%[out],%[stride],2)\n\t"
"vmovapd %%ymm11, 32(%[out],%[stride],2)\n\t"
: [a] "+&r"(a), [b] "+&r"(b), [out] "+&r"(out), [k] "+&r"(k), [i] "+&r"(ip)
: [stride] "r"(stride), [o0] "i"(0*BASE*8), [o1] "i"(1*BASE*8), [o2] "i"(2*BASE*8), [o3] "i"(3*BASE*8), [o4] "i"(4*BASE*8), [o5] "i"(5*BASE*8)
: "memory", "cc", "ymm0","ymm1","ymm2","ymm3","ymm4","ymm5","ymm6","ymm7","ymm8","ymm9","ymm10","ymm11","ymm12","ymm13","ymm14","ymm15");
}
static __attribute__((always_inline)) inline void kernel2(int n,const double *a,const double *b,double *c,int ldc) {
int k=n;
double *out=c;
ptrdiff_t stride=(ptrdiff_t)ldc*8;
asm volatile (
"vxorpd %%ymm0, %%ymm0, %%ymm0\n\t"
"vxorpd %%ymm1, %%ymm1, %%ymm1\n\t"
"vxorpd %%ymm2, %%ymm2, %%ymm2\n\t"
"vxorpd %%ymm3, %%ymm3, %%ymm3\n\t"
".p2align 6\n\t" "1:\n\t"
"prefetcht0 512(%[b])\n\t"
"prefetcht0 576(%[b])\n\t"
"prefetcht0 640(%[b])\n\t"
"prefetcht0 704(%[b])\n\t"
"vmovapd 0(%[b]), %%ymm12\n\t"
"vmovapd 32(%[b]), %%ymm13\n\t"
"vbroadcastsd %c[o0]+0(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+0(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"vmovapd 64(%[b]), %%ymm12\n\t"
"vmovapd 96(%[b]), %%ymm13\n\t"
"vbroadcastsd %c[o0]+8(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+8(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"vmovapd 128(%[b]), %%ymm12\n\t"
"vmovapd 160(%[b]), %%ymm13\n\t"
"vbroadcastsd %c[o0]+16(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+16(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"vmovapd 192(%[b]), %%ymm12\n\t"
"vmovapd 224(%[b]), %%ymm13\n\t"
"vbroadcastsd %c[o0]+24(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t"
"vbroadcastsd %c[o1]+24(%[a]), %%ymm14\n\t"
"vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t"
"vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t"
"add $32, %[a]\n\t"
"add $256, %[b]\n\t"
"sub $4, %[k]\n\t"
"jnz 1b\n\t"
"vmovapd %%ymm0, (%[out])\n\t"
"vmovapd %%ymm1, 32(%[out])\n\t"
"add %[stride], %[out]\n\t"
"vmovapd %%ymm2, (%[out])\n\t"
"vmovapd %%ymm3, 32(%[out])\n\t"
: [a] "+&r"(a), [b] "+&r"(b), [out] "+&r"(out), [k] "+&r"(k)
: [stride] "r"(stride), [o0] "i"(0*BASE*8), [o1] "i"(1*BASE*8)
: "memory", "cc", "ymm0","ymm1","ymm2","ymm3","ymm12","ymm13","ymm14");
}
static __attribute__((noinline)) void leaf128(const double*a,const double*b,double*c){
for(int k=0;k<BASE;k+=2)for(int j=0;j<BASE;j+=8){
double*p=packed_b+(size_t)j*(BASE+1)+k*8;
_mm256_store_pd(p,_mm256_load_pd(b+(size_t)k*BASE+j));
_mm256_store_pd(p+4,_mm256_load_pd(b+(size_t)k*BASE+j+4));
_mm256_store_pd(p+8,_mm256_load_pd(b+(size_t)(k+1)*BASE+j));
_mm256_store_pd(p+12,_mm256_load_pd(b+(size_t)(k+1)*BASE+j+4));
}
for(int j=0;j<BASE;j+=8){
for(int r=0;r+6<=BASE;r+=6)kernel_plain(BASE,a+(size_t)r*BASE,packed_b+(size_t)j*(BASE+1),c+(size_t)r*BASE+j,BASE,6);
kernel2(BASE,a+(size_t)(BASE-2)*BASE,packed_b+(size_t)j*(BASE+1),c+(size_t)(BASE-2)*BASE+j,BASE);
}
}
static void pack(int n,const double*s,int ld,double*d){
if(n==BASE){for(int i=0;i<BASE;i++)for(int j=0;j<BASE;j+=8){_mm256_stream_pd(d+(size_t)i*BASE+j,_mm256_load_pd(s+(size_t)i*ld+j));_mm256_stream_pd(d+(size_t)i*BASE+j+4,_mm256_load_pd(s+(size_t)i*ld+j+4));}return;}
int h=n/2;size_t z=(size_t)h*h;
pack(h,s,ld,d);pack(h,s+h,ld,d+z);pack(h,s+(size_t)h*ld,ld,d+2*z);pack(h,s+(size_t)h*ld+h,ld,d+3*z);
}
static void unpack(int n,const double*s,double*d,int ld){
if(n==BASE){for(int i=0;i<BASE;i++)for(int j=0;j<BASE;j+=8){_mm256_stream_pd(d+(size_t)i*ld+j,_mm256_load_pd(s+(size_t)i*BASE+j));_mm256_stream_pd(d+(size_t)i*ld+j+4,_mm256_load_pd(s+(size_t)i*BASE+j+4));}return;}
int h=n/2;size_t z=(size_t)h*h;
unpack(h,s,d,ld);unpack(h,s+z,d+h,ld);unpack(h,s+2*z,d+(size_t)h*ld,ld);unpack(h,s+3*z,d+(size_t)h*ld+h,ld);
}
static void multiply(int n,const double*a,const double*b,double*c,double*w){
if(n==BASE){leaf128(a,b,c);return;}
int h=n/2;size_t z=(size_t)h*h;
const double*a11=a,*a12=a+z,*a21=a+2*z,*a22=a+3*z;
const double*b11=b,*b12=b+z,*b21=b+2*z,*b22=b+3*z;
double*s1=w,*s2=w+z,*t1=w+2*z,*t2=w+3*z,*t3=w+4*z,*next=w+5*z;
for(size_t i=0;i<z;i+=4){
__m256d aa=_mm256_load_pd(a11+i),ab=_mm256_load_pd(a12+i),ac=_mm256_load_pd(a21+i),ad=_mm256_load_pd(a22+i);
__m256d p=_mm256_add_pd(ac,ad),q=_mm256_sub_pd(p,aa),r=_mm256_sub_pd(ab,q),v=_mm256_sub_pd(aa,ac);
_mm256_store_pd(s1+i,p);_mm256_store_pd(s2+i,q);_mm256_store_pd(c+3*z+i,r);_mm256_store_pd(c+i,v);
}
for(size_t i=0;i<z;i+=4){
__m256d aa=_mm256_load_pd(b11+i),ab=_mm256_load_pd(b12+i),ac=_mm256_load_pd(b21+i),ad=_mm256_load_pd(b22+i);
__m256d p=_mm256_sub_pd(ab,aa),q=_mm256_sub_pd(ad,p),r=_mm256_sub_pd(q,ac),v=_mm256_sub_pd(ad,ab);
_mm256_store_pd(t1+i,p);_mm256_store_pd(t2+i,q);_mm256_store_pd(c+2*z+i,v);_mm256_store_pd(t3+i,r);
}
multiply(h,c+3*z,b22,c+z,next);
multiply(h,a22,t3,c+3*z,next);
multiply(h,s1,t1,t3,next);
multiply(h,s2,t2,s1,next);
multiply(h,c,c+2*z,s2,next);
multiply(h,a11,b11,t1,next);
multiply(h,a12,b21,c,next);
for(size_t i=0;i<z;i+=4){
__m256d p1=_mm256_load_pd(t1+i),p2=_mm256_load_pd(c+i),p3=_mm256_load_pd(c+z+i),p4=_mm256_load_pd(c+3*z+i),p5=_mm256_load_pd(t3+i),p6=_mm256_load_pd(s1+i),p7=_mm256_load_pd(s2+i);
__m256d u2=_mm256_add_pd(p1,p6),u3=_mm256_add_pd(u2,p7),u4=_mm256_add_pd(u2,p5);
_mm256_store_pd(c+i,_mm256_add_pd(p1,p2));
_mm256_store_pd(c+z+i,_mm256_add_pd(u4,p3));
_mm256_store_pd(c+2*z+i,_mm256_sub_pd(u3,p4));
_mm256_store_pd(c+3*z+i,_mm256_add_pd(u3,p5));
}
}
void matrix_multiply(int n,const double*a,const double*b,double*c){pack(n,a,n,qa);pack(n,b,n,qb);_mm_sfence();multiply(n,qa,qb,qc,scratch);unpack(n,qc,c,n);_mm_sfence();}
| Compilation | N/A | N/A | Compile OK | Score: N/A | 显示更多 |
| Testcase #1 | 363.987 ms | 181 MB + 272 KB | Accepted | Score: 100 | 显示更多 |