提交记录 107382 - Judge Duck Online

用户	题目	状态	得分	用时	内存	语言	代码长度
pdoom	mmmd4k. 测测你的双精度矩阵乘法-4k	Accepted	100	1.922 s	473428 KB	C++17	94.57 KB

提交时间	评测时间
2026-09-28 20:47:42	2026-09-28 20:47:55

代码

/*
Credit: Dumas et al. rank48 circuit and saffah_cc_v41_agg1 #105696 /
saffah_codex_6s_agg2 #104821 leaf-engine improvements; full original credits
follow. New integration: preserve the verified rank48 DAG and slots while
using the produced-row frontier, reordered Strassen products, eight-step
plain kernel, two-row B packing and newer panel strides inside each1024
product. The upper Winograd pending task is inapplicable and omitted.
*/
/*
Final rank48 slot-layout experiment: fifteen untouched 4KiB pages
between the forty 1024-square scratch slots, with arena capacity and
child recursion workspace adjusted to the same slot pitch.
All existing algorithm and engine credits are retained below.
*/
/*
Pointwise register-lifetime experiment: reorder independent nodes inside
existing fusion groups. Original external-read-before-slot-overwrite edges
are preserved, as are expressions, full-line NT stores, matrix transfers,
slot layout, product order, and pending-combine boundaries.
*/
/*
Credit: Jean-Guillaume Dumas, Clement Pernet, Alexandre Sedoglavic,
Petr Tichavsky, arXiv:2609.12027 (2026), canonical rank-48 algorithm.
https://arxiv.org/abs/2609.12027
Canonical L/R/P straight-line programs: https://github.com/jgdumas/plinopt
This prototype replaces the upper two Winograd levels with their rank48
SLP, using a joint dependency schedule and 56 colored storage slots.
Sixteen slots borrow C before becoming the final C blocks; the other 40
slots occupy 320 MiB plus optional page padding. Bottom recursion remains the credited engine below.
Linear operations are fused in the reordered DAG; product reconstruction
is deferred safely until the pending product is first consumed.
Pointwise sources have sign=0; this variant uses the inherited Mat loader.
*/
/*
Credit: saffah_cc_v41_agg1, https://duck.ac/submission/100776,
for specializing the final two rows instead of computing six padded rows.
This version starts from that submission; all earlier engine credits follow.

New scheduling and kernel:
- Merge the final two rows into the preceding 30-row block.
- Process the resulting 32-row block in pairs of B panels: ten 6x8
  kernels followed by one 2x16 kernel while both panels remain in L2.
- The 2x16 tail uses eight independent accumulators, retaining the
  original per-element double-precision FMA order along K.
*/
/*
Credits: inherited engine credits are retained immediately below.
New scheduling: defer a completed last-level Strassen combination and
prefetch/combine small pieces between the following sibling's kernels.
A second three-product buffer prevents overwriting pending operands.
*/
/*
Credits:
- saffah_cc_v41_agg1, https://duck.ac/submission/100511 and
  https://duck.ac/submission/100475: carry next A packing inside the FMA
  loop, double-buffer A, and pack B in groups of eight K rows. This code
  starts from #100511, itself based on pdoom #100193.
- Earlier credits from pdoom #100193 are retained below.
Changes in this version:
- Specialize copy/add/subtract A carry at compile time; move one 32-byte
  vector per eight K steps, without a gating branch or sign mask.
- Carry two half-row chunks per next A row, during the final kernel calls
  of the current block. Use MC=30 to limit cache pressure.
- Give calls without packing their own plain four-step kernel, avoiding
  unused packing pointer operands and their register constraints.
*/
/*
Credits:
- saffah_cc_v41_260924, https://duck.ac/submission/99846:
  fused leaf operand sums/packing, one-pass operand-sum generation, and
  one-pass four-quadrant Strassen combine.
- saffah_cc_v41_agg1, https://duck.ac/submission/99875 and
  https://duck.ac/submission/99950: non-temporal intermediate/output stores.
- saffah_cc_v41_260924, https://duck.ac/submission/99141, and
  saffah_cc_v41_agg1, https://duck.ac/submission/99167:
  double-precision 6x8 blocking, B-panel prefetch, explicit 256-bit memory instructions, and
  row-contiguous A packing and row-first B packing were consulted for this problem's baseline.
- pdoom, https://duck.ac/submission/99966: preceding packing/assembly framework.
The double microkernel, output-backed temporary placement, and hybrid
Strassen/Winograd recursion below were implemented here.
New scheduling: precompute all Winograd operand sums in one traversal.
The third sum of each operand temporarily occupies an output quadrant;
M7 is evaluated first to free those quadrants before the other products.
No quantization is used: arithmetic remains binary64.
*/
#define Q0 _mm256_stream_pd
#pragma GCC optimize("O3,unroll-loops,no-strict-aliasing")
#include <sys/mman.h>
#pragma GCC target("avx2,fma,arch=skylake")
#include <immintrin.h>
#ifndef B2M_NO_BARA
#define B2M_NO_BARA 1
#endif
#define MD9_T2BASE 1
#include <stddef.h>
#ifndef BASE
#define BASE 512
#endif
#ifndef MC
#define MC 30
#endif
#ifndef NTMIN
#define NTMIN 512
#endif
#ifndef UPPER_PAGE_PAD
#define UPPER_PAGE_PAD 15872
#endif
#ifndef LEAF_PAD
#define LEAF_PAD 1152
#endif
#ifndef SUMNT
#define SUMNT 1024
#endif
static constexpr int MR=6,NR=8;alignas(4096)static double workspace[40*(1024*1024+7680)+6*BASE*BASE+4*LEAF_PAD+64];alignas(64)static double ap[(MC+MR)*(BASE+24)],ap2[(MC+MR)*(BASE+24)],bp[BASE*(BASE+24)];alignas(64)static double pk_zero[BASE+8];alignas(64)static double pk_mp[4]={0.0,0.0,0.0,0.0},pk_mn[4]={-0.0,-0.0,-0.0,-0.0};struct Mat{const double*x;int ld;const double*y=nullptr;int ly=0;int sign=0;};static inline __m256d ld(Mat a,int i,int j){__m256d x=_mm256_load_pd(a.x+(size_t)i*a.ld+j);if(a.sign){__m256d y=_mm256_load_pd(a.y+(size_t)i*a.ly+j);x=a.sign>0?_mm256_add_pd(x,y):_mm256_sub_pd(x,y);}return x;}static inline double scalar(Mat a,int i,int k){double x=a.x[(size_t)i*a.ld+k];if(a.sign)x+=a.sign>0?a.y[(size_t)i*a.ly+k]:-a.y[(size_t)i*a.ly+k];return x;}template<int SG>static __attribute__((always_inline))inline void kernel(int n,const double*a,const double*b,double*c,int ldc,int rows,const double*pks,const double*pky,double*pkd){alignas(32)double tmp[MR*8];double*out=rows==MR?c:tmp;ptrdiff_t stride=rows==MR?ldc*8:64;int k=n;asm volatile("vxorpd %%ymm0, %%ymm0, %%ymm0\n\t""vxorpd %%ymm1, %%ymm1, %%ymm1\n\t""vxorpd %%ymm2, %%ymm2, %%ymm2\n\t""vxorpd %%ymm3, %%ymm3, %%ymm3\n\t""vxorpd %%ymm4, %%ymm4, %%ymm4\n\t""vxorpd %%ymm5, %%ymm5, %%ymm5\n\t""vxorpd %%ymm6, %%ymm6, %%ymm6\n\t""vxorpd %%ymm7, %%ymm7, %%ymm7\n\t""vxorpd %%ymm8, %%ymm8, %%ymm8\n\t""vxorpd %%ymm9, %%ymm9, %%ymm9\n\t""vxorpd %%ymm10, %%ymm10, %%ymm10\n\t""vxorpd %%ymm11, %%ymm11, %%ymm11\n\t"".p2align 6\n\t""1:\n\t"".if %c[sg] != 0\n\t""prefetcht0 512(%[b])\n\t""prefetcht0 576(%[b])\n\t""prefetcht0 640(%[b])\n\t""prefetcht0 704(%[b])\n\t""vmovapd 0(%[b]), %%ymm12\n\t""vmovapd 32(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 64(%[b]), %%ymm12\n\t""vmovapd 96(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 128(%[b]), %%ymm12\n\t""vmovapd 160(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 192(%[b]), %%ymm12\n\t""vmovapd 224(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""prefetcht0 768(%[b])\n\t""prefetcht0 832(%[b])\n\t""prefetcht0 896(%[b])\n\t""prefetcht0 960(%[b])\n\t""vmovapd 256(%[b]), %%ymm12\n\t""vmovapd 288(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 320(%[b]), %%ymm12\n\t""vmovapd 352(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 384(%[b]), %%ymm12\n\t""vmovapd 416(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 448(%[b]), %%ymm12\n\t""vmovapd 480(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovupd 0(%[pks]), %%ymm15\n\t"".if %c[sg] == 1\n\t""vaddpd 0(%[pky]), %%ymm15, %%ymm15\n\t"".elseif %c[sg] == -1\n\t""vsubpd 0(%[pky]), %%ymm15, %%ymm15\n\t"".endif\n\t""vmovupd %%ymm15, 0(%[pkd])\n\t""add $32, %[pks]\n\t""add $32, %[pky]\n\t""add $32, %[pkd]\n\t""add $64, %[a]\n\t""add $512, %[b]\n\t""sub $8, %[k]\n\t"".else\n\t""prefetcht0 512(%[b])\n\t""prefetcht0 576(%[b])\n\t""prefetcht0 640(%[b])\n\t""prefetcht0 704(%[b])\n\t""vmovapd 0(%[b]), %%ymm12\n\t""vmovapd 32(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 64(%[b]), %%ymm12\n\t""vmovapd 96(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 128(%[b]), %%ymm12\n\t""vmovapd 160(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 192(%[b]), %%ymm12\n\t""vmovapd 224(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""add $32, %[a]\n\t""add $256, %[b]\n\t""sub $4, %[k]\n\t"".endif\n\t""jnz 1b\n\t""test %[nt], %[nt]\n\t""jz 2f\n\t""vmovntpd %%ymm0, (%[out])\n\t""vmovntpd %%ymm1, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm2, (%[out])\n\t""vmovntpd %%ymm3, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm4, (%[out])\n\t""vmovntpd %%ymm5, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm6, (%[out])\n\t""vmovntpd %%ymm7, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm8, (%[out])\n\t""vmovntpd %%ymm9, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm10, (%[out])\n\t""vmovntpd %%ymm11, 32(%[out])\n\t""jmp 3f\n\t""2:\n\t""vmovapd %%ymm0, (%[out])\n\t""vmovapd %%ymm1, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm2, (%[out])\n\t""vmovapd %%ymm3, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm4, (%[out])\n\t""vmovapd %%ymm5, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm6, (%[out])\n\t""vmovapd %%ymm7, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm8, (%[out])\n\t""vmovapd %%ymm9, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm10, (%[out])\n\t""vmovapd %%ymm11, 32(%[out])\n\t""3:\n\t":[a]"+&r"(a),[b]"+&r"(b),[out]"+&r"(out),[k]"+&r"(k),[pks]"+&r"(pks),[pky]"+&r"(pky),[pkd]"+&r"(pkd):[stride]"r"(stride),[nt]"r"(int(n>=NTMIN&&rows==MR)),[sg]"i"(SG),[o0]"i"(0*(BASE+24)*8),[o1]"i"(1*(BASE+24)*8),[o2]"i"(2*(BASE+24)*8),[o3]"i"(3*(BASE+24)*8),[o4]"i"(4*(BASE+24)*8),[o5]"i"(5*(BASE+24)*8):"memory","cc","ymm0","ymm1","ymm2","ymm3","ymm4","ymm5","ymm6","ymm7","ymm8","ymm9","ymm10","ymm11","ymm12","ymm13","ymm14","ymm15");if(rows!=MR)for(int i=0;i<rows;i++)for(int j=0;j<8;j+=4)_mm256_store_pd(c+(size_t)i*ldc+j,_mm256_load_pd(tmp+i*8+j));}static __attribute__((always_inline))inline void kernel_plain(int n,const double*a,const double*b,double*c,int ldc,int rows){alignas(32)double tmp[MR*8];double*out=rows==MR?c:tmp;ptrdiff_t stride=rows==MR?ldc*8:64;int k=n;asm volatile("vxorpd %%ymm0, %%ymm0, %%ymm0\n\t""vxorpd %%ymm1, %%ymm1, %%ymm1\n\t""vxorpd %%ymm2, %%ymm2, %%ymm2\n\t""vxorpd %%ymm3, %%ymm3, %%ymm3\n\t""vxorpd %%ymm4, %%ymm4, %%ymm4\n\t""vxorpd %%ymm5, %%ymm5, %%ymm5\n\t""vxorpd %%ymm6, %%ymm6, %%ymm6\n\t""vxorpd %%ymm7, %%ymm7, %%ymm7\n\t""vxorpd %%ymm8, %%ymm8, %%ymm8\n\t""vxorpd %%ymm9, %%ymm9, %%ymm9\n\t""vxorpd %%ymm10, %%ymm10, %%ymm10\n\t""vxorpd %%ymm11, %%ymm11, %%ymm11\n\t"".p2align 6\n\t""1:\n\t""prefetcht0 512(%[b])\n\t""prefetcht0 576(%[b])\n\t""prefetcht0 640(%[b])\n\t""prefetcht0 704(%[b])\n\t""vmovapd 0(%[b]), %%ymm12\n\t""vmovapd 32(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 64(%[b]), %%ymm12\n\t""vmovapd 96(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 128(%[b]), %%ymm12\n\t""vmovapd 160(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 192(%[b]), %%ymm12\n\t""vmovapd 224(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""prefetcht0 768(%[b])\n\t""prefetcht0 832(%[b])\n\t""prefetcht0 896(%[b])\n\t""prefetcht0 960(%[b])\n\t""vmovapd 256(%[b]), %%ymm12\n\t""vmovapd 288(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+32(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 320(%[b]), %%ymm12\n\t""vmovapd 352(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+40(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 384(%[b]), %%ymm12\n\t""vmovapd 416(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+48(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""vmovapd 448(%[b]), %%ymm12\n\t""vmovapd 480(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vbroadcastsd %c[o2]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm4\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm5\n\t""vbroadcastsd %c[o3]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm6\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm7\n\t""vbroadcastsd %c[o4]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm8\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm9\n\t""vbroadcastsd %c[o5]+56(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm10\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm11\n\t""add $64, %[a]\n\t""add $512, %[b]\n\t""sub $8, %[k]\n\t""jnz 1b\n\t""test %[nt], %[nt]\n\t""jz 2f\n\t""vmovntpd %%ymm0, (%[out])\n\t""vmovntpd %%ymm1, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm2, (%[out])\n\t""vmovntpd %%ymm3, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm4, (%[out])\n\t""vmovntpd %%ymm5, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm6, (%[out])\n\t""vmovntpd %%ymm7, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm8, (%[out])\n\t""vmovntpd %%ymm9, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm10, (%[out])\n\t""vmovntpd %%ymm11, 32(%[out])\n\t""jmp 3f\n\t""2:\n\t""vmovapd %%ymm0, (%[out])\n\t""vmovapd %%ymm1, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm2, (%[out])\n\t""vmovapd %%ymm3, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm4, (%[out])\n\t""vmovapd %%ymm5, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm6, (%[out])\n\t""vmovapd %%ymm7, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm8, (%[out])\n\t""vmovapd %%ymm9, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovapd %%ymm10, (%[out])\n\t""vmovapd %%ymm11, 32(%[out])\n\t""3:\n\t":[a]"+&r"(a),[b]"+&r"(b),[out]"+&r"(out),[k]"+&r"(k):[stride]"r"(stride),[nt]"r"(int(n>=NTMIN&&rows==MR)),[o0]"i"(0*(BASE+24)*8),[o1]"i"(1*(BASE+24)*8),[o2]"i"(2*(BASE+24)*8),[o3]"i"(3*(BASE+24)*8),[o4]"i"(4*(BASE+24)*8),[o5]"i"(5*(BASE+24)*8):"memory","cc","ymm0","ymm1","ymm2","ymm3","ymm4","ymm5","ymm6","ymm7","ymm8","ymm9","ymm10","ymm11","ymm12","ymm13","ymm14","ymm15");if(rows!=MR)for(int i=0;i<rows;i++)for(int j=0;j<8;j+=4)_mm256_store_pd(c+(size_t)i*ldc+j,_mm256_load_pd(tmp+i*8+j));}struct Pending{double*c;int ldc;const double*m;int pos=0;bool active=false;int rowlim=0;bool prog=false;};static Pending pending;alignas(4096)static double pending_work[3*(BASE*BASE+LEAF_PAD)+LEAF_PAD];static unsigned pending_tick=0;static constexpr int STEP=32,PERIOD=1;static inline void pending_prefetch(){if(!pending.active)return;if(pending.prog&&pending.pos/BASE>=pending.rowlim)return;int i=pending.pos/BASE,j=pending.pos%BASE;const double*c=pending.c+(size_t)i*pending.ldc+j;const double*m=pending.m+pending.pos;for(int t=0;t<STEP;t+=8){_mm_prefetch((const char*)(c+t),_MM_HINT_T1);_mm_prefetch((const char*)(c+BASE+t),_MM_HINT_T1);_mm_prefetch((const char*)(c+(size_t)BASE*pending.ldc+t),_MM_HINT_T1);_mm_prefetch((const char*)(c+(size_t)BASE*pending.ldc+BASE+t),_MM_HINT_T1);_mm_prefetch((const char*)(m+t),_MM_HINT_T1);_mm_prefetch((const char*)(m+(BASE*BASE+LEAF_PAD)+t),_MM_HINT_T1);_mm_prefetch((const char*)(m+2*(BASE*BASE+LEAF_PAD)+t),_MM_HINT_T1);}}static inline void pending_step(){if(pending.prog&&pending.pos/BASE>=pending.rowlim)return;int i=pending.pos/BASE,j=pending.pos%BASE;for(int t=0;t<STEP;t+=4){double*c11=pending.c+(size_t)i*pending.ldc+j+t,*c12=c11+BASE,*c21=c11+(size_t)BASE*pending.ldc,*c22=c21+BASE;const double*m=pending.m+pending.pos+t;__m256d m1=_mm256_load_pd(c11),m2=_mm256_load_pd(c21),m3=_mm256_load_pd(c12),m4=_mm256_load_pd(c22),m5=_mm256_load_pd(m),m6=_mm256_load_pd(m+(BASE*BASE+LEAF_PAD)),m7=_mm256_load_pd(m+2*(BASE*BASE+LEAF_PAD));__m256d r11=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(m1,m4),m5),m7),r12=_mm256_add_pd(m3,m5),r21=_mm256_add_pd(m2,m4),r22=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(m1,m2),m3),m6);_mm256_store_pd(c11,r11);_mm256_store_pd(c12,r12);_mm256_store_pd(c21,r21);_mm256_store_pd(c22,r22);}pending.pos+=STEP;if(pending.pos==BASE*BASE)pending.active=false;}static void pending_finish(){pending.prog=false;while(pending.active)pending_step();}static __attribute__((always_inline))inline void kernel2(int n,const double*a,const double*b,double*c,int ldc){int k=n;double*out=c;ptrdiff_t stride=(ptrdiff_t)ldc*8;asm volatile("vxorpd %%ymm0, %%ymm0, %%ymm0\n\t""vxorpd %%ymm1, %%ymm1, %%ymm1\n\t""vxorpd %%ymm2, %%ymm2, %%ymm2\n\t""vxorpd %%ymm3, %%ymm3, %%ymm3\n\t"".p2align 6\n\t""1:\n\t""prefetcht0 512(%[b])\n\t""prefetcht0 576(%[b])\n\t""prefetcht0 640(%[b])\n\t""prefetcht0 704(%[b])\n\t""vmovapd 0(%[b]), %%ymm12\n\t""vmovapd 32(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+0(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vmovapd 64(%[b]), %%ymm12\n\t""vmovapd 96(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+8(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vmovapd 128(%[b]), %%ymm12\n\t""vmovapd 160(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+16(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""vmovapd 192(%[b]), %%ymm12\n\t""vmovapd 224(%[b]), %%ymm13\n\t""vbroadcastsd %c[o0]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm0\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm1\n\t""vbroadcastsd %c[o1]+24(%[a]), %%ymm14\n\t""vfmadd231pd %%ymm12, %%ymm14, %%ymm2\n\t""vfmadd231pd %%ymm13, %%ymm14, %%ymm3\n\t""add $32, %[a]\n\t""add $256, %[b]\n\t""sub $4, %[k]\n\t""jnz 1b\n\t""vmovntpd %%ymm0, (%[out])\n\t""vmovntpd %%ymm1, 32(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm2, (%[out])\n\t""vmovntpd %%ymm3, 32(%[out])\n\t":[a]"+&r"(a),[b]"+&r"(b),[out]"+&r"(out),[k]"+&r"(k):[stride]"r"(stride),[o0]"i"(0*(BASE+24)*8),[o1]"i"(1*(BASE+24)*8):"memory","cc","ymm0","ymm1","ymm2","ymm3","ymm12","ymm13","ymm14");}static __attribute__((noinline))void kernel_tail16(int n,const double*a,const double*b,double*out,int ldc){const double*b1=b+8*(n+1);ptrdiff_t stride=(ptrdiff_t)ldc*8;int kk=n;asm volatile("vxorpd %%ymm0, %%ymm0, %%ymm0\n\t""vxorpd %%ymm1, %%ymm1, %%ymm1\n\t""vxorpd %%ymm2, %%ymm2, %%ymm2\n\t""vxorpd %%ymm3, %%ymm3, %%ymm3\n\t""vxorpd %%ymm4, %%ymm4, %%ymm4\n\t""vxorpd %%ymm5, %%ymm5, %%ymm5\n\t""vxorpd %%ymm6, %%ymm6, %%ymm6\n\t""vxorpd %%ymm7, %%ymm7, %%ymm7\n\t"".p2align 6\n\t""1:\n\t""prefetcht0 512(%[b])\n\t""prefetcht0 512(%[b1])\n\t""vbroadcastsd %c[o0]+0(%[a]), %%ymm12\n\t""vbroadcastsd %c[o1]+0(%[a]), %%ymm13\n\t""vmovapd 0(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm0\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm4\n\t""vmovapd 32(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm1\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm5\n\t""vmovapd 0(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm2\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm6\n\t""vmovapd 32(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm3\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm7\n\t""prefetcht0 576(%[b])\n\t""prefetcht0 576(%[b1])\n\t""vbroadcastsd %c[o0]+8(%[a]), %%ymm12\n\t""vbroadcastsd %c[o1]+8(%[a]), %%ymm13\n\t""vmovapd 64(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm0\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm4\n\t""vmovapd 96(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm1\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm5\n\t""vmovapd 64(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm2\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm6\n\t""vmovapd 96(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm3\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm7\n\t""prefetcht0 640(%[b])\n\t""prefetcht0 640(%[b1])\n\t""vbroadcastsd %c[o0]+16(%[a]), %%ymm12\n\t""vbroadcastsd %c[o1]+16(%[a]), %%ymm13\n\t""vmovapd 128(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm0\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm4\n\t""vmovapd 160(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm1\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm5\n\t""vmovapd 128(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm2\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm6\n\t""vmovapd 160(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm3\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm7\n\t""prefetcht0 704(%[b])\n\t""prefetcht0 704(%[b1])\n\t""vbroadcastsd %c[o0]+24(%[a]), %%ymm12\n\t""vbroadcastsd %c[o1]+24(%[a]), %%ymm13\n\t""vmovapd 192(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm0\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm4\n\t""vmovapd 224(%[b]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm1\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm5\n\t""vmovapd 192(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm2\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm6\n\t""vmovapd 224(%[b1]), %%ymm14\n\t""vfmadd231pd %%ymm14, %%ymm12, %%ymm3\n\t""vfmadd231pd %%ymm14, %%ymm13, %%ymm7\n\t""add $32, %[a]\n\t""add $256, %[b]\n\t""add $256, %[b1]\n\t""sub $4, %[kk]\n\t""jnz 1b\n\t""vmovntpd %%ymm0, 0(%[out])\n\t""vmovntpd %%ymm1, 32(%[out])\n\t""vmovntpd %%ymm2, 64(%[out])\n\t""vmovntpd %%ymm3, 96(%[out])\n\t""add %[stride], %[out]\n\t""vmovntpd %%ymm4, 0(%[out])\n\t""vmovntpd %%ymm5, 32(%[out])\n\t""vmovntpd %%ymm6, 64(%[out])\n\t""vmovntpd %%ymm7, 96(%[out])\n\t":[a]"+&r"(a),[b]"+&r"(b),[b1]"+&r"(b1),[out]"+&r"(out),[kk]"+&r"(kk):[stride]"r"(stride),[o0]"i"(0),[o1]"i"((BASE+24)*8):"memory","cc","ymm0","ymm1","ymm2","ymm3","ymm4","ymm5","ymm6","ymm7","ymm8","ymm9","ymm10","ymm11","ymm12","ymm13","ymm14","ymm15");}static void leaf(int n,Mat a,Mat b,double*c,int ldc){for(int k=0;k<n;k+=2)for(int j=0;j<n;j+=8){double*p=bp+(size_t)j*(n+1)+k*8;_mm256_store_pd(p+0,ld(b,k+0,j));_mm256_store_pd(p+4,ld(b,k+0,j+4));_mm256_store_pd(p+8,ld(b,k+1,j));_mm256_store_pd(p+12,ld(b,k+1,j+4));}pending_prefetch();double*pkb[2]={ap,ap2};int cur=0;{int mc0=n<MC?n:MC;for(int r=0;r<mc0;r+=MR){int rows=mc0-r<MR?mc0-r:MR;for(int i=0;i<rows;i++)for(int k=0;k<n;k+=4)_mm256_store_pd(pkb[0]+(size_t)(r+i)*(BASE+24)+k,ld(a,r+i,k));for(int i=rows;i<MR;i++)for(int k=0;k<n;k+=4)_mm256_store_pd(pkb[0]+(size_t)(r+i)*(BASE+24)+k,_mm256_setzero_pd());}}for(int ic=0;ic<n;){int mc=n-ic<=MC+MR?n-ic:MC;if(mc==MC+2){for(int j=0;j<n;j+=16){for(int jj=j;jj<j+16;jj+=NR)for(int r=0;r<MC;r+=MR){kernel_plain(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)jj*(n+1),c+(size_t)(ic+r)*ldc+jj,ldc,MR);if(pending.active&&(++pending_tick%PERIOD)==0){pending_step();if(pending.prog)pending_step();pending_prefetch();}}kernel_tail16(n,pkb[cur]+(size_t)MC*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+MC)*ldc+j,ldc);}ic+=mc;if(pending.prog)pending.rowlim=ic;continue;}const int ic2=ic+mc;const int nxt=(ic2<n)?(n-ic2<=MC+MR?n-ic2:MC):0;const int rows2=((nxt+MR-1)/MR)*MR;const int nch=rows2*2;const int qd=n/2;int ck=0;int skip=(n/NR)*((mc+MR-1)/MR)-nch;for(int j=0;j<n;j+=NR)for(int r=0;r<mc;r+=MR){if(mc-r==2){kernel2(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+r)*ldc+j,ldc);continue;}const double*pks=pk_zero,*pky=pk_zero,*pkm=pk_mp;double*pkd=pk_zero;int pkc=0;if(ck<nch&&--skip<0){const int row=ck/2,off=(ck%2)*qd;if(row<nxt){pks=a.x+(size_t)(ic2+row)*a.ld+(size_t)off;pky=a.sign?(a.y+(size_t)(ic2+row)*a.ly+(size_t)off):pk_zero;pkm=a.sign>0?pk_mp:pk_mn;}pkd=pkb[cur^1]+(size_t)row*(BASE+24)+(size_t)off;pkc=8;ck++;}if(!pkc)kernel_plain(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+r)*ldc+j,ldc,mc-r<MR?mc-r:MR);else if(a.sign==0)kernel<2>(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+r)*ldc+j,ldc,mc-r<MR?mc-r:MR,pks,pky,pkd);else if(a.sign>0)kernel<1>(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+r)*ldc+j,ldc,mc-r<MR?mc-r:MR,pks,pky,pkd);else kernel<-1>(n,pkb[cur]+(size_t)r*(BASE+24),bp+(size_t)j*(n+1),c+(size_t)(ic+r)*ldc+j,ldc,mc-r<MR?mc-r:MR,pks,pky,pkd);if(pending.active&&(++pending_tick%PERIOD)==0){pending_step();if(pending.prog)pending_step();pending_prefetch();}}for(int cc=ck;cc<nch;cc++){const int row=cc/2,off=(cc%2)*qd;if(row<nxt){for(int k=off;k<off+qd;k+=4)_mm256_store_pd(pkb[cur^1]+(size_t)row*(BASE+24)+k,ld(a,ic2+row,k));}else{for(int k=off;k<off+qd;k+=4)_mm256_store_pd(pkb[cur^1]+(size_t)row*(BASE+24)+k,_mm256_setzero_pd());}}cur^=1;ic+=mc;if(pending.prog)pending.rowlim=ic;}if(n>=NTMIN)_mm_sfence();}static const int lperm[7]={3,1,2,0,5,4,6};static void rec(int n,Mat a,Mat b,double*c,int ldc,double*work){const int h=BASE;const size_t z=(size_t)h*h;Mat aa[4]={{a.x,a.ld},{a.x+h,a.ld},{a.x+(size_t)h*a.ld,a.ld},{a.x+(size_t)h*a.ld+h,a.ld}};Mat bb[4]={{b.x,b.ld},{b.x+h,b.ld},{b.x+(size_t)h*b.ld,b.ld},{b.x+(size_t)h*b.ld+h,b.ld}};double*m=work+LEAF_PAD;if(pending.active&&pending.m==m)m=pending_work+LEAF_PAD;size_t pz=z+LEAF_PAD;double*out[7]={c,c+(size_t)h*ldc,c+h,c+(size_t)h*ldc+h,m,m+pz,m+2*pz};const int ai[7]={0,2,0,3,0,2,1},aj[7]={3,3,-1,-1,1,0,3},as[7]={1,1,0,0,1,-1,-1};const int bi[7]={0,0,1,2,3,0,2},bj[7]={3,-1,3,0,-1,1,3},bs[7]={1,0,-1,-1,0,1,1};for(int r=0;r<7;r++){const int q=lperm[r];Mat x=aa[ai[q]],y=bb[bi[q]];if(as[q]){x.y=aa[aj[q]].x;x.ly=a.ld;x.sign=as[q];}if(bs[q]){y.y=bb[bj[q]].x;y.ly=b.ld;y.sign=bs[q];}if(r==6){pending_finish();pending={c,ldc,m,0,true,0,true};}leaf(h,x,y,out[q],q<4?ldc:h);if(r==6)pending.prog=false;}}static void rank48_top(const double*a,const double*b,double*c){pending_finish();pending_tick=0;Mat aa[16],bb[16],slots[56];for(int q=0;q<16;q++){size_t off=(size_t)(q/4)*1024*4096+(q%4)*1024;aa[q]={a+off,4096};bb[q]={b+off,4096};}slots[0]={c+8388608,4096};slots[1]={workspace+(size_t)0*(1024*1024+7680),1024};slots[2]={workspace+(size_t)1*(1024*1024+7680),1024};slots[3]={c+3072,4096};slots[4]={workspace+(size_t)2*(1024*1024+7680),1024};slots[5]={c+12585984,4096};slots[6]={workspace+(size_t)3*(1024*1024+7680),1024};slots[7]={workspace+(size_t)4*(1024*1024+7680),1024};slots[8]={c+8391680,4096};slots[9]={workspace+(size_t)5*(1024*1024+7680),1024};slots[10]={workspace+(size_t)6*(1024*1024+7680),1024};slots[11]={workspace+(size_t)7*(1024*1024+7680),1024};slots[12]={workspace+(size_t)8*(1024*1024+7680),1024};slots[13]={c+8389632,4096};slots[14]={workspace+(size_t)9*(1024*1024+7680),1024};slots[15]={c+0,4096};slots[16]={workspace+(size_t)10*(1024*1024+7680),1024};slots[17]={workspace+(size_t)11*(1024*1024+7680),1024};slots[18]={c+4196352,4096};slots[19]={workspace+(size_t)12*(1024*1024+7680),1024};slots[20]={workspace+(size_t)13*(1024*1024+7680),1024};slots[21]={workspace+(size_t)14*(1024*1024+7680),1024};slots[22]={c+4195328,4096};slots[23]={workspace+(size_t)15*(1024*1024+7680),1024};slots[24]={workspace+(size_t)16*(1024*1024+7680),1024};slots[25]={workspace+(size_t)17*(1024*1024+7680),1024};slots[26]={workspace+(size_t)18*(1024*1024+7680),1024};slots[27]={workspace+(size_t)19*(1024*1024+7680),1024};slots[28]={workspace+(size_t)20*(1024*1024+7680),1024};slots[29]={workspace+(size_t)21*(1024*1024+7680),1024};slots[30]={c+12582912,4096};slots[31]={c+12584960,4096};slots[32]={workspace+(size_t)22*(1024*1024+7680),1024};slots[33]={workspace+(size_t)23*(1024*1024+7680),1024};slots[34]={c+2048,4096};slots[35]={workspace+(size_t)24*(1024*1024+7680),1024};slots[36]={workspace+(size_t)25*(1024*1024+7680),1024};slots[37]={c+4197376,4096};slots[38]={c+1024,4096};slots[39]={c+4194304,4096};slots[40]={workspace+(size_t)26*(1024*1024+7680),1024};slots[41]={workspace+(size_t)27*(1024*1024+7680),1024};slots[42]={c+12583936,4096};slots[43]={workspace+(size_t)28*(1024*1024+7680),1024};slots[44]={workspace+(size_t)29*(1024*1024+7680),1024};slots[45]={c+8390656,4096};slots[46]={workspace+(size_t)30*(1024*1024+7680),1024};slots[47]={workspace+(size_t)31*(1024*1024+7680),1024};slots[48]={workspace+(size_t)32*(1024*1024+7680),1024};slots[49]={workspace+(size_t)33*(1024*1024+7680),1024};slots[50]={workspace+(size_t)34*(1024*1024+7680),1024};slots[51]={workspace+(size_t)35*(1024*1024+7680),1024};slots[52]={workspace+(size_t)36*(1024*1024+7680),1024};slots[53]={workspace+(size_t)37*(1024*1024+7680),1024};slots[54]={workspace+(size_t)38*(1024*1024+7680),1024};slots[55]={workspace+(size_t)39*(1024*1024+7680),1024};double*work=workspace+(size_t)40*(1024*1024+7680);for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_Li12_0=ld(aa[12],i,j+0);__m256d v_Li12_1=ld(aa[12],i,j+4);__m256d v_Li4_0=ld(aa[4],i,j+0);__m256d v_Li4_1=ld(aa[4],i,j+4);__m256d v_Lx24_0=_mm256_sub_pd(v_Li12_0,v_Li4_0);__m256d v_Lx24_1=_mm256_sub_pd(v_Li12_1,v_Li4_1);__m256d v_Li7_0=ld(aa[7],i,j+0);__m256d v_Li7_1=ld(aa[7],i,j+4);__m256d v_Lx21_0=_mm256_add_pd(v_Li4_0,v_Li7_0);__m256d v_Lx21_1=_mm256_add_pd(v_Li4_1,v_Li7_1);__m256d v_Li6_0=ld(aa[6],i,j+0);__m256d v_Li6_1=ld(aa[6],i,j+4);__m256d v_Lo28_0=_mm256_sub_pd(v_Lx21_0,v_Li6_0);__m256d v_Lo28_1=_mm256_sub_pd(v_Lx21_1,v_Li6_1);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+0,v_Lo28_0);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+4,v_Lo28_1);__m256d v_Li14_0=ld(aa[14],i,j+0);__m256d v_Li14_1=ld(aa[14],i,j+4);__m256d v_Lx19_0=_mm256_sub_pd(v_Li6_0,v_Li14_0);__m256d v_Lx19_1=_mm256_sub_pd(v_Li6_1,v_Li14_1);__m256d v_Li2_0=ld(aa[2],i,j+0);__m256d v_Li2_1=ld(aa[2],i,j+4);__m256d v_Lo22_0=_mm256_sub_pd(v_Lx19_0,v_Li2_0);__m256d v_Lo22_1=_mm256_sub_pd(v_Lx19_1,v_Li2_1);Q0(const_cast<double*>(slots[11].x)+(size_t)i*slots[11].ld+j+0,v_Lo22_0);Q0(const_cast<double*>(slots[11].x)+(size_t)i*slots[11].ld+j+4,v_Lo22_1);__m256d v_Li8_0=ld(aa[8],i,j+0);__m256d v_Li8_1=ld(aa[8],i,j+4);__m256d v_Lo26_0=_mm256_add_pd(v_Li8_0,v_Lx24_0);__m256d v_Lo26_1=_mm256_add_pd(v_Li8_1,v_Lx24_1);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+0,v_Lo26_0);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+4,v_Lo26_1);__m256d v_Li11_0=ld(aa[11],i,j+0);__m256d v_Li11_1=ld(aa[11],i,j+4);__m256d v_Lx16_0=_mm256_add_pd(v_Li8_0,v_Li11_0);__m256d v_Lx16_1=_mm256_add_pd(v_Li8_1,v_Li11_1);__m256d v_Li1_0=ld(aa[1],i,j+0);__m256d v_Li1_1=ld(aa[1],i,j+4);__m256d v_Lx11_0=_mm256_add_pd(v_Li1_0,v_Li6_0);__m256d v_Lx11_1=_mm256_add_pd(v_Li1_1,v_Li6_1);__m256d v_Li3_0=ld(aa[3],i,j+0);__m256d v_Li3_1=ld(aa[3],i,j+4);__m256d v_Lo20_0=_mm256_sub_pd(_mm256_add_pd(v_Li1_0,v_Li2_0),v_Li3_0);__m256d v_Lo20_1=_mm256_sub_pd(_mm256_add_pd(v_Li1_1,v_Li2_1),v_Li3_1);Q0(const_cast<double*>(slots[7].x)+(size_t)i*slots[7].ld+j+0,v_Lo20_0);Q0(const_cast<double*>(slots[7].x)+(size_t)i*slots[7].ld+j+4,v_Lo20_1);__m256d v_Li0_0=ld(aa[0],i,j+0);__m256d v_Li0_1=ld(aa[0],i,j+4);__m256d v_Lx17_0=_mm256_add_pd(v_Li0_0,v_Li3_0);__m256d v_Lx17_1=_mm256_add_pd(v_Li0_1,v_Li3_1);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+0,v_Lx17_0);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+4,v_Lx17_1);__m256d v_Li10_0=ld(aa[10],i,j+0);__m256d v_Li10_1=ld(aa[10],i,j+4);__m256d v_Lo4_0=_mm256_sub_pd(v_Li10_0,v_Lx19_0);__m256d v_Lo4_1=_mm256_sub_pd(v_Li10_1,v_Lx19_1);Q0(const_cast<double*>(slots[1].x)+(size_t)i*slots[1].ld+j+0,v_Lo4_0);Q0(const_cast<double*>(slots[1].x)+(size_t)i*slots[1].ld+j+4,v_Lo4_1);__m256d v_Lx20_0=_mm256_mul_pd(_mm256_add_pd(v_Lx21_0,v_Lx17_0),_mm256_set1_pd(0.5));__m256d v_Lx20_1=_mm256_mul_pd(_mm256_add_pd(v_Lx21_1,v_Lx17_1),_mm256_set1_pd(0.5));Q0(const_cast<double*>(slots[6].x)+(size_t)i*slots[6].ld+j+0,v_Lx20_0);Q0(const_cast<double*>(slots[6].x)+(size_t)i*slots[6].ld+j+4,v_Lx20_1);__m256d v_Lo0_0=_mm256_add_pd(v_Li1_0,v_Lx17_0);__m256d v_Lo0_1=_mm256_add_pd(v_Li1_1,v_Lx17_1);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+0,v_Lo0_0);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+4,v_Lo0_1);__m256d v_Lo27_0=_mm256_add_pd(v_Li1_0,v_Lx20_0);__m256d v_Lo27_1=_mm256_add_pd(v_Li1_1,v_Lx20_1);Q0(const_cast<double*>(slots[9].x)+(size_t)i*slots[9].ld+j+0,v_Lo27_0);Q0(const_cast<double*>(slots[9].x)+(size_t)i*slots[9].ld+j+4,v_Lo27_1);__m256d v_Lo8_0=_mm256_sub_pd(v_Lo27_0,v_Lx11_0);__m256d v_Lo8_1=_mm256_sub_pd(v_Lo27_1,v_Lx11_1);Q0(const_cast<double*>(slots[10].x)+(size_t)i*slots[10].ld+j+0,v_Lo8_0);Q0(const_cast<double*>(slots[10].x)+(size_t)i*slots[10].ld+j+4,v_Lo8_1);__m256d v_Lo2_0=_mm256_sub_pd(v_Li10_0,v_Lx16_0);__m256d v_Lo2_1=_mm256_sub_pd(v_Li10_1,v_Lx16_1);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+0,v_Lo2_0);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+4,v_Lo2_1);__m256d v_Lo24_0=_mm256_add_pd(_mm256_add_pd(v_Li14_0,v_Lo2_0),v_Lo8_0);__m256d v_Lo24_1=_mm256_add_pd(_mm256_add_pd(v_Li14_1,v_Lo2_1),v_Lo8_1);Q0(const_cast<double*>(slots[19].x)+(size_t)i*slots[19].ld+j+0,v_Lo24_0);Q0(const_cast<double*>(slots[19].x)+(size_t)i*slots[19].ld+j+4,v_Lo24_1);__m256d v_Li9_0=ld(aa[9],i,j+0);__m256d v_Li9_1=ld(aa[9],i,j+4);__m256d v_Lo42_0=_mm256_add_pd(v_Lx16_0,v_Li9_0);__m256d v_Lo42_1=_mm256_add_pd(v_Lx16_1,v_Li9_1);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+0,v_Lo42_0);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+4,v_Lo42_1);__m256d v_Li13_0=ld(aa[13],i,j+0);__m256d v_Li13_1=ld(aa[13],i,j+4);__m256d v_Lx18_0=_mm256_add_pd(v_Li1_0,v_Li13_0);__m256d v_Lx18_1=_mm256_add_pd(v_Li1_1,v_Li13_1);__m256d v_Li5_0=ld(aa[5],i,j+0);__m256d v_Li5_1=ld(aa[5],i,j+4);__m256d v_Lo36_0=_mm256_sub_pd(v_Lx18_0,v_Li5_0);__m256d v_Lo36_1=_mm256_sub_pd(v_Lx18_1,v_Li5_1);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+0,v_Lo36_0);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+4,v_Lo36_1);__m256d v_Lo45_0=_mm256_sub_pd(v_Lx18_0,v_Li9_0);__m256d v_Lo45_1=_mm256_sub_pd(v_Lx18_1,v_Li9_1);Q0(const_cast<double*>(slots[12].x)+(size_t)i*slots[12].ld+j+0,v_Lo45_0);Q0(const_cast<double*>(slots[12].x)+(size_t)i*slots[12].ld+j+4,v_Lo45_1);__m256d v_Lx23_0=_mm256_sub_pd(v_Lo4_0,v_Lo45_0);__m256d v_Lx23_1=_mm256_sub_pd(v_Lo4_1,v_Lo45_1);__m256d v_Lo43_0=_mm256_mul_pd(_mm256_add_pd(v_Lx11_0,v_Lx23_0),_mm256_set1_pd(0.5));__m256d v_Lo43_1=_mm256_mul_pd(_mm256_add_pd(v_Lx11_1,v_Lx23_1),_mm256_set1_pd(0.5));Q0(const_cast<double*>(slots[14].x)+(size_t)i*slots[14].ld+j+0,v_Lo43_0);Q0(const_cast<double*>(slots[14].x)+(size_t)i*slots[14].ld+j+4,v_Lo43_1);__m256d v_Lx15_0=_mm256_sub_pd(v_Lo43_0,v_Li14_0);__m256d v_Lx15_1=_mm256_sub_pd(v_Lo43_1,v_Li14_1);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+0,v_Lx15_0);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+4,v_Lx15_1);__m256d v_Lo34_0=_mm256_sub_pd(v_Lo24_0,v_Lx23_0);__m256d v_Lo34_1=_mm256_sub_pd(v_Lo24_1,v_Lx23_1);Q0(const_cast<double*>(slots[13].x)+(size_t)i*slots[13].ld+j+0,v_Lo34_0);Q0(const_cast<double*>(slots[13].x)+(size_t)i*slots[13].ld+j+4,v_Lo34_1);__m256d v_Lo37_0=_mm256_sub_pd(v_Lo43_0,v_Lx23_0);__m256d v_Lo37_1=_mm256_sub_pd(v_Lo43_1,v_Lx23_1);Q0(const_cast<double*>(slots[15].x)+(size_t)i*slots[15].ld+j+0,v_Lo37_0);Q0(const_cast<double*>(slots[15].x)+(size_t)i*slots[15].ld+j+4,v_Lo37_1);__m256d v_Lx10_0=_mm256_sub_pd(v_Lo36_0,v_Lo37_0);__m256d v_Lx10_1=_mm256_sub_pd(v_Lo36_1,v_Lo37_1);__m256d v_Lo39_0=_mm256_add_pd(v_Li7_0,v_Lx10_0);__m256d v_Lo39_1=_mm256_add_pd(v_Li7_1,v_Lx10_1);Q0(const_cast<double*>(slots[16].x)+(size_t)i*slots[16].ld+j+0,v_Lo39_0);Q0(const_cast<double*>(slots[16].x)+(size_t)i*slots[16].ld+j+4,v_Lo39_1);__m256d v_Lx25_0=_mm256_add_pd(v_Lx24_0,v_Lx10_0);__m256d v_Lx25_1=_mm256_add_pd(v_Lx24_1,v_Lx10_1);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+0,v_Lx25_0);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+4,v_Lx25_1);__m256d v_Lx12_0=_mm256_add_pd(v_Lo27_0,v_Lo34_0);__m256d v_Lx12_1=_mm256_add_pd(v_Lo27_1,v_Lo34_1);Q0(const_cast<double*>(slots[20].x)+(size_t)i*slots[20].ld+j+0,v_Lx12_0);Q0(const_cast<double*>(slots[20].x)+(size_t)i*slots[20].ld+j+4,v_Lx12_1);}_mm_sfence();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_Ri0_0=ld(bb[0],i,j+0);__m256d v_Ri0_1=ld(bb[0],i,j+4);__m256d v_Ri12_0=ld(bb[12],i,j+0);__m256d v_Ri12_1=ld(bb[12],i,j+4);__m256d v_Rx21_0=_mm256_sub_pd(v_Ri0_0,v_Ri12_0);__m256d v_Rx21_1=_mm256_sub_pd(v_Ri0_1,v_Ri12_1);__m256d v_Ri3_0=ld(bb[3],i,j+0);__m256d v_Ri3_1=ld(bb[3],i,j+4);__m256d v_Rx24_0=_mm256_sub_pd(v_Ri0_0,v_Ri3_0);__m256d v_Rx24_1=_mm256_sub_pd(v_Ri0_1,v_Ri3_1);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+0,v_Rx24_0);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+4,v_Rx24_1);__m256d v_Ri8_0=ld(bb[8],i,j+0);__m256d v_Ri8_1=ld(bb[8],i,j+4);__m256d v_Ro29_0=_mm256_sub_pd(v_Rx21_0,v_Ri8_0);__m256d v_Ro29_1=_mm256_sub_pd(v_Rx21_1,v_Ri8_1);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+0,v_Ro29_0);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+4,v_Ro29_1);__m256d v_Ri11_0=ld(bb[11],i,j+0);__m256d v_Ri11_1=ld(bb[11],i,j+4);__m256d v_Rx19_0=_mm256_sub_pd(v_Ri11_0,v_Ri8_0);__m256d v_Rx19_1=_mm256_sub_pd(v_Ri11_1,v_Ri8_1);__m256d v_Ri9_0=ld(bb[9],i,j+0);__m256d v_Ri9_1=ld(bb[9],i,j+4);__m256d v_Ro20_0=_mm256_sub_pd(v_Ri9_0,v_Rx19_0);__m256d v_Ro20_1=_mm256_sub_pd(v_Ri9_1,v_Rx19_1);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+0,v_Ro20_0);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+4,v_Ro20_1);__m256d v_Ri10_0=ld(bb[10],i,j+0);__m256d v_Ri10_1=ld(bb[10],i,j+4);__m256d v_Ro2_0=_mm256_add_pd(v_Ri10_0,v_Rx19_0);__m256d v_Ro2_1=_mm256_add_pd(v_Ri10_1,v_Rx19_1);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+0,v_Ro2_0);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+4,v_Ro2_1);__m256d v_Ri2_0=ld(bb[2],i,j+0);__m256d v_Ri2_1=ld(bb[2],i,j+4);__m256d v_Ri1_0=ld(bb[1],i,j+0);__m256d v_Ri1_1=ld(bb[1],i,j+4);__m256d v_Ro28_0=_mm256_add_pd(_mm256_add_pd(v_Ri3_0,v_Ri2_0),v_Ri1_0);__m256d v_Ro28_1=_mm256_add_pd(_mm256_add_pd(v_Ri3_1,v_Ri2_1),v_Ri1_1);Q0(const_cast<double*>(slots[45].x)+(size_t)i*slots[45].ld+j+0,v_Ro28_0);Q0(const_cast<double*>(slots[45].x)+(size_t)i*slots[45].ld+j+4,v_Ro28_1);__m256d v_Ri13_0=ld(bb[13],i,j+0);__m256d v_Ri13_1=ld(bb[13],i,j+4);__m256d v_Rx16_0=_mm256_sub_pd(v_Ri1_0,v_Ri13_0);__m256d v_Rx16_1=_mm256_sub_pd(v_Ri1_1,v_Ri13_1);__m256d v_Ro22_0=_mm256_sub_pd(v_Rx16_0,v_Ri9_0);__m256d v_Ro22_1=_mm256_sub_pd(v_Rx16_1,v_Ri9_1);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+0,v_Ro22_0);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+4,v_Ro22_1);__m256d v_Ri14_0=ld(bb[14],i,j+0);__m256d v_Ri14_1=ld(bb[14],i,j+4);__m256d v_Rx17_0=_mm256_sub_pd(v_Ri14_0,v_Ri2_0);__m256d v_Rx17_1=_mm256_sub_pd(v_Ri14_1,v_Ri2_1);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+0,v_Rx17_0);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+4,v_Rx17_1);__m256d v_Ri5_0=ld(bb[5],i,j+0);__m256d v_Ri5_1=ld(bb[5],i,j+4);__m256d v_Ro36_0=_mm256_sub_pd(v_Ri5_0,v_Rx16_0);__m256d v_Ro36_1=_mm256_sub_pd(v_Ri5_1,v_Rx16_1);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+0,v_Ro36_0);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+4,v_Ro36_1);__m256d v_Rx20_0=_mm256_mul_pd(_mm256_sub_pd(v_Rx21_0,v_Rx17_0),_mm256_set1_pd(0.5));__m256d v_Rx20_1=_mm256_mul_pd(_mm256_sub_pd(v_Rx21_1,v_Rx17_1),_mm256_set1_pd(0.5));Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+0,v_Rx20_0);Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+4,v_Rx20_1);__m256d v_Ri6_0=ld(bb[6],i,j+0);__m256d v_Ri6_1=ld(bb[6],i,j+4);__m256d v_Ro16_0=_mm256_sub_pd(v_Rx20_0,v_Ri6_0);__m256d v_Ro16_1=_mm256_sub_pd(v_Rx20_1,v_Ri6_1);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+0,v_Ro16_0);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+4,v_Ro16_1);__m256d v_Ro26_0=_mm256_add_pd(v_Ri6_0,v_Rx17_0);__m256d v_Ro26_1=_mm256_add_pd(v_Ri6_1,v_Rx17_1);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+0,v_Ro26_0);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+4,v_Ro26_1);__m256d v_Ri7_0=ld(bb[7],i,j+0);__m256d v_Ri7_1=ld(bb[7],i,j+4);__m256d v_Rx18_0=_mm256_add_pd(v_Ri7_0,v_Ri6_0);__m256d v_Rx18_1=_mm256_add_pd(v_Ri7_1,v_Ri6_1);__m256d v_Ro1_0=_mm256_add_pd(v_Ri5_0,v_Rx18_0);__m256d v_Ro1_1=_mm256_add_pd(v_Ri5_1,v_Rx18_1);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+0,v_Ro1_0);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+4,v_Ro1_1);__m256d v_Ri4_0=ld(bb[4],i,j+0);__m256d v_Ri4_1=ld(bb[4],i,j+4);__m256d v_Ro42_0=_mm256_sub_pd(v_Ri4_0,v_Rx18_0);__m256d v_Ro42_1=_mm256_sub_pd(v_Ri4_1,v_Rx18_1);Q0(const_cast<double*>(slots[39].x)+(size_t)i*slots[39].ld+j+0,v_Ro42_0);Q0(const_cast<double*>(slots[39].x)+(size_t)i*slots[39].ld+j+4,v_Ro42_1);__m256d v_Lx12_0=ld(slots[20],i,j+0);__m256d v_Lx12_1=ld(slots[20],i,j+4);__m256d v_Lo37_0=ld(slots[15],i,j+0);__m256d v_Lo37_1=ld(slots[15],i,j+4);__m256d v_Lo47_0=_mm256_sub_pd(v_Lx12_0,v_Lo37_0);__m256d v_Lo47_1=_mm256_sub_pd(v_Lx12_1,v_Lo37_1);Q0(const_cast<double*>(slots[26].x)+(size_t)i*slots[26].ld+j+0,v_Lo47_0);Q0(const_cast<double*>(slots[26].x)+(size_t)i*slots[26].ld+j+4,v_Lo47_1);__m256d v_Lo36_0=ld(slots[0],i,j+0);__m256d v_Lo36_1=ld(slots[0],i,j+4);__m256d v_Lo38_0=_mm256_sub_pd(v_Lx12_0,v_Lo36_0);__m256d v_Lo38_1=_mm256_sub_pd(v_Lx12_1,v_Lo36_1);Q0(const_cast<double*>(slots[20].x)+(size_t)i*slots[20].ld+j+0,v_Lo38_0);Q0(const_cast<double*>(slots[20].x)+(size_t)i*slots[20].ld+j+4,v_Lo38_1);__m256d v_Li13_0=ld(aa[13],i,j+0);__m256d v_Li13_1=ld(aa[13],i,j+4);__m256d v_Lo43_0=ld(slots[14],i,j+0);__m256d v_Lo43_1=ld(slots[14],i,j+4);__m256d v_Lx14_0=_mm256_add_pd(v_Li13_0,v_Lo43_0);__m256d v_Lx14_1=_mm256_add_pd(v_Li13_1,v_Lo43_1);__m256d v_Lx15_0=ld(slots[17],i,j+0);__m256d v_Lx15_1=ld(slots[17],i,j+4);__m256d v_Lo20_0=ld(slots[7],i,j+0);__m256d v_Lo20_1=ld(slots[7],i,j+4);__m256d v_Lo33_0=_mm256_sub_pd(v_Lx15_0,v_Lo20_0);__m256d v_Lo33_1=_mm256_sub_pd(v_Lx15_1,v_Lo20_1);Q0(const_cast<double*>(slots[22].x)+(size_t)i*slots[22].ld+j+0,v_Lo33_0);Q0(const_cast<double*>(slots[22].x)+(size_t)i*slots[22].ld+j+4,v_Lo33_1);__m256d v_Li3_0=ld(aa[3],i,j+0);__m256d v_Li3_1=ld(aa[3],i,j+4);__m256d v_Lo6_0=_mm256_sub_pd(_mm256_sub_pd(v_Li3_0,v_Lo33_0),v_Lo47_0);__m256d v_Lo6_1=_mm256_sub_pd(_mm256_sub_pd(v_Li3_1,v_Lo33_1),v_Lo47_1);Q0(const_cast<double*>(slots[37].x)+(size_t)i*slots[37].ld+j+0,v_Lo6_0);Q0(const_cast<double*>(slots[37].x)+(size_t)i*slots[37].ld+j+4,v_Lo6_1);__m256d v_Lo39_0=ld(slots[16],i,j+0);__m256d v_Lo39_1=ld(slots[16],i,j+4);__m256d v_Lo1_0=_mm256_sub_pd(v_Lx14_0,v_Lo39_0);__m256d v_Lo1_1=_mm256_sub_pd(v_Lx14_1,v_Lo39_1);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+0,v_Lo1_0);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+4,v_Lo1_1);__m256d v_Lo2_0=ld(slots[18],i,j+0);__m256d v_Lo2_1=ld(slots[18],i,j+4);__m256d v_Lo14_0=_mm256_sub_pd(v_Lx15_0,v_Lo2_0);__m256d v_Lo14_1=_mm256_sub_pd(v_Lx15_1,v_Lo2_1);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+0,v_Lo14_0);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+4,v_Lo14_1);__m256d v_Lx25_0=ld(slots[21],i,j+0);__m256d v_Lx25_1=ld(slots[21],i,j+4);__m256d v_Lx26_0=_mm256_sub_pd(v_Lx25_0,v_Lo33_0);__m256d v_Lx26_1=_mm256_sub_pd(v_Lx25_1,v_Lo33_1);__m256d v_Lx13_0=_mm256_add_pd(v_Lo33_0,v_Lo39_0);__m256d v_Lx13_1=_mm256_add_pd(v_Lo33_1,v_Lo39_1);__m256d v_Li15_0=ld(aa[15],i,j+0);__m256d v_Li15_1=ld(aa[15],i,j+4);__m256d v_Lo5_0=_mm256_sub_pd(_mm256_add_pd(v_Li15_0,v_Lo33_0),v_Lo39_0);__m256d v_Lo5_1=_mm256_sub_pd(_mm256_add_pd(v_Li15_1,v_Lo33_1),v_Lo39_1);Q0(const_cast<double*>(slots[24].x)+(size_t)i*slots[24].ld+j+0,v_Lo5_0);Q0(const_cast<double*>(slots[24].x)+(size_t)i*slots[24].ld+j+4,v_Lo5_1);__m256d v_Li11_0=ld(aa[11],i,j+0);__m256d v_Li11_1=ld(aa[11],i,j+4);__m256d v_Lo10_0=_mm256_sub_pd(v_Li11_0,v_Lx13_0);__m256d v_Lo10_1=_mm256_sub_pd(v_Li11_1,v_Lx13_1);Q0(const_cast<double*>(slots[33].x)+(size_t)i*slots[33].ld+j+0,v_Lo10_0);Q0(const_cast<double*>(slots[33].x)+(size_t)i*slots[33].ld+j+4,v_Lo10_1);__m256d v_Lx17_0=ld(slots[3],i,j+0);__m256d v_Lx17_1=ld(slots[3],i,j+4);__m256d v_Lo17_0=_mm256_add_pd(v_Lx17_0,v_Lx26_0);__m256d v_Lo17_1=_mm256_add_pd(v_Lx17_1,v_Lx26_1);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+0,v_Lo17_0);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+4,v_Lo17_1);__m256d v_Lx20_0=ld(slots[6],i,j+0);__m256d v_Lx20_1=ld(slots[6],i,j+4);__m256d v_Lo18_0=_mm256_add_pd(v_Lx20_0,v_Lx26_0);__m256d v_Lo18_1=_mm256_add_pd(v_Lx20_1,v_Lx26_1);Q0(const_cast<double*>(slots[23].x)+(size_t)i*slots[23].ld+j+0,v_Lo18_0);Q0(const_cast<double*>(slots[23].x)+(size_t)i*slots[23].ld+j+4,v_Lo18_1);__m256d v_Lx22_0=_mm256_add_pd(v_Lo18_0,v_Lo5_0);__m256d v_Lx22_1=_mm256_add_pd(v_Lo18_1,v_Lo5_1);__m256d v_Lo11_0=_mm256_sub_pd(v_Lx22_0,v_Li15_0);__m256d v_Lo11_1=_mm256_sub_pd(v_Lx22_1,v_Li15_1);Q0(const_cast<double*>(slots[25].x)+(size_t)i*slots[25].ld+j+0,v_Lo11_0);Q0(const_cast<double*>(slots[25].x)+(size_t)i*slots[25].ld+j+4,v_Lo11_1);__m256d v_Lo34_0=ld(slots[13],i,j+0);__m256d v_Lo34_1=ld(slots[13],i,j+4);__m256d v_Lo44_0=_mm256_add_pd(v_Lx22_0,v_Lo34_0);__m256d v_Lo44_1=_mm256_add_pd(v_Lx22_1,v_Lo34_1);Q0(const_cast<double*>(slots[28].x)+(size_t)i*slots[28].ld+j+0,v_Lo44_0);Q0(const_cast<double*>(slots[28].x)+(size_t)i*slots[28].ld+j+4,v_Lo44_1);__m256d v_Lo24_0=ld(slots[19],i,j+0);__m256d v_Lo24_1=ld(slots[19],i,j+4);__m256d v_Lo32_0=_mm256_sub_pd(v_Lx22_0,v_Lo24_0);__m256d v_Lo32_1=_mm256_sub_pd(v_Lx22_1,v_Lo24_1);Q0(const_cast<double*>(slots[27].x)+(size_t)i*slots[27].ld+j+0,v_Lo32_0);Q0(const_cast<double*>(slots[27].x)+(size_t)i*slots[27].ld+j+4,v_Lo32_1);__m256d v_Li8_0=ld(aa[8],i,j+0);__m256d v_Li8_1=ld(aa[8],i,j+4);__m256d v_Lo16_0=_mm256_add_pd(v_Li8_0,v_Lx25_0);__m256d v_Lo16_1=_mm256_add_pd(v_Li8_1,v_Lx25_1);Q0(const_cast<double*>(slots[30].x)+(size_t)i*slots[30].ld+j+0,v_Lo16_0);Q0(const_cast<double*>(slots[30].x)+(size_t)i*slots[30].ld+j+4,v_Lo16_1);__m256d v_Lo46_0=_mm256_add_pd(v_Lo16_0,v_Lx14_0);__m256d v_Lo46_1=_mm256_add_pd(v_Lo16_1,v_Lx14_1);Q0(const_cast<double*>(slots[31].x)+(size_t)i*slots[31].ld+j+0,v_Lo46_0);Q0(const_cast<double*>(slots[31].x)+(size_t)i*slots[31].ld+j+4,v_Lo46_1);__m256d v_Lo21_0=_mm256_sub_pd(v_Lx20_0,v_Li8_0);__m256d v_Lo21_1=_mm256_sub_pd(v_Lx20_1,v_Li8_1);Q0(const_cast<double*>(slots[34].x)+(size_t)i*slots[34].ld+j+0,v_Lo21_0);Q0(const_cast<double*>(slots[34].x)+(size_t)i*slots[34].ld+j+4,v_Lo21_1);__m256d v_Lo31_0=_mm256_sub_pd(v_Lx13_0,v_Lo21_0);__m256d v_Lo31_1=_mm256_sub_pd(v_Lx13_1,v_Lo21_1);Q0(const_cast<double*>(slots[32].x)+(size_t)i*slots[32].ld+j+0,v_Lo31_0);Q0(const_cast<double*>(slots[32].x)+(size_t)i*slots[32].ld+j+4,v_Lo31_1);__m256d v_Lo19_0=_mm256_sub_pd(v_Lx20_0,v_Lo31_0);__m256d v_Lo19_1=_mm256_sub_pd(v_Lx20_1,v_Lo31_1);Q0(const_cast<double*>(slots[6].x)+(size_t)i*slots[6].ld+j+0,v_Lo19_0);Q0(const_cast<double*>(slots[6].x)+(size_t)i*slots[6].ld+j+4,v_Lo19_1);__m256d v_Lo3_0=_mm256_add_pd(v_Lx25_0,v_Lo19_0);__m256d v_Lo3_1=_mm256_add_pd(v_Lx25_1,v_Lo19_1);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+0,v_Lo3_0);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+4,v_Lo3_1);__m256d v_Lo13_0=_mm256_sub_pd(v_Lx15_0,v_Lo3_0);__m256d v_Lo13_1=_mm256_sub_pd(v_Lx15_1,v_Lo3_1);Q0(const_cast<double*>(slots[35].x)+(size_t)i*slots[35].ld+j+0,v_Lo13_0);Q0(const_cast<double*>(slots[35].x)+(size_t)i*slots[35].ld+j+4,v_Lo13_1);__m256d v_Li12_0=ld(aa[12],i,j+0);__m256d v_Li12_1=ld(aa[12],i,j+4);__m256d v_Li0_0=ld(aa[0],i,j+0);__m256d v_Li0_1=ld(aa[0],i,j+4);__m256d v_Lo29_0=_mm256_sub_pd(_mm256_sub_pd(v_Li8_0,v_Li12_0),v_Li0_0);__m256d v_Lo29_1=_mm256_sub_pd(_mm256_sub_pd(v_Li8_1,v_Li12_1),v_Li0_1);Q0(const_cast<double*>(slots[36].x)+(size_t)i*slots[36].ld+j+0,v_Lo29_0);Q0(const_cast<double*>(slots[36].x)+(size_t)i*slots[36].ld+j+4,v_Lo29_1);}_mm_sfence();rec(1024,slots[36],slots[41],const_cast<double*>(slots[51].x),slots[51].ld,work);rec(1024,slots[5],slots[45],const_cast<double*>(slots[36].x),slots[36].ld,work);rec(1024,slots[18],slots[40],const_cast<double*>(slots[45].x),slots[45].ld,work);rec(1024,slots[4],slots[49],const_cast<double*>(slots[18].x),slots[18].ld,work);rec(1024,slots[0],slots[38],const_cast<double*>(slots[4].x),slots[4].ld,work);rec(1024,slots[30],slots[48],const_cast<double*>(slots[0].x),slots[0].ld,work);rec(1024,slots[28],bb[12],const_cast<double*>(slots[30].x),slots[30].ld,work);rec(1024,slots[11],slots[46],const_cast<double*>(slots[28].x),slots[28].ld,work);rec(1024,slots[35],bb[8],const_cast<double*>(slots[11].x),slots[11].ld,work);rec(1024,slots[29],slots[44],const_cast<double*>(slots[35].x),slots[35].ld,work);for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_Rx20_0=ld(slots[43],i,j+0);__m256d v_Rx20_1=ld(slots[43],i,j+4);__m256d v_Ri1_0=ld(bb[1],i,j+0);__m256d v_Ri1_1=ld(bb[1],i,j+4);__m256d v_Ro18_0=_mm256_add_pd(v_Rx20_0,v_Ri1_0);__m256d v_Ro18_1=_mm256_add_pd(v_Rx20_1,v_Ri1_1);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+0,v_Ro18_0);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+4,v_Ro18_1);__m256d v_Ro1_0=ld(slots[44],i,j+0);__m256d v_Ro1_1=ld(slots[44],i,j+4);__m256d v_Ro20_0=ld(slots[47],i,j+0);__m256d v_Ro20_1=ld(slots[47],i,j+4);__m256d v_Rx23_0=_mm256_sub_pd(v_Ro1_0,v_Ro20_0);__m256d v_Rx23_1=_mm256_sub_pd(v_Ro1_1,v_Ro20_1);__m256d v_Ri8_0=ld(bb[8],i,j+0);__m256d v_Ri8_1=ld(bb[8],i,j+4);__m256d v_Ri6_0=ld(bb[6],i,j+0);__m256d v_Ri6_1=ld(bb[6],i,j+4);__m256d v_Rx11_0=_mm256_sub_pd(v_Ri8_0,v_Ri6_0);__m256d v_Rx11_1=_mm256_sub_pd(v_Ri8_1,v_Ri6_1);__m256d v_Ro43_0=_mm256_mul_pd(_mm256_add_pd(v_Rx11_0,v_Rx23_0),_mm256_set1_pd(0.5));__m256d v_Ro43_1=_mm256_mul_pd(_mm256_add_pd(v_Rx11_1,v_Rx23_1),_mm256_set1_pd(0.5));Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+0,v_Ro43_0);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+4,v_Ro43_1);__m256d v_Ro14_0=_mm256_sub_pd(v_Rx23_0,v_Ro43_0);__m256d v_Ro14_1=_mm256_sub_pd(v_Rx23_1,v_Ro43_1);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+0,v_Ro14_0);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+4,v_Ro14_1);__m256d v_Ro42_0=ld(slots[39],i,j+0);__m256d v_Ro42_1=ld(slots[39],i,j+4);__m256d v_Rx10_0=_mm256_add_pd(v_Ro42_0,v_Ro14_0);__m256d v_Rx10_1=_mm256_add_pd(v_Ro42_1,v_Ro14_1);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+0,v_Rx10_0);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+4,v_Rx10_1);__m256d v_Ro16_0=ld(slots[48],i,j+0);__m256d v_Ro16_1=ld(slots[48],i,j+4);__m256d v_Ro3_0=_mm256_sub_pd(v_Ro16_0,v_Rx11_0);__m256d v_Ro3_1=_mm256_sub_pd(v_Ro16_1,v_Rx11_1);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+0,v_Ro3_0);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+4,v_Ro3_1);__m256d v_Ro22_0=ld(slots[46],i,j+0);__m256d v_Ro22_1=ld(slots[46],i,j+4);__m256d v_Ri11_0=ld(bb[11],i,j+0);__m256d v_Ri11_1=ld(bb[11],i,j+4);__m256d v_Ro33_0=_mm256_add_pd(_mm256_add_pd(v_Ro22_0,v_Ri11_0),v_Ro3_0);__m256d v_Ro33_1=_mm256_add_pd(_mm256_add_pd(v_Ro22_1,v_Ri11_1),v_Ro3_1);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+0,v_Ro33_0);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+4,v_Ro33_1);__m256d v_Ro39_0=_mm256_sub_pd(v_Rx23_0,v_Ro33_0);__m256d v_Ro39_1=_mm256_sub_pd(v_Rx23_1,v_Ro33_1);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+0,v_Ro39_0);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+4,v_Ro39_1);__m256d v_Rx12_0=_mm256_sub_pd(v_Ro39_0,v_Ro16_0);__m256d v_Rx12_1=_mm256_sub_pd(v_Ro39_1,v_Ro16_1);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+0,v_Rx12_0);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+4,v_Rx12_1);}_mm_sfence();rec(1024,slots[21],slots[38],const_cast<double*>(slots[52].x),slots[52].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M13_0=ld(slots[11],i,j+0);__m256d v_M13_1=ld(slots[11],i,j+4);__m256d v_M3_0=ld(slots[52],i,j+0);__m256d v_M3_1=ld(slots[52],i,j+4);__m256d v_Pz29_0=_mm256_sub_pd(v_M13_0,v_M3_0);__m256d v_Pz29_1=_mm256_sub_pd(v_M13_1,v_M3_1);Q0(const_cast<double*>(slots[11].x)+(size_t)i*slots[11].ld+j+0,v_Pz29_0);Q0(const_cast<double*>(slots[11].x)+(size_t)i*slots[11].ld+j+4,v_Pz29_1);}_mm_sfence();rec(1024,slots[7],slots[47],const_cast<double*>(slots[38].x),slots[38].ld,work);rec(1024,slots[23],slots[29],const_cast<double*>(slots[7].x),slots[7].ld,work);rec(1024,slots[22],slots[40],const_cast<double*>(slots[23].x),slots[23].ld,work);rec(1024,slots[14],slots[49],const_cast<double*>(slots[22].x),slots[22].ld,work);rec(1024,slots[17],slots[5],const_cast<double*>(slots[14].x),slots[14].ld,work);for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_Ro43_0=ld(slots[49],i,j+0);__m256d v_Ro43_1=ld(slots[49],i,j+4);__m256d v_Ri7_0=ld(bb[7],i,j+0);__m256d v_Ri7_1=ld(bb[7],i,j+4);__m256d v_Rx14_0=_mm256_sub_pd(v_Ro43_0,v_Ri7_0);__m256d v_Rx14_1=_mm256_sub_pd(v_Ro43_1,v_Ri7_1);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+0,v_Rx14_0);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+4,v_Rx14_1);__m256d v_Ri11_0=ld(bb[11],i,j+0);__m256d v_Ri11_1=ld(bb[11],i,j+4);__m256d v_Rx15_0=_mm256_sub_pd(v_Ro43_0,v_Ri11_0);__m256d v_Rx15_1=_mm256_sub_pd(v_Ro43_1,v_Ri11_1);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+0,v_Rx15_0);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+4,v_Rx15_1);__m256d v_Ro22_0=ld(slots[46],i,j+0);__m256d v_Ro22_1=ld(slots[46],i,j+4);__m256d v_Ro37_0=_mm256_sub_pd(v_Rx15_0,v_Ro22_0);__m256d v_Ro37_1=_mm256_sub_pd(v_Rx15_1,v_Ro22_1);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+0,v_Ro37_0);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+4,v_Ro37_1);__m256d v_Ro14_0=ld(slots[5],i,j+0);__m256d v_Ro14_1=ld(slots[5],i,j+4);__m256d v_Rx12_0=ld(slots[48],i,j+0);__m256d v_Rx12_1=ld(slots[48],i,j+4);__m256d v_Ro47_0=_mm256_sub_pd(v_Ro14_0,v_Rx12_0);__m256d v_Ro47_1=_mm256_sub_pd(v_Ro14_1,v_Rx12_1);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+0,v_Ro47_0);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+4,v_Ro47_1);__m256d v_Ro42_0=ld(slots[39],i,j+0);__m256d v_Ro42_1=ld(slots[39],i,j+4);__m256d v_Ro38_0=_mm256_add_pd(v_Ro42_0,v_Rx12_0);__m256d v_Ro38_1=_mm256_add_pd(v_Ro42_1,v_Rx12_1);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+0,v_Ro38_0);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+4,v_Ro38_1);__m256d v_Ri1_0=ld(bb[1],i,j+0);__m256d v_Ri1_1=ld(bb[1],i,j+4);__m256d v_Rx24_0=ld(slots[50],i,j+0);__m256d v_Rx24_1=ld(slots[50],i,j+4);__m256d v_Ro0_0=_mm256_add_pd(v_Ri1_0,v_Rx24_0);__m256d v_Ro0_1=_mm256_add_pd(v_Ri1_1,v_Rx24_1);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+0,v_Ro0_0);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+4,v_Ro0_1);__m256d v_Rx10_0=ld(slots[41],i,j+0);__m256d v_Rx10_1=ld(slots[41],i,j+4);__m256d v_Rx25_0=_mm256_sub_pd(v_Rx24_0,v_Rx10_0);__m256d v_Rx25_1=_mm256_sub_pd(v_Rx24_1,v_Rx10_1);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+0,v_Rx25_0);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+4,v_Rx25_1);__m256d v_Ro27_0=_mm256_add_pd(v_Ri1_0,v_Rx25_0);__m256d v_Ro27_1=_mm256_add_pd(v_Ri1_1,v_Rx25_1);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+0,v_Ro27_0);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+4,v_Ro27_1);__m256d v_Ri12_0=ld(bb[12],i,j+0);__m256d v_Ri12_1=ld(bb[12],i,j+4);__m256d v_Ro34_0=_mm256_sub_pd(v_Rx10_0,v_Ri12_0);__m256d v_Ro34_1=_mm256_sub_pd(v_Rx10_1,v_Ri12_1);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+0,v_Ro34_0);Q0(const_cast<double*>(slots[41].x)+(size_t)i*slots[41].ld+j+4,v_Ro34_1);}_mm_sfence();rec(1024,slots[15],slots[46],const_cast<double*>(slots[53].x),slots[53].ld,work);rec(1024,slots[8],slots[17],const_cast<double*>(slots[15].x),slots[15].ld,work);rec(1024,slots[2],slots[39],const_cast<double*>(slots[17].x),slots[17].ld,work);rec(1024,slots[20],slots[48],const_cast<double*>(slots[39].x),slots[39].ld,work);rec(1024,slots[9],slots[21],const_cast<double*>(slots[20].x),slots[20].ld,work);for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_Ro27_0=ld(slots[21],i,j+0);__m256d v_Ro27_1=ld(slots[21],i,j+4);__m256d v_Rx14_0=ld(slots[47],i,j+0);__m256d v_Rx14_1=ld(slots[47],i,j+4);__m256d v_Ro25_0=_mm256_sub_pd(v_Ro27_0,v_Rx14_0);__m256d v_Ro25_1=_mm256_sub_pd(v_Ro27_1,v_Rx14_1);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+0,v_Ro25_0);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+4,v_Ro25_1);__m256d v_Ro34_0=ld(slots[41],i,j+0);__m256d v_Ro34_1=ld(slots[41],i,j+4);__m256d v_Ro45_0=_mm256_sub_pd(v_Rx14_0,v_Ro34_0);__m256d v_Ro45_1=_mm256_sub_pd(v_Rx14_1,v_Ro34_1);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+0,v_Ro45_0);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+4,v_Ro45_1);}_mm_sfence();rec(1024,slots[12],slots[47],const_cast<double*>(slots[9].x),slots[9].ld,work);rec(1024,slots[24],bb[13],const_cast<double*>(slots[12].x),slots[12].ld,work);rec(1024,slots[26],slots[5],const_cast<double*>(slots[24].x),slots[24].ld,work);rec(1024,aa[1],slots[21],const_cast<double*>(slots[26].x),slots[26].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M25_0=ld(slots[26],i,j+0);__m256d v_M25_1=ld(slots[26],i,j+4);__m256d v_M27_0=ld(slots[20],i,j+0);__m256d v_M27_1=ld(slots[20],i,j+4);__m256d v_Pz33_0=_mm256_sub_pd(v_M25_0,v_M27_0);__m256d v_Pz33_1=_mm256_sub_pd(v_M25_1,v_M27_1);Q0(const_cast<double*>(slots[26].x)+(size_t)i*slots[26].ld+j+0,v_Pz33_0);Q0(const_cast<double*>(slots[26].x)+(size_t)i*slots[26].ld+j+4,v_Pz33_1);__m256d v_Ri10_0=ld(bb[10],i,j+0);__m256d v_Ri10_1=ld(bb[10],i,j+4);__m256d v_Ri14_0=ld(bb[14],i,j+0);__m256d v_Ri14_1=ld(bb[14],i,j+4);__m256d v_Ri6_0=ld(bb[6],i,j+0);__m256d v_Ri6_1=ld(bb[6],i,j+4);__m256d v_Ro4_0=_mm256_sub_pd(_mm256_add_pd(v_Ri10_0,v_Ri14_0),v_Ri6_0);__m256d v_Ro4_1=_mm256_sub_pd(_mm256_add_pd(v_Ri10_1,v_Ri14_1),v_Ri6_1);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+0,v_Ro4_0);Q0(const_cast<double*>(slots[21].x)+(size_t)i*slots[21].ld+j+4,v_Ro4_1);__m256d v_Rx15_0=ld(slots[49],i,j+0);__m256d v_Rx15_1=ld(slots[49],i,j+4);__m256d v_Ro24_0=_mm256_sub_pd(v_Ro4_0,v_Rx15_0);__m256d v_Ro24_1=_mm256_sub_pd(v_Ro4_1,v_Rx15_1);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+0,v_Ro24_0);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+4,v_Ro24_1);__m256d v_Ro47_0=ld(slots[5],i,j+0);__m256d v_Ro47_1=ld(slots[5],i,j+4);__m256d v_Ro6_0=_mm256_sub_pd(_mm256_sub_pd(v_Ro24_0,v_Ro47_0),v_Ri14_0);__m256d v_Ro6_1=_mm256_sub_pd(_mm256_sub_pd(v_Ro24_1,v_Ro47_1),v_Ri14_1);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+0,v_Ro6_0);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+4,v_Ro6_1);__m256d v_Ro34_0=ld(slots[41],i,j+0);__m256d v_Ro34_1=ld(slots[41],i,j+4);__m256d v_Rx13_0=_mm256_sub_pd(v_Ro34_0,v_Ro24_0);__m256d v_Rx13_1=_mm256_sub_pd(v_Ro34_1,v_Ro24_1);__m256d v_Ri13_0=ld(bb[13],i,j+0);__m256d v_Ri13_1=ld(bb[13],i,j+4);__m256d v_Ro10_0=_mm256_sub_pd(v_Ri13_0,v_Rx13_0);__m256d v_Ro10_1=_mm256_sub_pd(v_Ri13_1,v_Rx13_1);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+0,v_Ro10_0);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+4,v_Ro10_1);__m256d v_Ro18_0=ld(slots[29],i,j+0);__m256d v_Ro18_1=ld(slots[29],i,j+4);__m256d v_Ro31_0=_mm256_sub_pd(v_Ro18_0,v_Rx13_0);__m256d v_Ro31_1=_mm256_sub_pd(v_Ro18_1,v_Rx13_1);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+0,v_Ro31_0);Q0(const_cast<double*>(slots[29].x)+(size_t)i*slots[29].ld+j+4,v_Ro31_1);__m256d v_Rx25_0=ld(slots[50],i,j+0);__m256d v_Rx25_1=ld(slots[50],i,j+4);__m256d v_Rx26_0=_mm256_sub_pd(v_Rx25_0,v_Ro24_0);__m256d v_Rx26_1=_mm256_sub_pd(v_Rx25_1,v_Ro24_1);__m256d v_Rx17_0=ld(slots[42],i,j+0);__m256d v_Rx17_1=ld(slots[42],i,j+4);__m256d v_Ro35_0=_mm256_add_pd(v_Rx17_0,v_Rx26_0);__m256d v_Ro35_1=_mm256_add_pd(v_Rx17_1,v_Rx26_1);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+0,v_Ro35_0);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+4,v_Ro35_1);__m256d v_Rx20_0=ld(slots[43],i,j+0);__m256d v_Rx20_1=ld(slots[43],i,j+4);__m256d v_Ro21_0=_mm256_sub_pd(v_Rx20_0,v_Rx26_0);__m256d v_Ro21_1=_mm256_sub_pd(v_Rx20_1,v_Rx26_1);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+0,v_Ro21_0);Q0(const_cast<double*>(slots[48].x)+(size_t)i*slots[48].ld+j+4,v_Ro21_1);__m256d v_Ro19_0=_mm256_add_pd(v_Rx20_0,v_Ro31_0);__m256d v_Ro19_1=_mm256_add_pd(v_Rx20_1,v_Ro31_1);Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+0,v_Ro19_0);Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+4,v_Ro19_1);__m256d v_Ro8_0=_mm256_sub_pd(v_Ro19_0,v_Rx25_0);__m256d v_Ro8_1=_mm256_sub_pd(v_Ro19_1,v_Rx25_1);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+0,v_Ro8_0);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+4,v_Ro8_1);__m256d v_Ro23_0=_mm256_sub_pd(v_Ro8_0,v_Rx15_0);__m256d v_Ro23_1=_mm256_sub_pd(v_Ro8_1,v_Rx15_1);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+0,v_Ro23_0);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+4,v_Ro23_1);__m256d v_Ri15_0=ld(bb[15],i,j+0);__m256d v_Ri15_1=ld(bb[15],i,j+4);__m256d v_Ro7_0=_mm256_add_pd(_mm256_add_pd(v_Ri15_0,v_Ro24_0),v_Ro34_0);__m256d v_Ro7_1=_mm256_add_pd(_mm256_add_pd(v_Ri15_1,v_Ro24_1),v_Ro34_1);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+0,v_Ro7_0);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+4,v_Ro7_1);__m256d v_Rx22_0=_mm256_sub_pd(v_Ro21_0,v_Ro7_0);__m256d v_Rx22_1=_mm256_sub_pd(v_Ro21_1,v_Ro7_1);__m256d v_Ro11_0=_mm256_add_pd(v_Ri15_0,v_Rx22_0);__m256d v_Ro11_1=_mm256_add_pd(v_Ri15_1,v_Rx22_1);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+0,v_Ro11_0);Q0(const_cast<double*>(slots[46].x)+(size_t)i*slots[46].ld+j+4,v_Ro11_1);__m256d v_Ro39_0=ld(slots[44],i,j+0);__m256d v_Ro39_1=ld(slots[44],i,j+4);__m256d v_Ro30_0=_mm256_sub_pd(v_Ro39_0,v_Rx22_0);__m256d v_Ro30_1=_mm256_sub_pd(v_Ro39_1,v_Rx22_1);Q0(const_cast<double*>(slots[54].x)+(size_t)i*slots[54].ld+j+0,v_Ro30_0);Q0(const_cast<double*>(slots[54].x)+(size_t)i*slots[54].ld+j+4,v_Ro30_1);__m256d v_Ro33_0=ld(slots[40],i,j+0);__m256d v_Ro33_1=ld(slots[40],i,j+4);__m256d v_Ro41_0=_mm256_sub_pd(v_Rx22_0,v_Ro33_0);__m256d v_Ro41_1=_mm256_sub_pd(v_Rx22_1,v_Ro33_1);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+0,v_Ro41_0);Q0(const_cast<double*>(slots[40].x)+(size_t)i*slots[40].ld+j+4,v_Ro41_1);}_mm_sfence();rec(1024,slots[10],slots[50],const_cast<double*>(slots[55].x),slots[55].ld,work);rec(1024,slots[19],slots[47],const_cast<double*>(slots[50].x),slots[50].ld,work);rec(1024,slots[13],slots[41],const_cast<double*>(slots[47].x),slots[47].ld,work);rec(1024,slots[33],slots[2],const_cast<double*>(slots[13].x),slots[13].ld,work);rec(1024,slots[25],slots[46],const_cast<double*>(slots[2].x),slots[2].ld,work);rec(1024,slots[16],slots[44],const_cast<double*>(slots[25].x),slots[25].ld,work);rec(1024,slots[1],slots[21],const_cast<double*>(slots[44].x),slots[44].ld,work);rec(1024,slots[6],slots[43],const_cast<double*>(slots[1].x),slots[1].ld,work);rec(1024,slots[32],slots[29],const_cast<double*>(slots[43].x),slots[43].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M31_0=ld(slots[43],i,j+0);__m256d v_M31_1=ld(slots[43],i,j+4);__m256d v_M19_0=ld(slots[1],i,j+0);__m256d v_M19_1=ld(slots[1],i,j+4);__m256d v_Pz34_0=_mm256_add_pd(v_M31_0,v_M19_0);__m256d v_Pz34_1=_mm256_add_pd(v_M31_1,v_M19_1);Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+0,v_Pz34_0);Q0(const_cast<double*>(slots[43].x)+(size_t)i*slots[43].ld+j+4,v_Pz34_1);__m256d v_M37_0=ld(slots[53],i,j+0);__m256d v_M37_1=ld(slots[53],i,j+4);__m256d v_Pz33_0=ld(slots[26],i,j+0);__m256d v_Pz33_1=ld(slots[26],i,j+4);__m256d v_Pz22_0=_mm256_sub_pd(_mm256_sub_pd(v_M37_0,v_Pz34_0),v_Pz33_0);__m256d v_Pz22_1=_mm256_sub_pd(_mm256_sub_pd(v_M37_1,v_Pz34_1),v_Pz33_1);__m256d v_M22_0=ld(slots[28],i,j+0);__m256d v_M22_1=ld(slots[28],i,j+4);__m256d v_Pz31_0=_mm256_add_pd(v_M22_0,v_Pz22_0);__m256d v_Pz31_1=_mm256_add_pd(v_M22_1,v_Pz22_1);Q0(const_cast<double*>(slots[28].x)+(size_t)i*slots[28].ld+j+0,v_Pz31_0);Q0(const_cast<double*>(slots[28].x)+(size_t)i*slots[28].ld+j+4,v_Pz31_1);__m256d v_M36_0=ld(slots[4],i,j+0);__m256d v_M36_1=ld(slots[4],i,j+4);__m256d v_Pz36_0=_mm256_sub_pd(v_M36_0,v_Pz22_0);__m256d v_Pz36_1=_mm256_sub_pd(v_M36_1,v_Pz22_1);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+0,v_Pz36_0);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+4,v_Pz36_1);}_mm_sfence();rec(1024,slots[34],slots[48],const_cast<double*>(slots[53].x),slots[53].ld,work);rec(1024,slots[37],slots[5],const_cast<double*>(slots[34].x),slots[34].ld,work);rec(1024,aa[7],slots[54],const_cast<double*>(slots[37].x),slots[37].ld,work);rec(1024,aa[6],slots[49],const_cast<double*>(slots[54].x),slots[54].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M8_0=ld(slots[55],i,j+0);__m256d v_M8_1=ld(slots[55],i,j+4);__m256d v_M23_0=ld(slots[54],i,j+0);__m256d v_M23_1=ld(slots[54],i,j+4);__m256d v_Pz42_0=_mm256_add_pd(v_M8_0,v_M23_0);__m256d v_Pz42_1=_mm256_add_pd(v_M8_1,v_M23_1);Q0(const_cast<double*>(slots[55].x)+(size_t)i*slots[55].ld+j+0,v_Pz42_0);Q0(const_cast<double*>(slots[55].x)+(size_t)i*slots[55].ld+j+4,v_Pz42_1);__m256d v_M16_0=ld(slots[0],i,j+0);__m256d v_M16_1=ld(slots[0],i,j+4);__m256d v_Pz30_0=_mm256_sub_pd(v_M16_0,v_Pz42_0);__m256d v_Pz30_1=_mm256_sub_pd(v_M16_1,v_Pz42_1);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+0,v_Pz30_0);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+4,v_Pz30_1);__m256d v_M34_0=ld(slots[47],i,j+0);__m256d v_M34_1=ld(slots[47],i,j+4);__m256d v_Pz35_0=_mm256_add_pd(v_M34_0,v_Pz42_0);__m256d v_Pz35_1=_mm256_add_pd(v_M34_1,v_Pz42_1);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+0,v_Pz35_0);Q0(const_cast<double*>(slots[47].x)+(size_t)i*slots[47].ld+j+4,v_Pz35_1);}_mm_sfence();rec(1024,slots[31],bb[6],const_cast<double*>(slots[49].x),slots[49].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M46_0=ld(slots[49],i,j+0);__m256d v_M46_1=ld(slots[49],i,j+4);__m256d v_Pz30_0=ld(slots[0],i,j+0);__m256d v_Pz30_1=ld(slots[0],i,j+4);__m256d v_Pz40_0=_mm256_add_pd(v_M46_0,v_Pz30_0);__m256d v_Pz40_1=_mm256_add_pd(v_M46_1,v_Pz30_1);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+0,v_Pz40_0);Q0(const_cast<double*>(slots[49].x)+(size_t)i*slots[49].ld+j+4,v_Pz40_1);}_mm_sfence();rec(1024,slots[27],bb[14],const_cast<double*>(slots[31].x),slots[31].ld,work);rec(1024,aa[11],slots[8],const_cast<double*>(slots[27].x),slots[27].ld,work);rec(1024,aa[8],slots[42],const_cast<double*>(slots[8].x),slots[8].ld,work);rec(1024,slots[3],bb[1],const_cast<double*>(slots[42].x),slots[42].ld,work);rec(1024,aa[3],slots[40],const_cast<double*>(slots[3].x),slots[3].ld,work);rec(1024,aa[13],bb[7],const_cast<double*>(slots[40].x),slots[40].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M43_0=ld(slots[22],i,j+0);__m256d v_M43_1=ld(slots[22],i,j+4);__m256d v_M9_0=ld(slots[40],i,j+0);__m256d v_M9_1=ld(slots[40],i,j+4);__m256d v_Pz39_0=_mm256_add_pd(v_M43_0,v_M9_0);__m256d v_Pz39_1=_mm256_add_pd(v_M43_1,v_M9_1);__m256d v_M47_0=ld(slots[24],i,j+0);__m256d v_M47_1=ld(slots[24],i,j+4);__m256d v_Pz41_0=_mm256_sub_pd(v_Pz39_0,v_M47_0);__m256d v_Pz41_1=_mm256_sub_pd(v_Pz39_1,v_M47_1);Q0(const_cast<double*>(slots[24].x)+(size_t)i*slots[24].ld+j+0,v_Pz41_0);Q0(const_cast<double*>(slots[24].x)+(size_t)i*slots[24].ld+j+4,v_Pz41_1);__m256d v_M21_0=ld(slots[53],i,j+0);__m256d v_M21_1=ld(slots[53],i,j+4);__m256d v_M35_0=ld(slots[8],i,j+0);__m256d v_M35_1=ld(slots[8],i,j+4);__m256d v_Pz23_0=_mm256_add_pd(_mm256_sub_pd(v_Pz39_0,v_M21_0),v_M35_0);__m256d v_Pz23_1=_mm256_add_pd(_mm256_sub_pd(v_Pz39_1,v_M21_1),v_M35_1);__m256d v_M4_0=ld(slots[44],i,j+0);__m256d v_M4_1=ld(slots[44],i,j+4);__m256d v_Pz37_0=_mm256_sub_pd(v_M4_0,v_Pz23_0);__m256d v_Pz37_1=_mm256_sub_pd(v_M4_1,v_Pz23_1);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+0,v_Pz37_0);Q0(const_cast<double*>(slots[44].x)+(size_t)i*slots[44].ld+j+4,v_Pz37_1);__m256d v_M24_0=ld(slots[50],i,j+0);__m256d v_M24_1=ld(slots[50],i,j+4);__m256d v_Pz32_0=_mm256_sub_pd(v_M24_0,v_Pz37_0);__m256d v_Pz32_1=_mm256_sub_pd(v_M24_1,v_Pz37_1);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+0,v_Pz32_0);Q0(const_cast<double*>(slots[50].x)+(size_t)i*slots[50].ld+j+4,v_Pz32_1);__m256d v_M18_0=ld(slots[7],i,j+0);__m256d v_M18_1=ld(slots[7],i,j+4);__m256d v_M17_0=ld(slots[42],i,j+0);__m256d v_M17_1=ld(slots[42],i,j+4);__m256d v_Pz10_0=_mm256_sub_pd(_mm256_sub_pd(v_M18_0,v_M17_0),v_Pz39_0);__m256d v_Pz10_1=_mm256_sub_pd(_mm256_sub_pd(v_M18_1,v_M17_1),v_Pz39_1);__m256d v_Pz29_0=ld(slots[11],i,j+0);__m256d v_Pz29_1=ld(slots[11],i,j+4);__m256d v_Pz43_0=_mm256_add_pd(v_Pz10_0,v_Pz29_0);__m256d v_Pz43_1=_mm256_add_pd(v_Pz10_1,v_Pz29_1);Q0(const_cast<double*>(slots[7].x)+(size_t)i*slots[7].ld+j+0,v_Pz43_0);Q0(const_cast<double*>(slots[7].x)+(size_t)i*slots[7].ld+j+4,v_Pz43_1);__m256d v_Pz40_0=ld(slots[49],i,j+0);__m256d v_Pz40_1=ld(slots[49],i,j+4);__m256d v_Pz45_0=_mm256_sub_pd(v_Pz10_0,v_Pz40_0);__m256d v_Pz45_1=_mm256_sub_pd(v_Pz10_1,v_Pz40_1);Q0(const_cast<double*>(slots[53].x)+(size_t)i*slots[53].ld+j+0,v_Pz45_0);Q0(const_cast<double*>(slots[53].x)+(size_t)i*slots[53].ld+j+4,v_Pz45_1);__m256d v_M14_0=ld(slots[14],i,j+0);__m256d v_M14_1=ld(slots[14],i,j+4);__m256d v_Pz34_0=ld(slots[43],i,j+0);__m256d v_Pz34_1=ld(slots[43],i,j+4);__m256d v_Pz27_0=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M14_0,v_Pz34_0),v_Pz45_0),v_Pz43_0);__m256d v_Pz27_1=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M14_1,v_Pz34_1),v_Pz45_1),v_Pz43_1);Q0(const_cast<double*>(slots[14].x)+(size_t)i*slots[14].ld+j+0,v_Pz27_0);Q0(const_cast<double*>(slots[14].x)+(size_t)i*slots[14].ld+j+4,v_Pz27_1);__m256d v_M42_0=ld(slots[17],i,j+0);__m256d v_M42_1=ld(slots[17],i,j+4);__m256d v_Pz38_0=_mm256_add_pd(v_M42_0,v_Pz27_0);__m256d v_Pz38_1=_mm256_add_pd(v_M42_1,v_Pz27_1);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+0,v_Pz38_0);Q0(const_cast<double*>(slots[17].x)+(size_t)i*slots[17].ld+j+4,v_Pz38_1);__m256d v_M1_0=ld(slots[35],i,j+0);__m256d v_M1_1=ld(slots[35],i,j+4);__m256d v_M39_0=ld(slots[25],i,j+0);__m256d v_M39_1=ld(slots[25],i,j+4);__m256d v_Pz17_0=_mm256_add_pd(_mm256_add_pd(v_M1_0,v_Pz45_0),v_M39_0);__m256d v_Pz17_1=_mm256_add_pd(_mm256_add_pd(v_M1_1,v_Pz45_1),v_M39_1);Q0(const_cast<double*>(slots[35].x)+(size_t)i*slots[35].ld+j+0,v_Pz17_0);Q0(const_cast<double*>(slots[35].x)+(size_t)i*slots[35].ld+j+4,v_Pz17_1);__m256d v_Pz36_0=ld(slots[4],i,j+0);__m256d v_Pz36_1=ld(slots[4],i,j+4);__m256d v_Po5_0=_mm256_sub_pd(v_Pz17_0,v_Pz36_0);__m256d v_Po5_1=_mm256_sub_pd(v_Pz17_1,v_Pz36_1);Q0(const_cast<double*>(slots[22].x)+(size_t)i*slots[22].ld+j+0,v_Po5_0);Q0(const_cast<double*>(slots[22].x)+(size_t)i*slots[22].ld+j+4,v_Po5_1);__m256d v_M26_0=ld(slots[18],i,j+0);__m256d v_M26_1=ld(slots[18],i,j+4);__m256d v_Pz30_0=ld(slots[0],i,j+0);__m256d v_Pz30_1=ld(slots[0],i,j+4);__m256d v_Pz24_0=_mm256_sub_pd(_mm256_add_pd(v_M26_0,v_Pz30_0),v_Pz23_0);__m256d v_Pz24_1=_mm256_sub_pd(_mm256_add_pd(v_M26_1,v_Pz30_1),v_Pz23_1);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+0,v_Pz24_0);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+4,v_Pz24_1);__m256d v_Pz20_0=_mm256_sub_pd(v_Pz36_0,v_Pz24_0);__m256d v_Pz20_1=_mm256_sub_pd(v_Pz36_1,v_Pz24_1);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+0,v_Pz20_0);Q0(const_cast<double*>(slots[4].x)+(size_t)i*slots[4].ld+j+4,v_Pz20_1);__m256d v_Pz33_0=ld(slots[26],i,j+0);__m256d v_Pz33_1=ld(slots[26],i,j+4);__m256d v_Pz44_0=_mm256_add_pd(v_Pz33_0,v_Pz23_0);__m256d v_Pz44_1=_mm256_add_pd(v_Pz33_1,v_Pz23_1);__m256d v_M45_0=ld(slots[9],i,j+0);__m256d v_M45_1=ld(slots[9],i,j+4);__m256d v_Pz35_0=ld(slots[47],i,j+0);__m256d v_Pz35_1=ld(slots[47],i,j+4);__m256d v_Pz16_0=_mm256_add_pd(_mm256_add_pd(v_M45_0,v_Pz44_0),v_Pz35_0);__m256d v_Pz16_1=_mm256_add_pd(_mm256_add_pd(v_M45_1,v_Pz44_1),v_Pz35_1);Q0(const_cast<double*>(slots[9].x)+(size_t)i*slots[9].ld+j+0,v_Pz16_0);Q0(const_cast<double*>(slots[9].x)+(size_t)i*slots[9].ld+j+4,v_Pz16_1);__m256d v_Po8_0=_mm256_add_pd(v_Pz38_0,v_Pz16_0);__m256d v_Po8_1=_mm256_add_pd(v_Pz38_1,v_Pz16_1);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+0,v_Po8_0);Q0(const_cast<double*>(slots[0].x)+(size_t)i*slots[0].ld+j+4,v_Po8_1);__m256d v_M3_0=ld(slots[52],i,j+0);__m256d v_M3_1=ld(slots[52],i,j+4);__m256d v_M29_0=ld(slots[51],i,j+0);__m256d v_M29_1=ld(slots[51],i,j+4);__m256d v_Pz13_0=_mm256_sub_pd(_mm256_sub_pd(v_Pz44_0,v_M3_0),v_M29_0);__m256d v_Pz13_1=_mm256_sub_pd(_mm256_sub_pd(v_Pz44_1,v_M3_1),v_M29_1);Q0(const_cast<double*>(slots[51].x)+(size_t)i*slots[51].ld+j+0,v_Pz13_0);Q0(const_cast<double*>(slots[51].x)+(size_t)i*slots[51].ld+j+4,v_Pz13_1);__m256d v_Pz31_0=ld(slots[28],i,j+0);__m256d v_Pz31_1=ld(slots[28],i,j+4);__m256d v_Pz19_0=_mm256_add_pd(v_Pz13_0,v_Pz31_0);__m256d v_Pz19_1=_mm256_add_pd(v_Pz13_1,v_Pz31_1);Q0(const_cast<double*>(slots[52].x)+(size_t)i*slots[52].ld+j+0,v_Pz19_0);Q0(const_cast<double*>(slots[52].x)+(size_t)i*slots[52].ld+j+4,v_Pz19_1);}_mm_sfence();rec(1024,aa[15],bb[15],const_cast<double*>(slots[5].x),slots[5].ld,work);rec(1024,aa[12],bb[3],const_cast<double*>(slots[48].x),slots[48].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M40_0=ld(slots[48],i,j+0);__m256d v_M40_1=ld(slots[48],i,j+4);__m256d v_M11_0=ld(slots[2],i,j+0);__m256d v_M11_1=ld(slots[2],i,j+4);__m256d v_Pz34_0=ld(slots[43],i,j+0);__m256d v_Pz34_1=ld(slots[43],i,j+4);__m256d v_Pz25_0=_mm256_sub_pd(_mm256_sub_pd(v_M40_0,v_M11_0),v_Pz34_0);__m256d v_Pz25_1=_mm256_sub_pd(_mm256_sub_pd(v_M40_1,v_M11_1),v_Pz34_1);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+0,v_Pz25_0);Q0(const_cast<double*>(slots[2].x)+(size_t)i*slots[2].ld+j+4,v_Pz25_1);__m256d v_M0_0=ld(slots[15],i,j+0);__m256d v_M0_1=ld(slots[15],i,j+4);__m256d v_M27_0=ld(slots[20],i,j+0);__m256d v_M27_1=ld(slots[20],i,j+4);__m256d v_Pz43_0=ld(slots[7],i,j+0);__m256d v_Pz43_1=ld(slots[7],i,j+4);__m256d v_Pz11_0=_mm256_sub_pd(_mm256_add_pd(_mm256_sub_pd(v_M0_0,v_M27_0),v_Pz43_0),v_Pz25_0);__m256d v_Pz11_1=_mm256_sub_pd(_mm256_add_pd(_mm256_sub_pd(v_M0_1,v_M27_1),v_Pz43_1),v_Pz25_1);__m256d v_Pz13_0=ld(slots[51],i,j+0);__m256d v_Pz13_1=ld(slots[51],i,j+4);__m256d v_Po0_0=_mm256_add_pd(v_Pz13_0,v_Pz11_0);__m256d v_Po0_1=_mm256_add_pd(v_Pz13_1,v_Pz11_1);Q0(const_cast<double*>(slots[15].x)+(size_t)i*slots[15].ld+j+0,v_Po0_0);Q0(const_cast<double*>(slots[15].x)+(size_t)i*slots[15].ld+j+4,v_Po0_1);__m256d v_Pz38_0=ld(slots[17],i,j+0);__m256d v_Pz38_1=ld(slots[17],i,j+4);__m256d v_Pz21_0=_mm256_sub_pd(v_Pz38_0,v_Pz11_0);__m256d v_Pz21_1=_mm256_sub_pd(v_Pz38_1,v_Pz11_1);__m256d v_M44_0=ld(slots[30],i,j+0);__m256d v_M44_1=ld(slots[30],i,j+4);__m256d v_Pz29_0=ld(slots[11],i,j+0);__m256d v_Pz29_1=ld(slots[11],i,j+4);__m256d v_Pz35_0=ld(slots[47],i,j+0);__m256d v_Pz35_1=ld(slots[47],i,j+4);__m256d v_Po12_0=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(v_M44_0,v_Pz29_0),v_Pz35_0),v_Pz21_0);__m256d v_Po12_1=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(v_M44_1,v_Pz29_1),v_Pz35_1),v_Pz21_1);Q0(const_cast<double*>(slots[30].x)+(size_t)i*slots[30].ld+j+0,v_Po12_0);Q0(const_cast<double*>(slots[30].x)+(size_t)i*slots[30].ld+j+4,v_Po12_1);__m256d v_M38_0=ld(slots[39],i,j+0);__m256d v_M38_1=ld(slots[39],i,j+4);__m256d v_Pz20_0=ld(slots[4],i,j+0);__m256d v_Pz20_1=ld(slots[4],i,j+4);__m256d v_Pz41_0=ld(slots[24],i,j+0);__m256d v_Pz41_1=ld(slots[24],i,j+4);__m256d v_Po4_0=_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M38_0,v_Pz21_0),v_Pz20_0),v_Pz41_0);__m256d v_Po4_1=_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M38_1,v_Pz21_1),v_Pz20_1),v_Pz41_1);Q0(const_cast<double*>(slots[39].x)+(size_t)i*slots[39].ld+j+0,v_Po4_0);Q0(const_cast<double*>(slots[39].x)+(size_t)i*slots[39].ld+j+4,v_Po4_1);__m256d v_M30_0=ld(slots[37],i,j+0);__m256d v_M30_1=ld(slots[37],i,j+4);__m256d v_M39_0=ld(slots[25],i,j+0);__m256d v_M39_1=ld(slots[25],i,j+4);__m256d v_Po7_0=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M30_0,v_Pz25_0),v_M39_0),v_Pz20_0);__m256d v_Po7_1=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M30_1,v_Pz25_1),v_M39_1),v_Pz20_1);Q0(const_cast<double*>(slots[37].x)+(size_t)i*slots[37].ld+j+0,v_Po7_0);Q0(const_cast<double*>(slots[37].x)+(size_t)i*slots[37].ld+j+4,v_Po7_1);__m256d v_M28_0=ld(slots[36],i,j+0);__m256d v_M28_1=ld(slots[36],i,j+4);__m256d v_M23_0=ld(slots[54],i,j+0);__m256d v_M23_1=ld(slots[54],i,j+4);__m256d v_Pz45_0=ld(slots[53],i,j+0);__m256d v_Pz45_1=ld(slots[53],i,j+4);__m256d v_Pz12_0=_mm256_sub_pd(_mm256_sub_pd(_mm256_add_pd(v_M28_0,v_M23_0),v_Pz45_0),v_Pz25_0);__m256d v_Pz12_1=_mm256_sub_pd(_mm256_sub_pd(_mm256_add_pd(v_M28_1,v_M23_1),v_Pz45_1),v_Pz25_1);Q0(const_cast<double*>(slots[54].x)+(size_t)i*slots[54].ld+j+0,v_Pz12_0);Q0(const_cast<double*>(slots[54].x)+(size_t)i*slots[54].ld+j+4,v_Pz12_1);__m256d v_Pz24_0=ld(slots[18],i,j+0);__m256d v_Pz24_1=ld(slots[18],i,j+4);__m256d v_Po6_0=_mm256_add_pd(v_Pz24_0,v_Pz12_0);__m256d v_Po6_1=_mm256_add_pd(v_Pz24_1,v_Pz12_1);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+0,v_Po6_0);Q0(const_cast<double*>(slots[18].x)+(size_t)i*slots[18].ld+j+4,v_Po6_1);}_mm_sfence();rec(1024,aa[14],bb[11],const_cast<double*>(slots[51].x),slots[51].ld,work);pending_finish();for(int i=0;i<1024;i++)for(int j=0;j<1024;j+=8){__m256d v_M12_0=ld(slots[51],i,j+0);__m256d v_M12_1=ld(slots[51],i,j+4);__m256d v_M9_0=ld(slots[40],i,j+0);__m256d v_M9_1=ld(slots[40],i,j+4);__m256d v_Pz28_0=_mm256_add_pd(v_M12_0,v_M9_0);__m256d v_Pz28_1=_mm256_add_pd(v_M12_1,v_M9_1);__m256d v_M15_0=ld(slots[5],i,j+0);__m256d v_M15_1=ld(slots[5],i,j+4);__m256d v_M40_0=ld(slots[48],i,j+0);__m256d v_M40_1=ld(slots[48],i,j+4);__m256d v_Po15_0=_mm256_add_pd(_mm256_add_pd(v_M15_0,v_M40_0),v_Pz28_0);__m256d v_Po15_1=_mm256_add_pd(_mm256_add_pd(v_M15_1,v_M40_1),v_Pz28_1);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+0,v_Po15_0);Q0(const_cast<double*>(slots[5].x)+(size_t)i*slots[5].ld+j+4,v_Po15_1);__m256d v_Pz42_0=ld(slots[55],i,j+0);__m256d v_Pz42_1=ld(slots[55],i,j+4);__m256d v_M33_0=ld(slots[23],i,j+0);__m256d v_M33_1=ld(slots[23],i,j+4);__m256d v_Pz26_0=_mm256_sub_pd(_mm256_sub_pd(v_Pz42_0,v_Pz28_0),v_M33_0);__m256d v_Pz26_1=_mm256_sub_pd(_mm256_sub_pd(v_Pz42_1,v_Pz28_1),v_M33_1);__m256d v_Pz32_0=ld(slots[50],i,j+0);__m256d v_Pz32_1=ld(slots[50],i,j+4);__m256d v_Pz14_0=_mm256_sub_pd(v_Pz32_0,v_Pz28_0);__m256d v_Pz14_1=_mm256_sub_pd(v_Pz32_1,v_Pz28_1);__m256d v_M2_0=ld(slots[45],i,j+0);__m256d v_M2_1=ld(slots[45],i,j+4);__m256d v_Pz27_0=ld(slots[14],i,j+0);__m256d v_Pz27_1=ld(slots[14],i,j+4);__m256d v_Po10_0=_mm256_sub_pd(_mm256_add_pd(v_M2_0,v_Pz27_0),v_Pz14_0);__m256d v_Po10_1=_mm256_sub_pd(_mm256_add_pd(v_M2_1,v_Pz27_1),v_Pz14_1);Q0(const_cast<double*>(slots[45].x)+(size_t)i*slots[45].ld+j+0,v_Po10_0);Q0(const_cast<double*>(slots[45].x)+(size_t)i*slots[45].ld+j+4,v_Po10_1);__m256d v_Pz12_0=ld(slots[54],i,j+0);__m256d v_Pz12_1=ld(slots[54],i,j+4);__m256d v_Pz18_0=_mm256_sub_pd(v_Po10_0,v_Pz12_0);__m256d v_Pz18_1=_mm256_sub_pd(v_Po10_1,v_Pz12_1);__m256d v_M32_0=ld(slots[31],i,j+0);__m256d v_M32_1=ld(slots[31],i,j+4);__m256d v_Pz40_0=ld(slots[49],i,j+0);__m256d v_Pz40_1=ld(slots[49],i,j+4);__m256d v_Pz37_0=ld(slots[44],i,j+0);__m256d v_Pz37_1=ld(slots[44],i,j+4);__m256d v_Po14_0=_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M32_0,v_Pz40_0),v_Pz37_0),v_Pz18_0);__m256d v_Po14_1=_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M32_1,v_Pz40_1),v_Pz37_1),v_Pz18_1);Q0(const_cast<double*>(slots[31].x)+(size_t)i*slots[31].ld+j+0,v_Po14_0);Q0(const_cast<double*>(slots[31].x)+(size_t)i*slots[31].ld+j+4,v_Po14_1);__m256d v_M6_0=ld(slots[34],i,j+0);__m256d v_M6_1=ld(slots[34],i,j+4);__m256d v_Pz19_0=ld(slots[52],i,j+0);__m256d v_Pz19_1=ld(slots[52],i,j+4);__m256d v_Pz41_0=ld(slots[24],i,j+0);__m256d v_Pz41_1=ld(slots[24],i,j+4);__m256d v_Po2_0=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M6_0,v_Pz32_0),v_Pz18_0),v_Pz19_0),v_Pz41_0);__m256d v_Po2_1=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(_mm256_add_pd(v_M6_1,v_Pz32_1),v_Pz18_1),v_Pz19_1),v_Pz41_1);Q0(const_cast<double*>(slots[34].x)+(size_t)i*slots[34].ld+j+0,v_Po2_0);Q0(const_cast<double*>(slots[34].x)+(size_t)i*slots[34].ld+j+4,v_Po2_1);__m256d v_Pz33_0=ld(slots[26],i,j+0);__m256d v_Pz33_1=ld(slots[26],i,j+4);__m256d v_M41_0=ld(slots[3],i,j+0);__m256d v_M41_1=ld(slots[3],i,j+4);__m256d v_Pz25_0=ld(slots[2],i,j+0);__m256d v_Pz25_1=ld(slots[2],i,j+4);__m256d v_Po3_0=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(_mm256_sub_pd(v_Pz33_0,v_M41_0),v_Pz25_0),v_Pz26_0),v_Pz19_0);__m256d v_Po3_1=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(_mm256_sub_pd(v_Pz33_1,v_M41_1),v_Pz25_1),v_Pz26_1),v_Pz19_1);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+0,v_Po3_0);Q0(const_cast<double*>(slots[3].x)+(size_t)i*slots[3].ld+j+4,v_Po3_1);__m256d v_M7_0=ld(slots[27],i,j+0);__m256d v_M7_1=ld(slots[27],i,j+4);__m256d v_M35_0=ld(slots[8],i,j+0);__m256d v_M35_1=ld(slots[8],i,j+4);__m256d v_Pz16_0=ld(slots[9],i,j+0);__m256d v_Pz16_1=ld(slots[9],i,j+4);__m256d v_Po11_0=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(v_M7_0,v_M35_0),v_Pz14_0),v_Pz16_0);__m256d v_Po11_1=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(v_M7_1,v_M35_1),v_Pz14_1),v_Pz16_1);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+0,v_Po11_0);Q0(const_cast<double*>(slots[8].x)+(size_t)i*slots[8].ld+j+4,v_Po11_1);__m256d v_M20_0=ld(slots[38],i,j+0);__m256d v_M20_1=ld(slots[38],i,j+4);__m256d v_Pz43_0=ld(slots[7],i,j+0);__m256d v_Pz43_1=ld(slots[7],i,j+4);__m256d v_Pz15_0=_mm256_add_pd(_mm256_sub_pd(v_M20_0,v_Pz43_0),v_Pz26_0);__m256d v_Pz15_1=_mm256_add_pd(_mm256_sub_pd(v_M20_1,v_Pz43_1),v_Pz26_1);__m256d v_M10_0=ld(slots[13],i,j+0);__m256d v_M10_1=ld(slots[13],i,j+4);__m256d v_M19_0=ld(slots[1],i,j+0);__m256d v_M19_1=ld(slots[1],i,j+4);__m256d v_Pz17_0=ld(slots[35],i,j+0);__m256d v_Pz17_1=ld(slots[35],i,j+4);__m256d v_Po9_0=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(_mm256_sub_pd(v_M10_0,v_M19_0),v_Pz32_0),v_Pz16_0),v_Pz15_0),v_Pz17_0);__m256d v_Po9_1=_mm256_add_pd(_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(_mm256_sub_pd(v_M10_1,v_M19_1),v_Pz32_1),v_Pz16_1),v_Pz15_1),v_Pz17_1);Q0(const_cast<double*>(slots[13].x)+(size_t)i*slots[13].ld+j+0,v_Po9_0);Q0(const_cast<double*>(slots[13].x)+(size_t)i*slots[13].ld+j+4,v_Po9_1);__m256d v_M5_0=ld(slots[12],i,j+0);__m256d v_M5_1=ld(slots[12],i,j+4);__m256d v_M17_0=ld(slots[42],i,j+0);__m256d v_M17_1=ld(slots[42],i,j+4);__m256d v_Po13_0=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M5_0,v_M17_0),v_Pz15_0),v_Pz17_0);__m256d v_Po13_1=_mm256_add_pd(_mm256_sub_pd(_mm256_add_pd(v_M5_1,v_M17_1),v_Pz15_1),v_Pz17_1);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+0,v_Po13_0);Q0(const_cast<double*>(slots[42].x)+(size_t)i*slots[42].ld+j+4,v_Po13_1);__m256d v_Pz31_0=ld(slots[28],i,j+0);__m256d v_Pz31_1=ld(slots[28],i,j+4);__m256d v_Po1_0=_mm256_add_pd(v_Pz31_0,v_Pz15_0);__m256d v_Po1_1=_mm256_add_pd(v_Pz31_1,v_Pz15_1);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+0,v_Po1_0);Q0(const_cast<double*>(slots[38].x)+(size_t)i*slots[38].ld+j+4,v_Po1_1);}_mm_sfence();pending_finish();_mm_sfence();}void matrix_multiply(int n,const double*a,const double*b,double*c){rank48_top(a,b,c);}

评测结果

Compilation

N/A

Compile OK

Score: N/A

显示更多

Compile OK

Testcase #1

1.922 s

462 MB + 340 KB

Accepted

Score: 100

显示更多

Time (ms): 1922.405696
Memory (KiB): 473428
Status: Run Finished

ok Accepted