u/Severe-Heat-518

Here's the code: https://godbolt.org/z/cf6c1Mvd8

The problem: The innermost dimension (hidden_size.z) is completely contiguous and vectorizes cleanly. However, because input_cell depends directly on the contents of the input buffer, it acts as a pseudo-random offset.

Every time visible_x steps forward, the memory pointer leaps across a massive stride in the dictionary buffer. The hardware prefetcher completely gives up, and every single step evicts the previous cache line out of L1.

While I prefer stick to an SoA design for vectorization, I have total flexibility on how that data is structured globally:

• Dictionary Transposition: I can completely rearrange the macro-dimension ordering of the dictionary array [hidden_col, in_field_idx, input_cell, hidden_size.z]. If swapping the position of input_cell or tiling it helps keep data "warm" in L1, I can change the layout. Just don't move the hidden_size.z because then I'd probably lose the vectorization, at least with the current algorithm.
• Memory Alignment: I can change the memory alignment (#[repr(align(64))]) of the buffers or pad them to perfectly match CPU cache lines.
• Loop Structuring / Tiling: I am entirely open to structural loop rewrites (e.g., loop blocking/tiling, using chunk iterators instead of dynamic slicing inside the loop, or altering the execution order of hidden_col vs visible_x/y).

Can anyone help me with this?