u/colski

This post investigates whether K4 matches the speculative idea that HALF of K4 is transposed plaintext.

The proposed transposition would be induced by the K3 palimpsest idea; and the two halves would be the text written in the alternate rows that Sanborn indicated with P (plaintext) and C (ciphertext).

https://preview.redd.it/y6o2khh2yf2h1.jpg?width=400&format=pjpg&auto=webp&s=acba002815e9fbc9101e0acbf66a46735102cd86

Let's start with typical frequencies for 48 English letters. I'm going to make one conceit, I'm going to swap "E"s and "K"s in the plaintext:

{'K': 6, 'T': 4, 'A': 4, 'O': 4, 'I': 4, 'N': 3, 'S': 3, 'R': 3, 'H': 2, 'L': 2, 'D': 2, 'C': 2, 'U': 1, 'M': 1, 'F': 1, 'G': 1, 'P': 1, 'W': 1, 'Y': 1, 'B': 1, 'V': 1, 'E': 0, 'J': 0, 'X': 0, 'Z': 0, 'Q': 0}

K4 frequencies after subtracting that:

{'U': 5, 'Z': 4, 'W': 4, 'Q': 4, 'B': 4, 'S': 3, 'J': 3, 'G': 3, 'F': 3, 'K': 2, 'E': 2, 'X': 2, 'T': 2, 'P': 2, 'L': 2, 'V': 1, 'R': 1, 'O': 1, 'D': 1, 'Y': 0, 'N': 0, 'M': 0, 'I': 0, 'H': 0, 'C': 0, 'A': 0}

Comparing these, it's plausible. Apart from Es (which would have to be changed to Ks) there are enough English-frequency letters in K4 for HALF of them to be transposed English and the other half to be transposed and substituted English.

So the complete story would be:

1. insert 192 nulls into the K3 plaintext to form 8x66 matrix.
2. overwrite 98 nulls. on rows 1,3,5,7 write K4 plaintext (swap Ks and Es). on rows 2,4,6,8 write K4 ciphertext (eg rot13). one ? probably at the end.
3. apply the K3 transposition (rotate, resize 24x22, rotate).
4. remove the nulls. we have K3?K4.

The puzzle then rests on understanding the order of overwriting nulls in step 2. For example, if writing vertically like the archimedes palimpsest, every other letter would be plaintext. If writing horizontally, there would be entire words of plaintext.

The pattern 011010 suggested by dYAhRo aligns uniquely with the nulls in 40 positions (5 per row). Ed or Jim suggested to Elonka it means "1-2-3". Are these the nulls we're supposed to fill? 3x40=120. when asked whether there are other "off by one" errors like XLAYERTWO, Ed said "well jim how are we going to answer that one?" and Jim replied "we don't". Ed said his secret is to "push into the square" (from the context, referencing this matrix) and "change the alphabet base" (the mask). Another comment was "sanborn seemed puzzled we hadn't recovered the original matrix and put it through all the shifts"

Speculation upon speculation, it's all we have. How could anyone possibly guess this method from the clues PALIMPSEST and LUCID MEMORY and dYAhRo and LAYERTWO ?

Okay, strap yourselves in. I'm going to show you something very palimpsesty. It's not a solution, just a demonstration of an idea.

S*LO*WL*YDhE*SP*AR*AThL*YS*LO*WLoY*TH*ER*EMoA*IN*SO*FPvA*SS*AG*EDo
E*BR*IS*THtA*TE*NC*UMtB*ER*ED*THpE*LO*WE*RPhA*RT*OF*THnE*DO*OR*WAt
YW*A*SR*EMdOV*E*DW*ITeHT*R*EM*BLtIN*G*HA*NDeSI*M*AD*EAiTI*N*YB*REs
AC*H*IN*THeEU*P*PE*RLlEF*T*HA*NDaCO*R*NE*RAlND*T*HE*NWyID*E*NI*NGa
TH*EH*O*LEsAL*IT*T*LEbII*NS*E*RThED*TH*E*CAbND*LE*A*NDlPE*ER*E*DIw
NT*HE*H*OTuAI*RE*S*CAiPI*NG*F*ROtMT*HE*C*HAiMB*ER*C*AUlSE*DT*H*EFt
LA*ME*TO*FyLI*CK*ER*BsUT*PR*ES*EsNT*LY*DE*TsAI*LS*OF*TaHE*RO*OM*Wi
IT*HI*NE*MeER*GE*DF*RsOM*TH*EM*IwST*XC*AN*YiOU*SE*EA*NtYT*HI*NG*Q?
          ^          ^          ^          ^          ^          ^

Hopefully you can (almost) recognise this matrix as "the original matrix", K3 plaintext written as a 8 by 42 matrix. However, you will notice that a number of NULLS (asterisks) have been inserted. That means that this matrix is 8x66. If you look even closer, you will also notice that a second message (K2 plaintext) appears in lowercase, written upwards (the columns are indicated). This has been designed to resemble the Archimedes palimpsest. The second plaintext is sort of pushed in between the first. I chose columns that don't interfere with the K3 plaintext.

If we take THIS matrix, rotate it clockwise, and reshape as 24x22 then it looks like this:

ILNTAYESTATHCW********
BLHMHEHAROIEEH********
ISIWNTHONRSLEO********
OLTETYMFTEHMHDeyusedth
ELAAEOAERIILUV********
TSGCRIPEEPEKET********
PDNADESTEWCRFR********
CLRIUARBAELTMTssibleth
OUPIEHBLMTIIFT********
EYTPNNTRRSHRGS********
HEELEEFEAMDOMS********
RRNBTWIEOTDLHLwsthatpo
SNMECIEYTTTDON********
LTXLHTRGOHCYEH********
NHWEADCEEAERNE********
HCRNREYTAAADPMisibleho
OAMNNSAAUIBDDI********
RISLELTMTNESRE********
HAOSEOCAEDFOAF********
ANNETFNTUDWAHPtallyinv
YHSPITEATEEEDI********
DSHRDEENOSIOTR********
NYOANOHEIBRGGM********
EDNRWEQWFIGEAD?itwasto

Hopefully you also (almost) recognise this matrix as the "in between matrix", starting ILNTAYES... But here it has 8 added columns, and the K2 plaintext is clearly legible and - amazingly - it's cleanly separated from the K3 plaintext.

ENDYAHROHNLSRHEOCPTEOIBIDYSHNAI
ACHTNREYULDSLLSLLNOHSNOSMRWXMNE
TPRNGATIHNRARPESLNNELEBLPIIACAE
WMTWNDITEENRAHCTENEUDRETNHAEOET
FOLSEDTIWENHAEIOYTEYQHEENCTAYCR
EIFTBRSPAMHHEWENATAMATEGYEERLBT
EEFOASFIOTUETUAEOTOARMAEERTNRTI
BSEDDNIAAHTTMSTEWPIEROAGRIEWFEB
AECTDDHILCEIHSITEGOEAOSDDRYDLOR
ITRKLMLEHAGTDHARDPNEOHMGFMFEUHE
ECDMRIPFEIMEHNLSSTTRTVDOHW?***t
***i***w***s***e***i***a***s***
s***s***y***t***l***i***t***i**
*u***w***l***b***h***b***s***a*
**y***l***a***l***e***s***i***e
***t***e***d***t***n***h***p***
t***t***o***v***o***o***h***h**

Finally, this is what that matrix looks like when rotated clockwise. Everything up to the questionmark is exactly as seen on the copperplate (except the ragged edge).

After removing the nulls (asterisks), the "K4 ciphertext" at the end is just a transposed version of the K2 plaintext (read every 6th letter).

Now, there are only 48 letters of K4 here, including the question mark, so we're 50 letters short of demonstrating how K4 could have been encoded. Ed Scheidt said:

>"another math problem is transposition, transposition is again the name implies, you're transposing something. So instead of a direct correlation that you can visualize, transposition is a little harder to visualize. It's sort of like looking at a puzzle and you're defining the parameters of the puzzle in the sense of the square, and then now you have the square and you're going to transpose the characters or the letters that are in the square to something that's a secret. So my secret is push into this square. So that was another step. And then the last step which has been good for 30 years, which I didn't know at the time it would be, but I masked the framework, in other words if you can change the language base then it becomes in my favor and not your favor of trying to break it. It becomes more of a challenge now, when it was used as the mask it was current, 2020 secret."

So my suggestion is: this is what Ed meant by "pushing into the square". He's talking about a palimpsest: writing both plaintexts in the same space.

The pattern of the "original matrix" at the top here, in terms of plaintext and nulls, is (10110110110). What I would wish it to be: (10010), because this is the pattern indicated by "dYAhR". Perhaps there's another way that I don't see to introduce this kind of "palimpsest". For example, adding 16 columns would work.

> Ed Scheidt: So substitution was one I had to use, another math problem is transposition, transposition is again the name implies, you're transposing
something. So instead of a direct correlation that you can visualize, transposition is a little harder to visualize. It's sort of like looking at a puzzle and you're defining the parameters of the puzzle in the sense of the square, and then now you have the square and you're going to transpose the characters or the letters that are in the square to something that's a secret. So my secret is push into this square. So that was another step. And then the last step which has been good for 30 years, which I didn't know at the time it would be, but I masked the framework, in other words if you can change the language base then it becomes in my favor and not your favor of trying to break it. It becomes more of a challenge now, when it was used as the mask it was current, 2020 secret.

I've already talked a bit about how the square is the same thing as Sanborn's "original matrix"; and I think "pushing in" is a reading order directive (aka "washing machine"). That's the K4 puzzle: how to extract this running key from K3. I think there are clues.

In the last step, the mask, Scheidt changed the language base. This post is a suggestion for what that could mean.

Give the string BERLIN, an everyday way you could change the language base is by replacing each letter with its phonetic alphabet equivalent:

BRAVO ECHO ROMEO LIMA INDIA NOVEMBER

The problem is, this makes the string much, much longer. And, the frequency of the letters will still match English.

I can also stack it like this:

BERLINCLOCK
RCOINOHISHI
AHMMDVAMCAL
VOEAIERAARO
O O AML RL
     BI  I
     EE  E
     R

Still the same codes, but now the phonetic alphabet appears in columns. We generated a bunch of extra codewords like RCOINOHISHI AHMMDVAMCAL, VOEAIERAARO and so on. If you note, those are alphabet substitutions of the original plaintext (the nth letter of the phonetic alphabet keyword).

One last step. I'm going to fill all the spaces with K (which is reminiscent of the Morse code) and I'm going to shift each successive row to the right in a staircase pattern.

BERLINCLOCK
KRCOINOHISH
KKAHMMDVAMC
KKKVOEAIERA
KKKKOKOKAML
KKKKKKKKKKB

Now I have used the phonetic alphabet to create 8 keystreams, which I can combine together using the vigenere table (addition in columns modulo 26). The information of each letter is spread across the following 4-8 symbols of ciphertext.

I've changed the language base in a way that has masked the original plaintext. I guarantee you will have 26 distinct letters in your ciphertext, and very random distribution. Even repeated short words will be corrupted.

But, now the problem is: how to reverse this process? For me this is the best part. There is only one way to reverse this: slowly, very desperately slowly. The weakness is that the first letter B is uncorrupted, and if you subtract the keyword BRAVO from the first five letters, then the second letter is revealed to be E, with keyword ECHO starting at the second position. And so on to the end.

Pencil and paper. Changed the letter base. Very hard, minimal effort. Is this what Scheidt has been talking about?

I'm not sure. This doesn't generate kryptossy letters. But it does generate random ciphertext distributions. If you used an alphabet order like ETAOIN as your plaintext alphabet? In that case, you would pad with E not K.

WHISKEY..
.TANGO...
..FOXTROT

THFQGQYYBKCRWKELJEGKU
RGIMMIIDSMUIHXSIGUCWO
UKEPKADSDUMZIKFLHGTHE
QULSJAUDOGCHLANHAOEY

context: it's a puzzle I made just for you

V sbyybjrq gur ehyrf

edit: I changed the formatting just to make it more readable.