u/oatmealcraving

With locality sensitive hashing better to forget the h word.

You get thousands of little geometry sensors that 50:50 tell you on which side of a random hyperplane your input data is.

Despite being 50:50 each bit is quite informative.

Several bits together narrow down which geometric region the input is in.

If the inputs have regularities, bits can be codependent while still showing 50:50 on off behavior. That's kind of a subtle point but a gift from random projections - locality sensitive hashing.

Starting from there you can build LSH context dependent neural networks where parameter selection (information routing) is determined using LSH bits.

I've a bunch of notes of say preliminary draft quality.

Maybe start with this one:

https://archive.org/details/atlas-lsh-neural-networks-hierarchical-geometry-rather-than-hierarchical-features

And then click on 'uploaded by" for more should you wish to.

You could have a neural network structured from matrices (A or B) and (C or D) and maybe using binarized random projection of the input to make the decisions to get one of CA,CB,DA,DB. That would make full use of all the parameters in the matrices. DCBA on its own would just affine collapse to a single matrix and waste parameters.

Taking the idea a little further:

https://archive.org/details/atlas-lsh-neural-networks-an-intuitive-overview

You can click on uploaded by for some further things.

There have been a few attempts at creating a new "The Radio Board" forum since the original one passed into the next life with its owner.

I don't know if there are any rules about who gets to try or what the story is.

Anyway I'm interested in continuing the saga, I don't know how this approach will work for people: https://padlet.com/seanhaddps/the-radio-board-hwqnn4bdb7gakotq

A bizarre linear system for neural networks applications:

https://archive.org/details/the-oddness-of-hd

One softwarez is:

https://archive.org/details/hd-dh-hd-dh-graph-viewer which we looked at before.

Another softwarez is: https://archive.org/details/h-12-d-dynamics

but that is for the atypical H₁₂ system, but you can still see the oscillations in some configurations. If you switch configurations you can see transitory oscillations as well. That is not due to dissipative effect, it is due to energy only slowly draining out of the old oscillation modes into the new one.

I keep getting banned on physics and neural network forums about this, where it might more properly be discussed. They really are incapable of absorbing information from "orthogonal" channels.

Rather than use say ReLU to prevent affine collapse, how about this:

https://archive.org/details/information-preserving-space-warping-to-prevent-affine-collapse-in-neural-networks

https://archive.org/details/global-modulation-versus-local-activation-functions-in-neural-systems

Example (maybe more later): https://archive.org/details/swnet-pn

ReLU decisions are only used to zero channels, what if we allow interactions between the decisions?

https://discourse.numenta.org/t/richer-than-relu/12364/1