Yeah, I said transformer because that seems to be the state of the art in AI architectures, but purpose built neural network hardware might not actually benefit from the same architecture.

A neural network made from analog hardware could theoretically replace a significant portion of an LLM’s processing and not be limited by things like floating point precision or clock speeds. Who needs floating point when you can literally just multiply voltages together with a couple transistor junctions at the speed of light?