August 17, 2025
Hello, Neural Network 1.5! Float vs Double.
Last time I added a brief note about choosing float over double, where I said that float is better - because we’ll work mostly with small numbers, so the precision is not an issue, and because GPUs prefer floats (at least, mass-consumer GPUs). Also, logic says that float should be faster - addition and subtraction should take the same amount of cycles, but memory throughput should be better with float, since more data can fit into a single cache line.
Read more