I thought it was just me … but then I talked to a bunch of mathematicians
A year or two ago when I was working on the core compiler functionality for Fault, I got into this weird habit of programming in my sleep. I don’t mean I sleepwalked over to my computer and started cranking out API endpoints (although apparently some people also do this). I mean I would wrestle with a complex problem all day, only to give up, go to bed and wake up with the answer. Within five minutes that bug that had been driving me crazy for hours was fixed, or I had a strategy for organizing my abstractions, or a set of new ideas to test. While my consciousness was quiet and empty, my subconsciousness continued to work on the problem.
Some people describe it as dreaming a solution. That’s not the way it works for me. I don’t dream about it. I don’t have any particular recollection of even thinking about the problem. I just wake up and a solution is sitting in my mind, all queued up and ready to go.
This phenomena is so common it actually has a name. It’s called incubation and it can also be triggered by long walks, or just distracting your conscious mind with something else so that your subconscious is left alone to mull over the problem in depth. There are a lot of theories as to the how and why it works, most rely on the notion that our conscious mind can actually be a mean little shit, distracting us with anxieties and frustrations around the fact that the problem exists and keeping us from focusing on the problem itself. When that part of the brain turns off, or gets distracted by something else, the rest of the brain is free to play openly with the problem.
The other theory about why we solve problems in our sleep comes from what we believe our brains are actually doing during sleep in the first place. Sleep is when our brains reorganize our memory, finding new patterns and building up our metadata around our experiences. Some of this problem solving may actually be secondary, a consequence of a new pattern forming that immediately makes the problem look different.