It is easy to forget that our behavior is governed almost entirely by the unconscious:
That we humans do much of what we do without following explicit rules is no more mysterious than my cat hunting without knowing rules about hunting or a tree growing without knowing rules about forming leaves. We only think it's mysterious if we think explicitness is the norm, but explicitness is a rare thing, restricted to humans, and used only now and again because it is often more efficient to allow causal, neural connections in the brain and body to execute an action with little (or, indeed, no) conscious calculation.
And it generally would not do you any good even if you made that knowledge explicit:
[Tacit knowledge] is knowledge stored in the muscles, nerve pathways, and synaptic connections. In principle, if not in practice, science could describe all of it. We still wouldn't be able to use it to guide our actions in a self-conscious way because we aren't built for that.
And collective knowledge is perhaps the thing that separates us most from non-life:
The one real mystery left lies in collective tacit knowledge. This is mysterious because we can't describe it and we don't know in detail how we acquire it. It is mysterious because we can only "borrow it": it is not our property but is social and collective. Take language. What constitutes our constantly changing natural language is not up to any individual, it is a matter of where the collective of language speakers takes it. [...]
Just as I think there could never be a fully automated editor of my books, I also think the limits to intelligent machines and automation will lie in a much better understanding of tacit knowledge - and especially of collective tacit knowledge. Our human interaction and social life (rather than the mere possession of the bodies once thought necessary for computers/robots to become intelligent) may provide a fundamental limit to the indefinite extension of machine intelligence.
