Greater specialization makes innovation much harder

From Innovation Shortfall, Low Hanging Fruit, and the Frontiers of Knowledge via a link in Adapt*, by Tim Harford:
"Big takeaway: “The shorter the period that innovators spend innovating, the less their output as individuals over their lifetime. If innovation is central to technological progress, then forces that reduce the length of active innovative careers will reduce the rate of technological progress. This effect will be particularly strong if innovators do their best work when they are young.”

These facts ought to give us pause. Jones postulates that the main reason we see an age shift is because the frontiers of knowledge are farther and farther away. The shoulders of giants are higher. The low fruit has been taken. Because of how complex and specialized our knowledge has become, Jones says the young must spend more and more years training and acquiring knowledge to reach the cutting edge.

Now it may be that the “burden of knowledge” is heavier than it used to be. But my own take is that we shouldn’t confuse the accumulation of credential crud for the difficulty of reaching the frontiers of knowledge. On the macro scale, we often cite the Mancur Olson thesis that special interests sap the dynamism of economic growth over time. I believe the Jones data and Cowen’s arguments present further evidence supporting the Olson story. For instance, why should we believe PhD programs are designed to efficiently move students from ignorance to the frontier of knowledge as quickly as possible? Academics are not mainly truth-seekers. They’re incentivized to pursue other aims such as pleasing their advisors, their tenure review panel, or a grant making body. These aims do not coincide with discovering new and useful knowledge. Secondly, we can go back further. A lot of education from K-12 to college core requirements is a waste of time. How much of this knowledge is relevant to future work making discoveries? If anything, Jones should suggest we try to make education more efficient.

But I digress. In all this discussion, Cardwell’s Law looms in the background. No country stays technologically superior forever. As yesterday’s innovators becomes today’s vested interests, stagnation follows. There are reasons for that. All of these arguments–Thiel’s, Cowen’s, Jones’s–suggest we may need a thousand nations sooner than we think. We will soon find not the singularity, but the stagnation is near."