Scientific progress has slowed considerably. In chemistry the progress slowed to a crawl by 1770, then Lavoisier published and it took off. In physics the Newtonian progress slowed to nothing then quantum theory spurred on a massive advance. There are likely similar examples from the other scientific fields. 

Progress has once again slowed to a crawl. It began crawling in the 1990's. There are no new theories (I mean real theories, not a hypothesis and guesses dressed up as a new theory). And even the theories we have, on which we depend utterly for interpreting observations, don't really work all that well. Periodic theory (the theories behind the periodicity of the elements) is weak at best, but no progress has been made since the 1950's when the atom smashers added the new elements. Molecular orbital theory is used to explain everything about molecules, yet it can't predict anything (literally, anything) about a molecule without someone nudging it along in the right direction as the calculation is set up. 

There is just no doubt that scientific progress has slowed. 

The big picture: There is an ongoing debate about why the rate of scientific progress appears to be slowing down despite an increase in the number of scientists, amount of funding for their work, and the quantity of papers they publish.

• Some researchers believe the cause is fundamental: The low-hanging fruits of discovery have simply already been plucked, meaning scientists have to work harder and more money needs to be invested to get what remains.
• Others say as knowledge accumulates in ever greater amounts, researchers face a higher burden to learn about a field.
• But in science, new ideas can be recombined with old ones and begin to multiply, which makes it difficult to tease out the drivers of stagnation, says Dashun Wang, who directs the Center for Science of Science and Innovation at Northwestern University's Kellogg School of Management.

Axios news, October 14, 2021

Last October Chu and Evans published (PNAS October 12, 2021 118 (41) e2021636118; https://doi.org/10.1073/pnas.2021636118 https://www.pnas.org/content/118/41/e2021636118 a huge analysis of 1.8 billion citations made in 90 million papers to see who was reading and citing what publications.

The size of scientific fields may impede the rise of new ideas. Examining 1.8 billion citations among 90 million papers across 241 subjects, we find a deluge of papers does not lead to turnover of central ideas in a field, but rather to ossification of canon. Scholars in fields where many papers are published annually face difficulty getting published, read, and cited unless their work references already widely cited articles. New papers containing potentially important contributions cannot garner field-wide attention through gradual processes of diffusion. These findings suggest fundamental progress may be stymied if quantitative growth of scientific endeavors—in number of scientists, institutes, and papers—is not balanced by structures fostering disruptive scholarship and focusing attention on novel ideas.

Chu & Evans, PNAS, 118, 2021

Here's the problem: with everyone publishing so much, and all those papers needing to be reviewed, scientists have just gotten used to using old theories to explain all observations, even when the old theories don't fit all that well. When everyone has the same explanation, what point is there in proposing a new one, one which can't be easily defended?

It's a problem the academics have created, a direct consequence of "publish-or-perish." To gain tenure in a research university young faculty need to produce publications with merit. So they work hard to publish using accepted explanations, even if the papers they publish are dull as dishwater. If the journal is sufficiently prestigious it counts toward tenure. But it is the young researchers who have the best chance at advancing a new theory, or interpreting an old theory more meaningfully, but they won't risk tenure for it.

So we produce piles and piles of dreck. Papers that aren't read by anyone not involved in publishing that paper. I skip by entire journals when I look through the titles and see nothing of note. Mind you, I get suckered into a bad article because the title was designed to make it look like it was one thing, when it was really just boring crap.

Here are some other takeaways, compiled in the Axios article:

What they found: A new analysis suggests another possible source of stagnation: A deluge of new publications is leading to an "ossification of canon," Johan Chu of Northwestern University and James Evans of the University of Chicago write.

• Chu and Evans argue scientists are "cognitively overloaded" by the flood of papers to read, review and cite, so they look for how the work in front of them relates to well-established research. And the researchers predict if ideas arrive too fast in publications, they will compete for attention and wash each other out.

They found a massive inequality in how papers in those fields are cited, they report in the Proceedings of the National Academies of Sciences.

• For example, when the field of electrical and electronic engineering published about 10,000 papers each year, the top 1% most-cited papers got almost 9% of total citations and the bottom 50% least-cited papers got 44% of citations. When the field grew to produce 100,000 papers per year, the top 1% received almost 17% of citations whereas the bottom 50% got about 20%. That large inequality wasn't seen in smaller fields.
• And "the most-cited papers maintain their number of citations year over year when fields are large, while all other papers' citation counts decay," they write. That crystallizes the canon.
• They also found that in smaller fields, papers slowly rise into the top bracket, whereas in larger fields, the papers that do make it into the canon get there fast. "It is predetermined socially rather than scholarly," says Chu, who led the work.
• In larger fields, most papers build on others — without disrupting the canon — but in smaller disciplines, papers are more likely to be disruptive.

The bottom line: As a scientist, "you get a lot of reward for doing something better. You get an immediate paper and a lot of appreciation," says Konrad Kording, a professor of neuroscience at the University of Pennsylvania.

• There is a deep addiction among scientists to publish paper after paper in order to be funded and advance their careers, he says.
• "You don't get a reward or appreciation for doing something different."

Axios news

More to come...

ADDED 20 February 2024: Roger Pielke Jr. posted today an update to a post he made in 2020 about how the original old science just keeps dominating thought, even when it's now obvious the original was wrong or now pointless. He specifically addresses a climate model that was the extreme end of reality when proposed, within month invalidated, and which has been disclaimed by the IPCC. Yet it remains the most published model every month since it was proposed.