This is an edited version of my Sunday Smiles essay.  

One of my many interests is physics, and I have long thought that the discipline of Physics is caught in a cul-de-sac and that a new paradigm is necessary to explain significant lacunae in our understanding of how seemingly contradictory data fit together. 

The Dark Energy/Dark Matter problem is one of the largest issues. Our observations of the universe and understanding of how it should work suggest that the vast majority of the mass in the universe is not observable. We keep looking for some evidence of Dark Matter to explain why the universe isn't behaving as it should, and never seem to find it. 

The most obvious example of this is that the rotation of galaxies is inconsistent with the amount of mass we observe. If the observable mass of a galaxy were all there were, galaxies would rotate very differently than they do. A simplified explanation I just found explains this if you want to spend a few minutes to get the idea:

The rate of expansion of the universe is similarly mysterious--gravity is not doing what our model of the universe says it should be. So physicists have postulated the existencee of Dark Energy. 

Dark Energy/Dark Matter are the proposed solutions to the essentially mysterious behavior of facts that we observe but cannot explain. But they are no solution at all. They are just an "X" factor to make the equations work. We have no idea what Dark Matter is; we simply infer it from how the universe works as we observe it. 

An alternate explanation for these facts is that we still don't know how the universe works at galactic and larger scales, and our models are wrong or incomplete. Our physical models work quite well at scales we have direct experience with, but not at different scales. 

You can think of this as similar to the ability of Newtonian mechanics to work at the human scale and our normal rates of movement, but we need Relativity to explain things when relative speeds are high and Quantum Mechanics to explain the behavior of matter when things get very small.

The way academic science works in the modern world makes exploring the possibility that our models in Physics are wrong or incomplete extremely difficult for a variety of reasons. We have lots more scientists, and invest orders of magnitude more money in exploring the way things work, yet progress is very modest. 

 One of the scientists I follow explains why academic science appears to be in a rut:

What Sabine Hossenfelder describes here is a very real phenomenon, and we have a real-world example of this fact during the COVID-19 pandemic. The "science" and the research done about and during the pandemic was entirely political, not driven by curiosity about what was happening. In fact, almost no studies were done at all regarding all the non-pharmaceutical interventions despite there being tremendous controversy. Fauci funded no randomized controlled trials on any of the big issues.  

Modern science is, much more than at any time in history, a highly collaborative process driven by bureaucracies, funding structures, institutions, multi-billion dollar projects, and frankly tyrannical bureaucrats who are often mediocre scientists but excellent players in the political games that now dominate academia. 

You must not be afraid of your uniqueness and you must care less and less what people think of you.

You have no responsibility to live up to what other people think you ought to accomplish.

I have no responsibility to be like they expect me to be.

— Prof. Richard Feynman (@FeynmanBot) June 14, 2024

Curiosity--genuine gee-whiz curiosity--gets you nowhere in academic science. Playing the game does. Science is all about following the money. (No, this doesn't mean there aren't good scientists or good science being done, just that any correlation between what is rewarded and good science is almost accidental in most cases). 

Before World War II, scientists tended to be lone wolves, and the best scientists could break new ground precisely because they didn't fit well into a highly structured environment. Einstein did his most groundbreaking work outside of academia--he worked in a Patent Office!--and the scientists we tend to revere were extraordinary lone wolves who tended to be outcasts until proven to be geniuses. If you read up on the geniuses behind Quantum Mechanics, most of them were prickly, although they did talk with each other and compete like mad. 

It is imperative in science to doubt; it is absolutely necessary, for progress in science, to have uncertainty as a fundamental part of your inner nature.

To make progress in understanding, we must remain modest and allow that we do not know.

— Prof. Richard Feynman (@FeynmanBot) June 14, 2024

I am in no position to know if Sabine could have been a groundbreaking genius or just a decent scientist, but I can say that her description of academia as a place where you succeed by being a timid worker bee or a highly competent political bureaucrat is absolutely accurate. It was not always thus, but the current model of how most science is done almosts dictates that this be so. 

Grants drive employment in science, and people who dispense grants direct money to programs that appeal to them--with which they tend to agree. The bigger the grants one gets, the more powerful one becomes, and you get big grants by being mainstream enough. You can push boundaries a bit, but not too much. It is the rarest of things to find somebody in a bureaucracy who dispenses money to people he thinks are even a little nuts. 

Great scientists often look nuts until they look like geniuses. 

"Publish or perish" incentives also tend to encourage junk science, especially in psychology and biology. I have written before about the horrible state of nutrition science, and Vinay Prasad did a great video on why it is basically junk. 

It's not just biological sciences where "publish or perish" produces horrible incentives. After all, you must go through peer review, which means pleasing your peers. Telling your peers they are wrong is also unpopular. 

During the pandemic, you got to see in real-time how the research funders can corrupt it. Dr. Fauci essentially coerced scientists to produce the shoddy and even fraudulent "Proximal Origins" paper on the virus, in which scientists published a paper that their private correspondence shows they did not believe to be correct. 

They lied, in other words, because Fauci controlled their research budgets, and in return for lying, millions of dollars flowed to their labs. 

The incentives are all wrong, and as a result, innovation and productivity in science have slowed to a crawl. I wrote about this last year and it is worth your time to go back and take a look. 

Let me quote from a study on disruptive science:

Despite exponential growth in recent decades of research papers and patents, a new University of Minnesota study published in Nature suggests science and technology are becoming less disruptive.

Carlson School of Management Associate Professor Russell Funk, doctoral student Michael Park and Professor Erin Leahey of the University of Arizona analyzed data from 45 million papers and 3.9 million patents across six decades for their research. They used a “disruptiveness score,” which is based on the patterns of citations five years after publication, to assess the extent to which papers and patents push ideas toward new trajectories. They determined:

• Papers and patents are less likely to be disruptive, or make previous findings obsolete and push science and technology in a new direction, such as the discovery of the DNA double helix structure.
• Instead, papers and patents are more likely to be consolidating, or further developing previous work — e.g., the Kohn-Sham equation which improved upon existing equations about electron particles.
• Scientists and inventors are increasingly using narrower slices of knowledge to develop their new work.
• This pattern holds across all major fields of science, including technology, medicine and social sciences.

Given the grant-driven nature of science and its increasing dependence on collaborative work, this shouldn't surprise us. If you are part of a large team, you can't have everybody going off in different directions, and if you need to get money from senior scientists in charge of grant proposals, you can't be trying to prove them to be fools. 

Given where science is today, it can't be conducted by people like me who have very eclectic interests and far too little patience; you need people who can delve deeply and have extraordinary patience. On the other hand, the current model is no longer working. We are spending extraordinary amounts of money to make relatively little progress or even going down rabbit holes that people with money want others to explore. 

One of my "bucket list" desires is to live to see the revolution in physics that I suspect is coming. It's impossible to know when it will come, but I fear that the institutional imperatives of academic science dictate that it will take longer than it ought to. 

I guess that means I had better live for a long time! Biologists, get on that!