Second peak bang on

At the dawn of the 21st century, we were pretty sure we had solved cosmology. The Lambda Cold Dark Matter (LCDM) model made strong predictions for the power spectrum of the Cosmic Microwave Background (CMB). One was that the flat Robertson-Walker geometry that we were assuming for LCDM predicted the location of the first peak should be at ℓ = 220. As I discuss in the history of the rehabilitation of Lambda, this was a genuinely novel prediction that was clearly confirmed first by BOOMERanG and subsequently by many other experiments, especially WMAP. As such, it was widely (and rightly) celebrated among cosmologists. The WMAP team has been awarded major prizes, including the Gruber cosmology prize and the Breakthrough prize.

As I discussed in the previous post, the location of the first peak was not relevant to the problem I had become interested in: distinguishing whether dark matter existed or not. Instead, it was the amplitude of the second peak of the acoustic power spectrum relative to the first that promised a clear distinction between LCDM and the no-CDM ansatz inspired by MOND. This was also first tested by BOOMERanG:

postboomer — The CMB power spectrum observed by BOOMERanG in 2000. The first peak is located exactly where LCDM predicted it to be. The second peak was not detected, but was clearly smaller than expected in LCDM. It was consistent with the prediction of no-CDM.

In a nutshell, LCDM predicted a big second peak while no-CDM predicted a small second peak. Quantitatively, the amplitude ratio A_1:2 was predicted to be in the range 1.54 – 1.83 for LCDM, and 2.22 – 2.57 for no-CDM. Note that A_1:2 is smaller for LCDM because the second peak is relatively big compared to the first.

BOOMERanG confirmed the major predictions of both competing theories. The location of the first peak was exactly where it was expected to be for a flat Roberston-Walker geometry. The amplitude of the second peak was that expected in no-CDM. One can have the best of both worlds by building a model with high Lambda and no CDM, but I don’t take that too seriously: Lambda is just a place holder for our ignorance – in either theory.

I had made this prediction in the hopes that cosmologists would experience the same crisis of faith that I had when MOND appeared in my data. Now it was the data that they valued that was misbehaving – in precisely the way I had predicted with a model that was motivated by MOND (albeit not MOND itself). Surely they would see reason?

There is a story that Diogenes once wandered the streets of Athens with a lamp in broad daylight in search of an honest man. I can relate. Exactly one member of the CMB community wrote to me to say “Gee, I was wrong to dismiss you.” [I paraphrase only a little.] When I had the opportunity to point out to them that I had made this prediction, the most common reaction was “no you didn’t.” Exactly one of the people with whom I had this conversation actually bothered to look up the published paper, and that person also wrote to say “Gee, I guess you did.” Everyone else simply ignored it.

The sociology gets worse from here. There developed a counter-narrative that the BOOMERang data were wrong, therefore my prediction fitting it was wrong. No one asked me about it; I learned of it in a chance conversation a couple of year later in which it was asserted as common knowledge that “the data changed on you.” Let’s examine this statement.

The BOOMERanG data were early, so you expect data to improve. At the time, I noted that the second peak “is only marginally suggested by the data so far”, so I said that “as data accumulate, the second peak should become clear.” It did.

The predicted range quoted above is rather generous. It encompassed the full variation allowed by Big Bang Nucleosynthesis (BBN) at the time (1998/1999). I intentionally considered the broadest range of parameters that were plausible to be fair to both theories. However, developments in BBN were by then disfavoring low-end baryon densities, so the real expectation for the predicted range was narrower. Excluding implausibly low baryon densities, the predicted ranges were 1.6 – 1.83 for LCDM and 2.36 – 2.4 for no-CDM. Note that the prediction of no-CDM is considerably more precise than that of LCDM. This happens because all the plausible models run together in the absence of the forcing term provided by CDM. For hypothesis testing, this is great: the ratio has to be this one value, and only this value.

A few years later, WMAP provided a much more accurate measurement of the peak locations and amplitudes. WMAP measured A_1:2 = 2.34 ± 0.09. This is bang on the no-CDM prediction of 2.4.

peaks_predict_wmap — Peak locations measured by WMAP in 2003 (points) compared to the a priori (1999) predictions of LCDM (red tone lines) and no-CDM (blue tone lines).

The prediction for the amplitude ratio A_1:2 that I made over twenty years ago remains correct in the most recent CMB data. The same model did not successfully predict the third peak, but I didn’t necessarily expect it to: the no-CDM ansatz (which is just General Relativity without cold dark matter) had to fail at some point. But that gets ahead of the story: no-CDM made a very precise prediction for the second peak. LCDM did not.

LCDM only survives because people were willing to disregard existing bounds – in this case, on the baryon density. It was easier to abandon the most accurately measured and the only over-constrained pillar of Big Bang cosmology than acknowledge a successful prediction that respected all those things. For a few years, the attitude was “BBN was close, but not quite right.” In time, what appears to be confirmation bias kicked in, and the measured abundances of the light elements migrated towards the “right” value – as specified by CMB fits.

LCDM does give an excellent fit to the power spectrum of the CMB. However, only the location of the first peak was predicted correctly in advance. Everything subsequent to that (at higher ℓ) is the result of a multi-parameter fit with sufficient flexibility to accommodate any physically plausible power spectrum. However, there is no guarantee that the parameters of the fit will agree with independent data. For a long while they did, but now we see the emergence of tensions in not only the baryon density, but also the amplitude of the power spectrum, and most famously, the value of the Hubble constant. Perhaps this is the level of accuracy that is necessary to begin to perceive genuine anomalies. Beyond the need to invoke invisible entities in the first place.

I could say a lot more, and perhaps will in future. For now, I’d just like to emphasize that I made a very precise, completely novel prediction for the amplitude of the second peak. That prediction came true. No one else did that. Heck of a coincidence, if there’s nothing to it.

A pre-history of the prediction of the amplitude of the second peak of the cosmic microwave background

In the previous post, I wrote about a candidate parent relativistic theory for MOND that could fit the acoustic power spectrum of the cosmic microwave background (CMB). That has been a long time coming, and probably is not the end of the road. There is a long and largely neglected history behind this, so let’s rewind a bit.

I became concerned about the viability of the dark matter paradigm in the mid-1990s. Up until that point, I was a True Believer, as much as anyone. Clearly, there had to be dark matter, specifically some kind of non-baryonic cold dark matter (CDM), and almost certainly a WIMP. Alternatives like MACHOs (numerous brown dwarfs) were obviously wrong (Big Bang Nucleosynthesis [BBN] taught us that there are not enough baryons), so microlensing experiments searching for them would make great variable star catalogs but had no chance of detecting dark matter. In short, I epitomized the impatient attitude against non-WIMP alternatives that persists throughout much of the community to this day.

It thus came as an enormous surprise that the only theory to successfully predict – in advance – our observations of low surface brightness galaxies was MOND. Dark matter, as we understood it at the time, predicted nothing of the sort. This made me angry.

How could it be so?

To a scientist, a surprising result is a sign to think again. Maybe we do not understand this thing we thought we understood. Is it merely a puzzle – some mistake in our understanding or implementation of our preferred theory? Or is it a genuine anomaly – an irrecoverable failure? How is it that a completely different theory can successfully predict something that my preferred theory did not?

In this period, I worked very hard to make things work out for CDM. It had to be so! Yet every time I thought I had found a solution, I realized that I had imposed an assumption that guaranteed the desired result. I created and rejected tautology after tautology. This process unintentionally foretold the next couple of decades of work in galaxy formation theory: I’ve watched others pursue the same failed ideas and false leads over and over and over again.

After months of pounding my head against the proverbial wall, I realized that if I was going to remain objective, I shouldn’t just be working on dark matter. I should also try just to see how things worked in MOND. Suddenly I found myself working much less hard. The things that made no sense in terms of dark matter tumbled straight out of MOND.

This concerned me gravely. Could we really be so wrong about something so important? I went around giving talks, expressing the problems about which I was concerned, and asking how it could be that MOND got so many novel predictions correct in advance if there was nothing to it.

Reactions varied. The first time I mentioned it in a brief talk at the Institute of Astronomy in Cambridge, friend and fellow postdoc Adi Nusser became visibly agitated. He bolted outside as soon as I was done, and I found him shortly later with a cigarette turned mostly to ash as if in one long draw. I asked him what he thought and he replied the he was “NOT HAPPY!” Neither was I. It made no sense.

I first spoke at length on the subject in a colloquium at the Department of Terrestrial Magnetism, where Vera Rubin worked, along with other astronomers and planetary scientists. I was concerned about how Vera would react, so I was exceedingly thorough, spending most of the time on the dark matter side of the issue. She reacted extremely well, as did the rest of the audience, many telling me it was the best talk they had heard in five years. (I have heard this many times since; apparently 5 years is some sort of default for a long time that is short of forever.)

Several months later, I gave the same talk at the University of Pennsylvania to an audience of mostly particle physicists and early-universe cosmologists. A rather different reaction ensued. One person shouted “WHAT HAVE YOU DONE WRONG!” It wasn’t a question.

These polar opposite reactions from different scientific audiences made me realize that sociology was playing a role. As I continued to give the talk to other groups, the pattern above repeated, with the reception being more positive the further an audience was from cosmology.

I started asking people what would concern them about the paradigm. What would falsify CDM? Sometimes this brought bemused answers, like that of Tad Pryor: “CDM has been falsified many times.” (This was in 1997, at which time CDM meant standard SCDM which was indeed pretty thoroughly falsified at that point: we were on the cusp of the transition to LCDM.) More often it met with befuddlement: “Why would you even ask that?” It was disappointing how often this was the answer, as a physical theory is only considered properly scientific if it is falsifiable. [All of the people who had this reaction agreed to that much: I often specifically asked.] The only thing that was clear was that most cosmologists couldn’t care less what galaxies did. Galaxies were small, non-linear entities, they argued… to the point that, as Martin Rees put it, “we shouldn’t be surprised at anything they do.”

I found this attitude to be less than satisfactory. However, I could see its origin. I only became aware of MOND because it reared its ugly head in my data. I had come face to face with the beast, and it shook my mostly deeply held scientific beliefs. Lacking this experience, it must have seemed to them like the proverbial boy crying wolf.

So, I started to ask cosmologists what would concern them. Again, most gave no answer; it was simply inconceivable to them that something could be fundamentally amiss. Among those who did answer, the most common refrain was “Well, if the CMB did something weird.” They never specified what they meant by this, so I set out to quantify what would be weird.

This was 1998. At that time, we knew the CMB existed (the original detection in the 1960s earning Penzias and Wilson a Nobel prize) and that there were temperature fluctuations on large scales at the level of one part in 100,000 (the long-overdue detection of said fluctuations by the COBE satellite earning Mathers and Smoot another Nobel prize). Other experiments were beginning to detect the fluctuations on finer angular scales, but nothing definitive was yet known about the locations and amplitudes of the peaks that were expected in the power spectrum. However, the data were improving rapidly, and an experiment called BOOMERanG was circulating around the polar vortex of Antartica. Daniel Eisenstein told me in a chance meeting that “The data are in the can.”

This made the issue of quantifying what was weird a pressing one. The best prediction is one that comes before the fact, totally blind to the data. But what was weird?

At the time, there was no flavor of relativistic MOND yet in existence. But we know that MOND is indistinguishable from Newton in the limit of high accelerations, and whatever theory contains MOND in the appropriate limit must also contain General Relativity. So perhaps the accelerations in the early universe when the CMB occurred were high enough that MOND effects did not yet occur. This isn’t necessarily the case, but making this ansatz was the only way to proceed at that time. Then it was just General Relativity with or without dark matter. That’s what was weird: no dark matter. So what difference did that make?

Using the then-standard code CMBFAST, I computed predictions for the power spectrum for two families models: LCDM and no-CDM. The parameters of LCDM were already well known at that time. There was even an imitation of the Great Debate about it between Turner and Peebles, though it was more consensus than debate. This enabled a proper prediction of what the power spectrum should be.

Most of the interest in cosmology then concerned the geometry of the universe. We had convinced ourselves that we had to bring back Lambda, but this made a strong prediction for the location of the first peak – a prediction that was confirmed by BOOMERanG in mid-2000.

The geometry on which most cosmologists were focused was neither here nor there to the problem I had set myself. I had no idea what the geometry of a MOND universe might be, and no way to predict the locations of the peaks in the power spectrum. I had to look for relative differences, and these proved not to be all that weird. The difference between LCDM and no-CDM was, in fact, rather subtle.

The main difference I found between models with and without dark matter was a difference in the amplitude of the second peak relative to the first. As I described last time, baryons act to damp the oscillations, while dark matter acts to drive them. Take away the dark matter and there is only damping, resulting in the second peak getting dragged down. The primary variable controlling the ratio of the first-to-second peak amplitude was the baryon fraction. Without dark matter, the baryon fraction is 1. In LCDM, it was then thought to be in the range 0.05 – 0.15. (The modern value is 0.16.)

This is the prediction I published in 1999:

the red lines in the left plot represent LCDM, the blue lines in the right plot no-CDM. The data that were available at the time I wrote the paper are plotted as the lengthy error bars. The location of the first peak had sorta been localized, but nothing was yet known about the amplitude of the second. Here was a clear, genuinely a priori prediction: for a given amplitude of the first peak, the amplitude of the second was smaller without CDM than with it.

Quantitatively, the ratio of the amplitude of the first to second peak was predicted to be in the range 1.54 – 1.83 for LCDM. This range represents the full range of plausible LCDM parameters as we knew them at the time, which as I noted above, we thought we knew very well. For the case of no-CDM, the predicted range was 2.22 – 2.57. In both cases, the range of variation was dominated by the uncertainty in the baryon density from BBN. While this allowed for a little play, the two hypotheses should be easily distinguishable, since the largest ratio possible in LCDM was clearly less than the smallest possible in no-CDM.

And that is as far as I am willing to write today. This is already a long post, so we’ll return to the results of this test in the future.

A Significant Theoretical Advance

The missing mass problem has been with us many decades now. Going on a century if you start counting from the work of Oort and Zwicky in the 1930s. Not quite a half a century if we date it from the 1970s when most of the relevant scientific community started to take it seriously. Either way, that’s a very long time for a major problem to go unsolved in physics. The quantum revolution that overturned our classical view of physics was lightning fast in comparison – see the discussion of Bohr’s theory in the foundation of quantum mechanics in David Merritt’s new book.

To this day, despite tremendous efforts, we have yet to obtain a confirmed laboratory detection of a viable dark matter particle – or even a hint of persuasive evidence for the physics beyond the Standard Model of Particle Physics (e.g., supersymmetry) that would be required to enable the existence of such particles. We cannot credibly claim (as many of my colleagues insist they can) to know that such invisible mass exists. All we really know is that there is a discrepancy between what we see and what we get: the universe and the galaxies within it cannot be explained by General Relativity and the known stable of Standard Model particles.

If we assume that General Relativity is both correct and sufficient to explain the universe, which seems like a very excellent assumption, then we are indeed obliged to invoke non-baryonic dark matter. The amount of astronomical evidence that points in this direction is overwhelming. That is how we got to where we are today: once we make the obvious, imminently well-motivated assumption, then we are forced along a path in which we become convinced of the reality of the dark matter, not merely as a hypothetical convenience to cosmological calculations, but as an essential part of physical reality.

I think that the assumption that General Relativity is correct is indeed an excellent one. It has repeatedly passed many experimental and observational tests too numerous to elaborate here. However, I have come to doubt the assumption that it suffices to explain the universe. The only data that test it on scales where the missing mass problem arises is the data from which we infer the existence of dark matter. Which we do by assuming that General Relativity holds. The opportunity for circular reasoning is apparent – and frequently indulged.

It should not come as a shock that General Relativity might not be completely sufficient as a theory in all circumstances. This is exactly the motivation for and the working presumption of quantum theories of gravity. That nothing to do with cosmology will be affected along the road to quantum gravity is just another assumption.

I expect that some of my colleagues will struggle to wrap their heads around what I just wrote. I sure did. It was the hardest thing I ever did in science to accept that I might be wrong to be so sure it had to be dark matter – because I was sure it was. As sure of it as any of the folks who remain sure of it now. So imagine my shock when we obtained data that made no sense in terms of dark matter, but had been predicted in advance by a completely different theory, MOND.

When comparing dark matter and MOND, one must weigh all evidence in the balance. Much of the evidence is gratuitously ambiguous, so the conclusion to which one comes depends on how one weighs the more definitive lines of evidence. Some of this points very clearly to MOND, while other evidence prefers non-baryonic dark matter. One of the most important lines of evidence in favor of dark matter is the acoustic power spectrum of the cosmic microwave background (CMB) – the pattern of minute temperature fluctuations in the relic radiation field imprinted on the sky a few hundred thousand years after the Big Bang.

The equations that govern the acoustic power spectrum require General Relativity, but thankfully the small amplitude of the temperature variations permits them to be solved in the limit of linear perturbation theory. So posed, they can be written as a damped and driven oscillator. The power spectrum favors features corresponding to standing waves at the epoch of recombination when the universe transitioned rather abruptly from an opaque plasma to a transparent neutral gas. The edge of a cloud provides an analog: light inside the cloud scatters off the water molecules and doesn’t get very far: the cloud is opaque. Any light that makes it to the edge of the cloud meets no further resistance, and is free to travel to our eyes – which is how we perceive the edge of the cloud. The CMB is the expansion-redshifted edge of the plasma cloud of the early universe.

An easy way to think about a damped and a driven oscillator is a kid being pushed on a swing. The parent pushing the child is a driver of the oscillation. Any resistance – like the child dragging his feet – damps the oscillation. Normal matter (baryons) damps the oscillations – it acts as a net drag force on the photon fluid whose oscillations we observe. If there is nothing going on but General Relativity plus normal baryons, we should see a purely damped pattern of oscillations in which each peak is smaller than the one before it, as seen in the solid line here:

CMB_Pl_CLonly — The CMB acoustic power spectrum predicted by General Relativity with no cold dark matter (line) and as observed by the Planck satellite (data points).

As one can see, the case of no Cold Dark Matter (CDM) does well to explain the amplitudes of the first two peaks. Indeed, it was the only hypothesis to successfully predict this aspect of the data in advance of its observation. The small amplitude of the second peak came as a great surprise from the perspective of LCDM. However, without CDM, there is only baryonic damping. Each peak should have a progressively lower amplitude. This is not observed. Instead, the third peak is almost the same amplitude as the second, and clearly higher than expected in the pure damping scenario of no-CDM.

CDM provides a net driving force in the oscillation equations. It acts like the parent pushing the kid. Even though the kid drags his feet, the parent keeps pushing, and the amplitude of the oscillation is maintained. For the third peak at any rate. The baryons are an intransigent child and keep dragging their feet; eventually they win and the power spectrum damps away on progressively finer angular scales (large 𝓁 in the plot).

As I wrote in this review, the excess amplitude of the third peak over the no-CDM prediction is the best evidence to my mind in favor of the existence of non-baryonic CDM. Indeed, this observation is routinely cited by many cosmologists to absolutely require dark matter. It is argued that the observed power spectrum is impossible without it. The corollary is that any problem the dark matter picture encounters is a mere puzzle. It cannot be an anomaly because the CMB tells us that CDM has to exist.

Impossible is a high standard. I hope the reader can see the flaw in this line of reasoning. It is the same as above. In order to compute the oscillation power spectrum, we have assumed General Relativity. While not replacing it, the persistent predictive successes of a theory like MOND implies the existence of a more general theory. We do not know that such a theory cannot explain the CMB until we develop said theory and work out its predictions.

That said, it is a tall order. One needs a theory that provides a significant driving term without a large amount of excess invisible mass. Something has to push the swing in a universe full of stuff that only drags its feet. That does seem nigh on impossible. Or so I thought until I heard a talk by Pedro Ferreira where he showed how the scalar field in TeVeS – the relativistic MONDian theory proposed by Bekenstein – might play the same role as CDM. However, he and his collaborators soon showed that the desired effect was indeed impossible, at least in TeVeS: one could not simultaneously fit the third peak and the data preceding the first. This was nevertheless an important theoretical development, as it showed how it was possible, at least in principle, to affect the peak ratios without massive amounts of non-baryonic CDM.

At this juncture, there are two options. One is to seek a theory that might work, and develop it to the point where it can be tested. This is a lot of hard work that is bound to lead one down many blind alleys without promise of ultimate success. The much easier option is to assume that it cannot be done. This is the option adopted by most cosmologists, who have spent the last 15 years arguing that the CMB power spectrum requires the existence of CDM. Some even seem to consider it to be a detection thereof, in which case we might wonder why we bother with all those expensive underground experiments to detect the stuff.

Rather fewer people have invested in the approach that requires hard work. There are a few brave souls who have tried it; these include Constantinos Skordis and Tom Złosnik. Very recently, the have shown a version of a relativistic MOND theory (which they call RelMOND) that does fit the CMB power spectrum. Here is the plot from their paper:

Note that black line in their plot is the fit of the LCDM model to the Planck power spectrum data. Their theory does the same thing, so it necessarily fits the data as well. Indeed, a good fit appears to follow for a range of parameters. This is important, because it implies that little or no fine-tuning is needed: this is just what happens. That is arguably better than the case for LCDM, in which the fit is very fine-tuned. Indeed, that was a large point of making the measurement, as it requires a very specific set of parameters in order to work. It also leads to tensions with independent measurements of the Hubble constant, the baryon density, and the amplitude of the matter power spectrum at low redshift.

As with any good science result, this one raises a host of questions. It will take time to explore these. But this in itself is a momentous result. Irrespective if RelMOND is the right theory or, like TeVeS, just a step on a longer path, it shows that the impossible is in fact possible. The argument that I have heard repeated by cosmologists ad nauseam like a rosary prayer, that dark matter is the only conceivable way to explain the CMB power spectrum, is simply WRONG.

A Philosophical Approach to MOND

A Philosophical Approach to MOND is a new book by David Merritt. This is a major development in the both the science of cosmology and astrophysics, on the one hand, and the philosophy and history of science on the other. It should be required reading for anyone interested in any of these topics.

For many years, David Merritt was a professor of astrophysics who specialized in gravitational dynamics, leading a number of breakthroughs in the effects of supermassive black holes in galaxies on the orbits of stars around them. He has since transitioned to the philosophy of science. This may not sound like a great leap, but it is: these are different scholarly fields, each with their own traditions, culture, and required background education. Changing fields like this is a bit like switching boats mid-stream: even a strong swimmer may flounder in the attempt given the many boulders academic disciplines traditionally place in the stream of knowledge to mark their territory. Merritt has managed the feat with remarkable grace, devouring the background reading and coming up to speed in a different discipline to the point of a lucid fluency.

For the most part, practicing scientists have little interaction with philosophers and historians of science. Worse, we tend to have little patience for them. The baseline presumption of many physical scientists is that we know what we’re doing; there is nothing the philosophers can teach us. In the daily practice of what Kuhn called normal science, this is close to true. When instead we are faced with potential paradigm shifts, the philosophy of science is critical, and the absence of training in it on the part of many scientists becomes glaring.

In my experience, most scientists seem to have heard of Popper and Kuhn. If that. Physical scientists will almost always pay lip service to Popper’s ideal of falsifiablity, and that’s pretty much the extent of it. Living up to applying that ideal is another matter. If an idea that is near and dear to their hearts and careers is under threat, the knee-jerk response is more commonly “let’s not get carried away!”

There is more to the philosophy of science than that. The philosophers of science have invested lots of effort in considering both how science works in practice (e.g., Kuhn) and how it should work (Popper, Lakatos, …) The practice and the ideal of science are not always the same thing.

The debate about dark matter and MOND hinges on the philosophy of science in a profound way. I do not think it is possible to make real progress out of our current intellectual morass without a deep examination of what science is and what it should be.

Merritt takes us through the methodology of scientific research programs, spelling out what we’ve learned from past experience (the history of science) and from careful consideration of how science should work (its philosophical basis). For example, all scientists agree that it is important for a scientific theory to have predictive power. But we are disturbingly fuzzy on what that means. I frequently hear my colleagues say things like “my theory predicts that” in reference to some observation, when in fact no such prediction was made in advance. What they usually mean is that it fits well with the theory. This is sometimes true – they could have predicted the observation in advance if they had considered that particular case. But sometimes it is retroactive fitting more than prediction – consistency, perhaps, but it could have gone a number of other ways equally well. Worse, it is sometimes a post facto assertion that is simply false: not only was the prediction not made in advance, but the observation was genuinely surprising at the time it was made. Only in retrospect is it “correctly” “predicted.”

The philosophers have considered these situations. One thing I appreciate is Merritt’s review of the various takes philosophers have on what counts as a prediction. I wish I had known these things when I wrote the recent review in which I took a very restrictive definition to avoid the foible above. The philosophers provide better definitions, of which more than one can be usefully applicable. I’m not going to go through them here: you should read Merritt’s book, and those of the philosophers he cites.

From this philosophical basis, Merritt makes a systematic, dare I say, scientific, analysis of the basic tenets of MOND and MONDian theories, and how they fare with regard to their predictions and observational tests. Along the way, he also considers the same material in the light of the dark matter paradigm. Of comparable import to confirmed predictions are surprising observations: if a new theory predicts that the sun will rise in the morning, that isn’t either new or surprising. If instead a theory expects one thing but another is observed, that is surprising, and it counts against that theory even if it can be adjusted to accommodate the new fact. I have seen this happen over and over with dark matter: surprising observations (e.g., the absence of cusps in dark matter halos, the small numbers of dwarf galaxies, downsizing in which big galaxies appear to form earliest) are at first ignored, doubted, debated, then partially explained with some mental gymnastics until it is Known and of course, we knew it all along. Merritt explicitly points out examples of this creeping determinism, in which scientists come to believe they predicted something they merely rationalized post-facto (hence the preeminence of genuinely a priori predictions that can’t be fudged).

Merritt’s book is also replete with examples of scientists failing to take alternatives seriously. This is natural: we have invested an enormous amount of time developing physical science to the point we have now reached; there is an enormous amount of background material that cannot simply be ignored or discarded. All too often, we are confronted with crackpot ideas that do exactly this. This makes us reluctant to consider ideas that sound crazy on first blush, and most of us will rightly display considerable irritation when asked to do so. For reasons both valid and not, MOND skirts this bondary. I certainly didn’t take it seriously myself, nor really considered it at all, until its predictions came true in my own data. It was so far below my radar that at first I did not even recognize that this is what had happened. But I did know I was surprised; what I was seeing did not make sense in terms of dark matter. So, from this perspective, I can see why other scientists are quick to dismiss it. I did so myself, initially. I was wrong to do so, and so are they.

A common failure mode is to ignore MOND entirely: despite dozens of confirmed predictions, it simply remains off the radar for many scientists. They seem never to have given it a chance, so they simply don’t pay attention when it gets something right. This is pure ignorance, which is not a strong foundation from which to render a scientific judgement.

Another common reaction is to acknowledge then dismiss. Merritt provides many examples where eminent scientists do exactly this with a construction like: “MOND correctly predicted X but…” where X is a single item, as if this is the only thing that [they are aware that] it does. Put this way, it is easy to dismiss – a common refrain I hear is “MOND fits rotation curves but nothing else.” This is a long-debunked falsehood that is asserted and repeated until it achieves the status of common knowledge within the echo chamber of scientists who refuse to think outside the dark matter box.

This is where the philosophy of science is crucial to finding our way forward. Merritt’s book illuminates how this is done. If you are reading these words, you owe it to yourself to read his book.

The Hubble Constant from the Baryonic Tully-Fisher Relation

The distance scale is fundamental to cosmology. How big is the universe? is pretty much the first question we ask when we look at the Big Picture.

The primary yardstick we use to describe the scale of the universe is Hubble’s constant: the H₀ in

v = H₀ D

that relates the recession velocity (redshift) of a galaxy to its distance. More generally, this is the current expansion rate of the universe. Pick up any book on cosmology and you will find a lengthy disquisition on the importance of this fundamental parameter that encapsulates the size, age, critical density, and potential fate of the cosmos. It is the first of the Big Two numbers in cosmology that expresses the still-amazing fact that the entire universe is expanding.

Quantifying the distance scale is hard. Throughout my career, I have avoided working on it. There are quite enough, er, personalities on the case already.

No need for me to add to the madness.

Not that I couldn’t. The Tully-Fisher relation has long been used as a distance indicator. It played an important role in breaking the stranglehold that H₀ = 50 km/s/Mpc had on the minds of cosmologists, including myself. Tully & Fisher (1977) found that it was approximately 80 km/s/Mpc. Their method continues to provide strong constraints to this day: Kourkchi et al. find H₀ = 76.0 ± 1.1(stat) ± 2.3(sys) km s^-1 Mpc^-1. So I’ve been happy to stay out of it.

Until now.

I am motivated in part by the calibration opportunity provided by gas rich galaxies, in part by the fact that tension in independent approaches to constrain the Hubble constant only seems to be getting worse, and in part by a recent conference experience. (Remember when we traveled?) Less than a year ago, I was at a cosmology conference in which I heard an all-too-typical talk that asserted that the Planck H₀ = 67.4 ± 0.5 km/s/Mpc had to be correct and everybody who got something different was a stupid-head. I’ve seen this movie before. It is the same community (often the very same people) who once insisted that H₀ had to be 50, dammit. They’re every bit as overconfident as before, suffering just as much from confirmation bias (LCDM! LCDM! LCDM!), and seem every bit as likely to be correct this time around.

So, is it true? We have the data, we’ve just refrained from using it in this particular way because other people were on the case. Let’s check.

The big hassle here is not measuring H₀ so much as quantifying the uncertainties. That’s the part that’s really hard. So all credit goes to Jim Schombert, who rolled up his proverbial sleeves and did all the hard work. Federico Lelli and I mostly just played the mother-of-all-jerks referees (I’ve had plenty of role models) by asking about every annoying detail. To make a very long story short, none of the items under our control matter at a level we care about, each making < 1 km/s/Mpc difference to the final answer.

In principle, the Baryonic Tully-Fisher relation (BTFR) helps over the usual luminosity-based version by including the gas, which extends application of the relation to lower mass galaxies that can be quite gas rich. Ignoring this component results in a mess that can only be avoided by restricting attention to bright galaxies. But including it introduces an extra parameter. One has to adopt a stellar mass-to-light ratio to put the stars and the gas on the same footing. I always figured that would make things worse – and for a long time, it did. That is no longer the case. So long as we treat the calibration sample that defines the BTFR and the sample used to measure the Hubble constant self-consistently, plausible choices for the mass-to-light ratio return the same answer for H₀. It’s all relative – the calibration changes with different choices, but the application to more distant galaxies changes in the same way. Same for the treatment of molecular gas and metallicity. It all comes out in the wash. Our relative distance scale is very precise. Putting an absolute number on it simply requires a lot of calibrating galaxies with accurate, independently measured distances.

Here is the absolute calibration of the BTFR that we obtain:

btf_cep_trgb — The Baryonic Tully-Fisher relation calibrated with 50 galaxies with direct distance determinations from either the Tip of the Red Giant Branch method (23) or Cepheids (27).

In constructing this calibrated BTFR, we have relied on distance measurements made or compiled by the Extragalactic Distance Database, which represents the cumulative efforts of Tully and many others to map out the local universe in great detail. We have also benefited from the work of Ponomareva et al, which provides new calibrator galaxies not already in our SPARC sample. Critically, they also measure the flat velocity from rotation curves, which is a huge improvement in accuracy over the more readily available linewidths commonly employed in Tully-Fisher work, but is expensive to obtain so remains the primary observational limitation on this procedure.

Still, we’re in pretty good shape. We now have 50 galaxies with well measured distances as well as the necessary ingredients to construct the BTFR: extended, resolved rotation curves, HI fluxes to measure the gas mass, and Spitzer near-IR data to estimate the stellar mass. This is a huge sample for which to have all of these data simultaneously. Measuring distances to individual galaxies remains challenging and time-consuming hard work that has been done by others. We are not about to second-guess their results, but we can note that they are sensible and remarkably consistent.

There are two primary methods by which the distances we use have been measured. One is Cepheids – the same type of variable stars that Hubble used to measure the distance to spiral nebulae to demonstrate their extragalactic nature. The other is the tip of the red giant branch (TRGB) method, which takes advantage of the brightest red giants having nearly the same luminosity. The sample is split nearly 50/50: there are 27 galaxies with a Cepheid distance measurement, and 23 with the TRGB. The two methods (different colored points in the figure) give the same calibration, within the errors, as do the two samples (circles vs. diamonds). There have been plenty of mistakes in the distance scale historically, so this consistency is important. There are many places where things could go wrong: differences between ourselves and Ponomareva, differences between Cepheids and the TRGB as distance indicators, mistakes in the application of either method to individual galaxies… so many opportunities to go wrong, and yet everything is consistent.

Having followed the distance scale problem my entire career, I cannot express how deeply impressive it is that all these different measurements paint a consistent picture. This is a credit to a large community of astronomers who have worked diligently on this problem for what seems like aeons. There is a temptation to dismiss distance scale work as having been wrong in the past, so it can be again. Of course that is true, but it is also true that matters have improved considerably. Forty years ago, it was not surprising when a distance indicator turned out to be wrong, and distances changed by a factor of two. That stopped twenty years ago, thanks in large part to the Hubble Space Telescope, a key goal of which had been to nail down the distance scale. That mission seems largely to have been accomplished, with small differences persisting only at the level that one expects from experimental error. One cannot, for example, make a change to the Cepheid calibration without creating a tension with the TRGB data, or vice-versa: both have to change in concert by the same amount in the same direction. That is unlikely to the point of wishful thinking.

Having nailed down the absolute calibration of the BTFR for galaxies with well-measured distances, we can apply it to other galaxies for which we know the redshift but not the distance. There are nearly 100 suitable galaxies available in the SPARC database. Consistency between them and the calibrator galaxies requires

H₀ = 75.1 +/- 2.3 (stat) +/- 1.5 (sys) km/s/Mpc.

This is consistent with the result for the standard luminosity-linewidth version of the Tully-Fisher relation reported by Kourkchi et al. Note also that our statistical (random/experimental) error is larger, but our systematic error is smaller. That’s because we have a much smaller number of galaxies. The method is, in principle, more precise (mostly because rotation curves are more accurate than linewidhts), so there is still a lot to be gained by collecting more data.

Our measurement is also consistent with many other “local” measurements of the distance scale,

but not with “global” measurements. See the nice discussion by Telescoper and the paper from which it comes. A Hubble constant in the 70s is the answer that we’ve consistently gotten for the past 20 years by a wide variety of distinct methods, including direct measurements that are not dependent on lower rungs of the distance ladder, like gravitational lensing and megamasers. These are repeatable experiments. In contrast, as I’ve pointed out before, it is the “global” CMB-fitted value of the Hubble parameter that has steadily diverged from the concordance region that originally established LCDM.

So, where does this leave us? In the past, it was easy to dismiss a tension of this sort as due to some systematic error, because that happened all the time – in the 20th century. That’s not so true anymore. It looks to me like the tension is real.

Tracing the baryons in star forming galaxies

Galaxies are big. Our own Milky Way contains about fifty billion solar masses of stars, and another ten billion of interstellar gas, roughly speaking. The average star is maybe half a solar mass, so crudely speaking, that’s one hundred billion stars. Give or take. For comparison, the population of the Earth has not quite reached eight billion humans. So if you gave each one of us our own personal starship, in order to visit every star in the Galaxy, each one of us would have to visit a dozen stars. Give or take. I’m getting old, so I call dibs on Proxima Centauri through Procyon.

Figure 1 shows a picture of NGC 628, a relatively nearby spiral galaxy. What you see here is mostly stars, along with some interstellar dust and ionized gas. In addition to those components, there are also stellar remnants left behind by dead stars (mostly white dwarfs, some neutron stars, and the occasional black hole). In the space between the stars resides colder forms of interstellar gas, including both atomic gas (individual atoms in space) and molecular gas (the cold, dense material from which new stars form). How much is there of each component?

ngc628_final — **Fig 1.** The spiral galaxy NGC 628. The continuum light of stars in contrasted by dark dust lanes and highlighted by pink pinpoints of Balmer line emission. These are regions of interstellar gas illuminated by the UV emission of short-lived, massive O stars. Not visible here is the interstellar atomic and molecular gas from which stars form.

The bulk of the normal mass (excluding dark matter) in big spiral galaxies like the Milky Way is stars and their remnants. But there is also diffuse material in the vast interstellar medium – the ample space between the stars. This includes dust and several distinct phases of gas – molecular, atomic, and ionized (plasma). The dust and plasma are easy to see, but don’t add up to much – a mere millions of solar masses each for the whole Milky Way. The atomic and molecular gas add up to a lot more, but cannot be seen optically.

Atomic gas can be traced by 21 cm emission from the spin-flip transition of atomic hydrogen using radio telescopes. This is commonly referred to with the spectroscopic notation “HI”. The HI mass – the mass of atomic hydrogen – is usually the second largest mass component in spirals, after stars. In dwarf galaxies, the atomic gas often outweighs the stars (Fig. 2).

MgMst — **Fig 2.** Gas mass vs. stellar mass for galaxies in the SPARC database (blue) and an independent sample selected from SDSS (grey) by Bradford. The line is the line of equality where gas mass and stellar mass are equal. The red point is the Milky Way. Like other bright spirals, it is more stars than gas. Among lower mass dwarf galaxies, the reverse is commonly true: those in the field have more gas than stars.

Stars and atomic (HI) gas are the big two. When it comes to star forming galaxies, more massive spirals are usually star dominated while less massive dwarfs are usually dominated by atomic gas. But what about molecular gas?

Molecular gas is important to the star formation process. It is the densest (a very relative term!) material in the interstellar medium, the place where cold gas can condense into the nuggets that sometimes form stars. How much of this necessary ingredient is there?

The bulk of the mass of molecular gas is in molecular hydrogen, H₂. Spectroscopically, H₂ is a really boring molecule. It has no transitions in wavelength regimes that are readily accessible to observation. So, unlike atomic hydrogen, which brazenly announces its presence throughout the universe via the 21 cm line, molecular hydrogen is nigh-on invisible.

So we use proxies. The most commonly employed proxy for tracing molecular gas mass is carbon monoxide. CO is one of many molecules that accompany the much more abundance molecular hydrogen, and CO produces emission features that are more readily accessible observationally in the mm wavelength range. That has made it the tracer of choice.

CO is far from an ideal tracer of mass. Carbon and oxygen are both trace elements compared to hydrogen, so the correspondence between CO emission and molecular gas mass depends on the relative abundance of both. If that sounds dodgy, it gets worse. It also depends on the interstellar radiation field, the opacity thereto (molecular gas is inevitably associated with dense clumps of dust that shield it from the ambient radiation), and the spatial overlap of the two components – CO and H₂ thrive in similar but not identical regions of space. Hence, converting the observed intensity of CO into a molecular hydrogen mass is a highly sensitive procedure that we typically bypass by assuming it is a universal constant.

It’s astronomy. We do what we can.

People have obsessed long and hard about the CO-to-H2 conversion, so we do have a reasonable idea what it is. While many debates can be had over the details, we have a decent idea of what the molecular gas mass is in some galaxies, at least to a first approximation. Molecular gas is usually outweighed by atomic gas, but sometimes it is comparable. So we’d like to keep track of it for the mass budget.

LCOMHIMst — **Fig 3.** The mass in molecular hydrogen gas as a function of atomic hydrogen (left) and stellar mass (right) from xGASS. The dotted line is the line of equality; molecular gas is usually outweighed by both atomic gas and stars. The red line at right is where the molecular gas mass is 7% of the stellar mass.

Obtaining CO observations is expensive, and often impossible: there are a lot of star forming galaxies where it simply isn’t detected. So we presume there is molecular gas there – that’s where the stars form, but we can’t always see it. So it would be handy to have another proxy besides CO.

Atomic gas is a lousy proxy for molecular gas. The mass of one hardly correlates with the other (Fig. 3). The two phases coexist in a complex and ever-changing variable quasi-equilibrium, with the amount of each at any given moment subject to change so that a snapshot of many galaxies provides a big mess.

Fortunately, the molecular gas mass correlates better with other properties – notably star formation. This makes sense, because stars form from molecular gas. So in some appropriately averaged sense, one follows the other. Star formation can be traced in a variety of ways, like the Balmer emission in Fig. 1. We can see the stars forming and infer the amount of molecular gas required to fuel that star formation even if we can’t detect the gas directly (Fig. 4).

MH2SFRMst — **Fig 4.** The current star formation rate (left) and molecular gas mass (right) as a function of stellar mass. The grey and black points are from xGASS, with the black points being those where the current star formation rate is within a factor of two of the past average (i.e., the stellar mass divided by the age of the universe). Blue points are low surface brightness galaxies. These extend the relation at left to much lower mass, but are generally not detected in CO. The molecular gas at right (open squares) is inferred by the amount required to sustain the observed star formation.

I’ve done a lot of work on low surface brightness galaxies, a class of objects that have proven particularly difficult to detect in CO. They have low dust contents, low oxygen abundances, relatively hard interstellar radiation fields – all factors that mitigate against CO. Yet we do see them forming stars, sometimes just one O star at a time, and we know how much molecular gas it takes to do that. So we can use star formation as a proxy for molecular gas mass. This is probably no worse than using CO, and perhaps even better – or would be, if we didn’t have to rely on CO to calibrate it in the first place.

Accurate tracers of star formation are also somewhat expensive to obtain. There are situations in which we need an estimate for the molecular gas mass where we don’t have either CO or a measurement of the star formation rate. So… we need a proxy for the proxy. Fortunately, that is provided by the stellar mass.

The stellar mass of a star-forming galaxy correlates with both its molecular gas mass and its star formation rate (Figs. 3 and 4). This is not surprising. It takes molecules to form stars, and it takes star formation to build up stellar mass. Indeed, the stellar mass is the time-integral of the star formation rate, so a correlation between the two (as seen in the left panel of Fig. 4) is mathematically guaranteed.

This brings us to the seven percent solution. Going through all the calibration steps, the molecular gas mass is, on average, about 7% of the stellar mass (the red lines in Figs. 3 and 4). The uncertainties in this are considerable. I’ve tried to work this out previously, and typically came up with numbers in the 5 – 10% range. So it seems to be in there somewhere.

This is adequate for some purposes, but not for others. One thing I want it for is to keep track of the total mass budget of baryons in galaxies so that we can calibrate the Baryonic Tully-Fisher relation. For this purpose it is adequate since molecular gas ranks behind both stars and atomic gas in the mass budget of almost every rotating galaxy. If it is 5% or 10% instead of 7%, this is a difference of a few percent of something that is itself typically < 10% of the total, and often less. A few percent of a few percent is a good working definition of negligible – especially in astronomy.

On top of all that, we also have to keep track of the stuff that isn’t hydrogen – helium and everything else in the periodic table, which astronomers often refer to collectively as “metals.” This makes for all sorts of partially-deserved jokes – oxygen isn’t a metal! but it is number 3 in cosmic abundance after hydrogen and helium. Like many anachronisms, the practice has good historical precedent. Early efforts to measure the abundances of the chemical elements in stars first gave results for iron. As other elements were probed, their abundances followed a pattern that scaled pretty well with the abundance of iron relative to hydrogen. So once again we have a proxy – this time, the iron abundance being a stand-in for that of everything else. Hence the persistence of the terminology – the metallicity of a star is a shorthand for the fraction of its mass that is not hydrogen and helium.

And that fraction is small. We usually write the mass fractions of hydrogen, helium, and everything else (metals) as

X + Y + Z = 1

where X is the fraction of mass in hydrogen, Y that in helium, and Z is everything else. For the sun, Lodders gives X = 0.7389, Y = 0.2463, and Z = 0.0148. Do I believe all those significant digits? No. Is there a good reason for them to be there? Yes. So without delving into those details, let’s just note that the universe is about 3 parts hydrogen, one part helium, with a sprinkling of everything else. Everything else being all the elements in the periodic table that aren’t hydrogen or helium – all the carbon and nitrogen and oxygen and silicon and magnesium and noble gases and actual metals – these all add up to about 1.5% of the mass of the sun, which is typical of nearby stars. So you can see why they’re all just metals to many astronomers.

For the mass of gas in galaxies, we need to correct what we measure in hydrogen for the presence of helium and metals. We measure the mass of atomic hydrogen using the 21 cm line, but that’s just the hydrogen. There is a corresponding amount of helium and metals that goes along with it. So we estimate the mass fraction in hydrogen, X, and use divide by that to get the total mass: M_gas = M_HI/X. We do the same for molecular gas, etc.

There are measurements of the metallicities of entire galaxies, but – you guessed it – this isn’t observationally cheap, and isn’t always available. So we need another proxy. Luckily for us, it turns out that once again there is a pretty good correlation of metallicity with stellar mass: galaxies with lots of stars have made lots of supernovae that have processed lots of material into metals. Most of it is still hydrogen, so this is a very subtle effect: 1/X = 1.34 for the tiniest dwarf, going up to about 1.4 for a galaxy like the Milky Way. Still, we know this happens, so we can account for it, at least in a statistical way.

For those who are curious about the details, or want the actual formulae to use, please refer to this AAS research note. Next time, I hope to discuss an application for all this.

Predictive Power in Science

“Winning isn’t everything. It’s the only thing.”
Red Sanders

This is a wise truth that has often been poorly interpreted. I despise some of the results that this sports quote has had in American culture. It has fostered a culture of bad sportsmanship in some places: an acceptance, even a dictum, that the ends justify the means – up to and including cheating, provided you can get away with it.

Winning every time is an impossible standard. In any competitive event, someone will win a particular game, and someone else will lose. Every participant will be on the losing side some of the time. Learning to lose gracefully despite a great effort is an essential aspect of sportsmanship that must be taught and learned, because it sure as hell isn’t part of human nature.

But there is wisdom here. The quote originates with a football coach. Football is a sport where there is a lot of everything – to even have a chance of winning, you have to do everything right. Not just performance on the field, but strategic choices made before and during the game, and mundane but essential elements like getting the right personnel on the field for each play. What? We’re punting? I thought it was third down!

You can do everything right and still lose. And that’s what I interpret the quote to really mean. You have to do everything to compete. But people will only judge you to be successful if you win.

To give a recent example, the Kansas City Chiefs won this year’s Superbowl. It was only a few months ago, though it seems much longer in pandemic time. The Chiefs dominated the Superbowl, but they nearly didn’t make it past the AFC Championship game.

The Tennessee Titans dominated the early part of the AFC Championship game. They had done everything right. They had peaked at the right time as a team in the overly long and brutal NFL season. They had an excellent game plan, just as they had had in handily defeating the highly favored New England Patriots on the way to the Championship game. Their defense admirably contained the high octane Chiefs offense. It looked like they were going to the Superbowl.

Then one key injury occurred. The Titans lost the only defender who could match up one on one with tight end Travis Kelce. This had an immediate impact on the game, as they Chiefs quickly realized they could successfully throw to Kelce over and over after not having been able to do so at all. The Titans were obliged to double-cover, which opened up other opportunities. The Chief’s offense went from impotent to unstoppable.

I remember this small detail because Kelce is a local boy. He attended the same high school as my daughters, playing on the same field they would (only shortly later) march on with the marching band during half times. If it weren’t for this happenstance of local interest, I probably wouldn’t have noticed this detail of the game, much less remember it.

The bigger point is that the Titans did everything right as a team. They lost anyway. All most people will remember is that the Chiefs won the Superbowl, not that the Titans almost made it there. Hence the quote:

“Winning isn’t everything. It’s the only thing.”

The hallmark of science is predictive power. This is what distinguishes it from other forms of knowledge. The gold standard is a prediction that is made and published in advance of the experiment that tests it. This eliminates the ability to hedge: either we get it right in advance, or we don’t.

The importance of such a prediction depends on how surprising it is. Predicting that the sun will rise tomorrow is not exactly a bold prediction, is it? If instead we have a new idea that changes how we think about how the world works, and makes a prediction that is distinct from current wisdom, then that’s very important. Judging how important a particular prediction may be is inevitably subjective.

It is rare that we actually meet the gold standard of a priori prediction, but it does happen. A prominent example is the prediction of gravitational lensing by General Relativity. Einstein pointed out that his theory predicted twice the light-bending that Newtonian theory did. Eddington organized an expedition to measure this effect during a solar eclipse, and claimed to confirm Einstein’s prediction within a few years of it having been made. This is reputed to have had a strong impact that led to widespread acceptance of the new theory. Some of that was undoubtedly due to Eddington’s cheerleading: it does not suffice merely to make a successful prediction, that it has happened needs to become widely known.

It is impossible to anticipate every conceivable experimental result and publish a prediction for it in advance. So there is another situation: does a theory predict what is observed? This has several standards. The highest standard deserves a silver medal. This happens when you work out the prediction of a theory, and you find that it gives exactly what is observed, with very little leeway. If you had had the opportunity to make the prediction in advance, it would have risen to the gold standard.

Einstein provides another example of a silver-standard prediction. A long standing problem in planetary dynamics was the excess precession of the perihelion of Mercury. The orientation of the elliptical orbit of Mercury changes slowly, with the major axis of the ellipse pivoting by 574 arcseconds per century. That’s a tiny rate of angular change, but we’ve been keeping very accurate records of where the planets are for a very long time, so it was well measured. Indeed, it was recognized early that precession would be cause by torques from other planets: it isn’t just Mercury going around the sun; the rest of the solar system matters too. Planetary torques are responsible for most of the effect, but not all. By 1859, Urbain Le Verrier had worked out that the torques from known planets should only amount to 532 arcseconds per century. [I am grossly oversimplifying some fascinating history. Go read up on it!] The point is that there was an excess, unexplained precession of 43 arcseconds per century. This discrepancy was known, known to be serious, and had no satisfactory explanation for many decades before Einstein came on the scene. No way he could go back in time and make a prediction before he was born! But when he worked out the implications of his new theory for this problem, the right answer fell straight out. It explained an ancient and terrible problem without any sort of fiddling: it had to be so.

The data for the precession of the perihelion of Mercury were far superior to the first gravitational lensing measurements made by Eddington and his colleagues. The precession was long known and accurately measured, the post facto prediction clean and irresolute. So in this case, the silver standard was perhaps better than the gold standard. Hence the question once posed to me by a philosopher of science: why we should care if the prediction came in advance of the observation? If X is a consequence of a theory, and X is observed, what difference does it make which came first?

In principle, none. In practice, it depends. I made the hedge above of “very little leeway.” If there is zero leeway, then silver is just as good as gold. There is no leeway to fudge it, so the order doesn’t matter.

It is rare that there is no leeway to fudge it. Theorists love to explore arcane facets of their ideas. They are exceedingly clever at finding ways to “explain” observations that their theory did not predict, even those that seem impossible for their theory to explain. So the standard by which such a post-facto “prediction” must be judged depends on the flexibility of the theory, and the extent to which one indulges said flexibility. If it is simply a matter of fitting for some small number of unknown parameters that are perhaps unknowable in advance, then I would award that a bronze medal. If instead one must strain to twist the theory to make it work out, then that merits at best an asterisk: “we fit* it!” can quickly become “*we’re fudging it!” That’s why truly a priori prediction is the gold standard. There is no way to go back in time and fudge it.

An important corollary is that if a theory gets its predictions right in advance, then we are obliged to acknowledge the efficacy of that theory. The success of a priori predictions is the strongest possible sign that the successful theory is a step in the right direction. This is how we try to maintain objectivity in science: it is how we know when to suck it up and say “OK, my favorite theory got this wrong, but this other theory I don’t like got its prediction exactly right. I need to re-think this.” This ethos has been part of science for as long as I can remember, and a good deal longer than that. I have heard some argue that this is somehow outdated and that we should give up this ethos. This is stupid. If we give up the principle of objectivity, science would quickly degenerate into a numerological form of religion: my theory is always right! and I can bend the numbers to make it seem so.

Hence the hallmark of science is predictive power. Can a theory be applied to predict real phenomena? It doesn’t matter whether the prediction is made in advance or not – with the giant caveat that “predictions” not be massaged to fit the facts. There is always a temptation to massage one’s favorite theory – and obfuscate the extent to which one is doing so. Consequently, truly a priori prediction must necessarily remain the gold standard in science. The power to make such predictions is fundamental.

Predictive power in science isn’t everything. It’s the only thing.

As I was writing this, I received email to the effect that these issues are also being discussed elsewhere, by Jim Baggot and Sabine Hossenfelder. I have not yet read what they have to say.

The halo mass function

I haven’t written much here of late. This is mostly because I have been busy, but also because I have been actively refraining from venting about some of the sillier things being said in the scientific literature. I went into science to get away from the human proclivity for what is nowadays called “fake news,” but we scientists are human too, and are not immune from the same self-deception one sees so frequently exercised in other venues.

So let’s talk about something positive. Current grad student Pengfei Li recently published a paper on the halo mass function. What is that and why should we care?

One of the fundamental predictions of the current cosmological paradigm, ΛCDM, is that dark matter clumps into halos. Cosmological parameters are known with sufficient precision that we have a very good idea of how many of these halos there ought to be. Their number per unit volume as a function of mass (so many big halos, so many more small halos) is called the halo mass function.

An important test of the paradigm is thus to measure the halo mass function. Does the predicted number match the observed number? This is hard to do, since dark matter halos are invisible! So how do we go about it?

Galaxies are thought to form within dark matter halos. Indeed, that’s kinda the whole point of the ΛCDM galaxy formation paradigm. So by counting galaxies, we should be able to count dark matter halos. Counting galaxies was an obvious task long before we thought there was dark matter, so this should be straightforward: all one needs is the measured galaxy luminosity function – the number density of galaxies as a function of how bright they are, or equivalently, how many stars they are made of (their stellar mass). Unfortunately, this goes tragically wrong.

Galaxy stellar mass function and the predicted halo mass function — Fig. 5 from the review by Bullock & Boylan-Kolchin. The number density of objects is shown as a function of their mass. Colored points are galaxies. The solid line is the predicted number of dark matter halos. The dotted line is what one would expect for galaxies if all the normal matter associated with each dark matter halo turned into stars.

This figure shows a comparison of the observed stellar mass function of galaxies and the predicted halo mass function. It is from a recent review, but it illustrates a problem that goes back as long as I can remember. We extragalactic astronomers spent all of the ’90s obsessing over this problem. [I briefly thought that I had solved this problem, but I was wrong.] The observed luminosity function is nearly flat while the predicted halo mass function is steep. Consequently, there should be lots and lots of faint galaxies for every bright one, but instead there are relatively few. This discrepancy becomes progressively more severe to lower masses, with the predicted number of halos being off by a factor of many thousands for the faintest galaxies. The problem is most severe in the Local Group, where the faintest dwarf galaxies are known. Locally it is called the missing satellite problem, but this is just a special case of a more general problem that pervades the entire universe.

Indeed, the small number of low mass objects is just one part of the problem. There are also too few galaxies at large masses. Even where the observed and predicted numbers come closest, around the scale of the Milky Way, they still miss by a large factor (this being a log-log plot, even small offsets are substantial). If we had assigned “explain the observed galaxy luminosity function” as a homework problem and the students had returned as an answer a line that had the wrong shape at both ends and at no point intersected the data, we would flunk them. This is, in effect, what theorists have been doing for the past thirty years. Rather than entertain the obvious interpretation that the theory is wrong, they offer more elaborate interpretations.

Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everybody gets busy on the proof.
J. K. Galbraith

Theorists persist because this is what CDM predicts, with or without Λ, and we need cold dark matter for independent reasons. If we are unwilling to contemplate that ΛCDM might be wrong, then we are obliged to pound the square peg into the round hole, and bend the halo mass function into the observed luminosity function. This transformation is believed to take place as a result of a variety of complex feedback effects, all of which are real and few of which are likely to have the physical effects that are required to solve this problem. That’s way beyond the scope of this post; all we need to know here is that this is the “physics” behind the transformation that leads to what is currently called Abundance Matching.

Abundance matching boils down to drawing horizontal lines in the above figure, thus matching galaxies with dark matter halos with equal number density (abundance). So, just reading off the graph, a galaxy of stellar mass M_* = 10⁸ M_☉ resides in a dark matter halo of 10¹¹ M_☉, one like the Milky Way with M_* = 5 x 10¹⁰ M_☉ resides in a 10¹² M_☉ halo, and a giant galaxy with M_* = 10¹² M_☉ is the “central” galaxy of a cluster of galaxies with a halo mass of several 10¹⁴ M_☉. And so on. In effect, we abandon the obvious and long-held assumption that the mass in stars should be simply proportional to that in dark matter, and replace it with a rolling fudge factor that maps what we see to what we predict. The rolling fudge factor that follows from abundance matching is called the stellar mass–halo mass relation. Many of the discussions of feedback effects in the literature amount to a post hoc justification for this multiplication of forms of feedback.

This is a lengthy but insufficient introduction to a complicated subject. We wanted to get away from this, and test the halo mass function more directly. We do so by use of the velocity function rather than the stellar mass function.

The velocity function is the number density of galaxies as a function of how fast they rotate. It is less widely used than the luminosity function, because there is less data: one needs to measure the rotation speed, which is harder to obtain than the luminosity. Nevertheless, it has been done, as with this measurement from the HIPASS survey:

Galaxy velocity function — The number density of galaxies as a function of their rotation speed (Zwaan et al. 2010). The bottom panel shows the raw number of galaxies observed; the top panel shows the velocity function after correcting for the volume over which galaxies can be detected. Faint, slow rotators cannot be seen as far away as bright, fast rotators, so the latter are always over-represented in galaxy catalogs.

The idea here is that the flat rotation speed is the hallmark of a dark matter halo, providing a dynamical constraint on its mass. This should make for a cleaner measurement of the halo mass function. This turns out to be true, but it isn’t as clean as we’d like.

Those of you who are paying attention will note that the velocity function Martin Zwaan measured has the same basic morphology as the stellar mass function: approximately flat at low masses, with a steep cut off at high masses. This looks no more like the halo mass function than the galaxy luminosity function did. So how does this help?

To measure the velocity function, one has to use some readily obtained measure of the rotation speed like the line-width of the 21cm line. This, in itself, is not a very good measurement of the halo mass. So what Pengfei did was to fit dark matter halo models to galaxies of the SPARC sample for which we have good rotation curves. Thanks to the work of Federico Lelli, we also have an empirical relation between line-width and the flat rotation velocity. Together, these provide a connection between the line-width and halo mass:

Halo mass-line width relation — The relation Pengfei found between halo mass (M200) and line-width (W) for the NFW (ΛCDM standard) halo model fit to rotation curves from the SPARC galaxy sample.

Once we have the mass-line width relation, we can assign a halo mass to every galaxy in the HIPASS survey and recompute the distribution function. But now we have not the velocity function, but the halo mass function. We’ve skipped the conversion of light to stellar mass to total mass and used the dynamics to skip straight to the halo mass function:

Empirical halo mass function — The halo mass function. The points are the data; these are well fit by a Schechter function (black line; this is commonly used for the galaxy luminosity function). The red line is the prediction of ΛCDM for dark matter halos.

The observed mass function agrees with the predicted one! Test successful! Well, mostly. Let’s think through the various aspects here.

First, the normalization is about right. It does not have the offset seen in the first figure. As it should not – we’ve gone straight to the halo mass in this exercise, and not used the luminosity as an intermediary proxy. So that is a genuine success. It didn’t have to work out this well, and would not do so in a very different cosmology (like SCDM).

Second, it breaks down at high mass. The data shows the usual Schechter cut-off at high mass, while the predicted number of dark matter halos continues as an unabated power law. This might be OK if high mass dark matter halos contain little neutral hydrogen. If this is the case, they will be invisible to HIPASS, the 21cm survey on which this is based. One expects this, to a certain extent: the most massive galaxies tend to be gas-poor ellipticals. That helps, but only by shifting the turn-down to slightly higher mass. It is still there, so the discrepancy is not entirely cured. At some point, we’re talking about large dark matter halos that are groups or even rich clusters of galaxies, not individual galaxies. Still, those have HI in them, so it is not like they’re invisible. Worse, examining detailed simulations that include feedback effects, there do seem to be more predicted high-mass halos that should have been detected than actually are. This is a potential missing gas-rich galaxy problem at the high mass end where galaxies are easy to detect. However, the simulations currently available to us do not provide the information we need to clearly make this determination. They don’t look right, so far as we can tell, but it isn’t clear enough to make a definitive statement.

Finally, the faint-end slope is about right. That’s amazing. The problem we’ve struggled with for decades is that the observed slope is too flat. Here a steep slope just falls out. It agrees with the ΛCDM down to the lowest mass bin. If there is a missing satellite-type problem here, it is at lower masses than we probe.

That sounds great, and it is. But before we get too excited, I hope you noticed that the velocity function from the same survey is flat like the luminosity function. So why is the halo mass function steep?

When we fit rotation curves, we impose various priors. That’s statistics talk for a way of keeping parameters within reasonable bounds. For example, we have a pretty good idea of what the mass-to-light ratio of a stellar population should be. We can therefore impose as a prior that the fit return something within the bounds of reason.

One of the priors we imposed on the rotation curve fits was that they be consistent with the stellar mass-halo mass relation. Abundance matching is now part and parcel of ΛCDM, so it made sense to apply it as a prior. The total mass of a dark matter halo is an entirely notional quantity; rotation curves (and other tracers) pretty much never extend far enough to measure this. So abundance matching is great for imposing sense on a parameter that is otherwise ill-constrained. In this case, it means that what is driving the slope of the halo mass function is a prior that builds-in the right slope. That’s not wrong, but neither is it an independent test. So while the observationally constrained halo mass function is consistent with the predictions of ΛCDM; we have not corroborated the prediction with independent data. What we really need at low mass is some way to constrain the total mass of small galaxies out to much larger radii that currently available. That will keep us busy for some time to come.

Fuzzy Thing!

I was contacted today by a colleague at NASA’s Goddard Space Flight Center who was seeking to return some photographic plates of Halley’s comet that had been obtained with the Burrell Schmidt telescope. I at first misread the email – I get so many requests for data, I initially assumed that he was looking for said plates. That sent me into a frenzy of where the heck are they? about data obtained by others well before my time as the director of the Warner & Swasey Observatory. Comet Halley last came by in 1986.

Fortunately, reading comprehension kicked in, and I realized that all I really needed to figure out was where they should go. The lower pressure version of where the heck are they? That would be the Pisgah Astronomical Research Institute, which has had the good sense to archive the vast treasury of astronomical plates that many observatories obtained in the pre-digital era but don’t always have the ability to preserve. But this post isn’t about that; it is just a spark to the memory.

In 1986, I was a first-year graduate student in the Princeton physics department. As such, I had at that time little more competence in observing the sky than any other physicist (practically none). Nevertheless, I traipsed out into an open field at the dark edge of town on a clear night with a pair of binoculars and a vague knowledge of what part of the sky Comet Halley should be in. How hard could it be to spot the most famous comet in history?

Impossibly hard. There was nothing to see, so far as I could find. The apparition of 1986 was a bust. This informed in me a bad attitude towards comets. There had never been a good apparition in my lifetime (all of 22 years at that point), and Halley certainly wasn’t one. I accepted that decent comets must be a rare occurrence.

Flash forward a decade to 1996, by which time I was an accomplished observer with a good working knowledge of the celestial sphere. A new comet was discovered – Hyakutake – and with it came much hype. Yeah, yeah, I’d heard it all before. Boring. Comets were always a flop.

Comet Hyakutake made a close approach to Earth in March of 1996. Its wikipedia page is pretty good, with a nice illustration of its orbit and its path on the sky as perceived from the Earth. I was working at DTM at the time, where there were lots of planetary scientists as well as a few astronomers. Someone posted an ephemeris, so despite my distrust of comets I found myself peeking at what its trajectory would be. Nevertheless, we had a long period of cloudy weather, so there was nothing to see even if there was something to see, which I expected there wasn’t.

At this time, my elder daughter Caitlyn was two years old. I made a habit of taking her out and pointing things out in the sky. We watched the sunset, the moon set after it near new moon, and the moon rise near full moon. She seemed content to listen to her old man babble about the lights in the sky. Apparently more of that sank in than I realized.

My wife Anne was teaching at Loyola, and her department chair had invited us over for a party around the vernal equinox. We enjoyed the adult company and Caitlyn put up well with it – up to a point. It got dark and we bid our farewells and headed out. We had parked across the street, and on the way out Betsy (our hostess) said “Stacy – you’re an astronomer. Where’s the comet?”

I got this pained expression. Stupid comets. But it had cleared up for the first time in nearly a week, and looking up from the front door, I could quickly orient myself on the sky. Doing so, I realize that the comet was behind the house. So I pointed up and over, towards the back yard and through the roof: “Over there.” I continued across the street to the car with the toddler cradled in my left arm, fiddling with the keys with my right hand.

We did not have a nice car: one had to insert the key manually into the door to unlock it. As I went around the car to get to the driver’s side, I was focused on this mundane task. It did not occur to me to look up in the direction I had just pointed. I felt Caitlyn stretch her arm to point at the sky, exclaiming “Fuzzy thing!”

I looked up. There is was: a big, bright, fuzzy ball. A brilliant cometary apparition, the coma easily visible even in Baltimore. My two-year old daughter spotted it and accurately classified it before I even looked up.

Comet Hyakutake was an amazing event. Not only spectacular to look at, but it drove home celestial mechanics in a visceral way. It was at this time very close to Earth (by the scale of such things). That meant it made noticeable progress in its orbit from night to night. You couldn’t see it moving just staring at it, but one night is was here, the next night it was there, the following night over there. It was skipping through the constellations at a dizzying speed for an object that takes c. 70,000 years to complete one orbit. But we were close enough that one could easily see the progress it made across the sky from night to night, if not minute to minute. If you wanted to take a picture with a telescope, you had to track the telescope to account for this – hence the star trails in the image above: the stars appear as streaks because the telescope is moving with the comet, not with the sky.

The path of Comet Huyakutake across the sky.

This figure (credit: Tom Ruen) shows the orbital path of Comet Huyakutake projected on the sky (constellations outlined in blue). Most of the time, the comet is far away near the aphelion of its orbit. As it fell in towards the sun, its path made annual ellipses due to the reflex motion of the Earth’s own orbit – the parallax. These grew in size until the comet came sweeping by in the month of March, 1996. Think about it: it spent tens of thousands of years spiraling down towards us, only to shoot by, transitioning well across the sky in only a couple of weeks. Celestial mechanics made visible.

Not long after Hyakutake started to fade, Comet Hale-Bopp became visible. Hale-Bopp did not pass as close to the Earth as Hyakutake, so it didn’t leap across the sky like Tom Bombadil. But Hale-Bopp was a physically larger comet. As such, it got bright and stayed bright for a long time, remaining visible to the naked eye for a record year and half. In the months after Hyakutake’s apparition, we could see Hale-Bopp chasing the sunset from the balcony of our apartment. Caitlyn and I would sit there and watch it as the twilight faded into dark. Her experience of comets had been the opposite of mine: where in my thirty years (before that point) they had been rare and disappointing, in her (by then) three years they had been common and spectacular.

The sky is full of marvels. You never know when you might get to see one.

New web domain

I happened to visit this blog as a visitor from a computer not mine. Seeing it that way made me realize how obnoxious the ads had become. So WordPress’s extortion worked; I’ve agreed to send them a few $ every month to get rid of the ads. With it comes a new domain name: tritonstation.com. Bookmarks to the previous website (tritonstation.wordpress.com) should redirect here. Let me know if a problem arises, or the barrage of ads fails to let up. I may restructure the web page so there is more here than just this blog, but that will have to await my attention in my copious spare time.

As it happens, I depart soon to attend an IAU meeting on galaxy dynamics. This is being held in part to honor the career of Prof. Jerry Sellwood, with whom I had the pleasure to work while a postdoc at Rutgers. He hosted a similar meeting at Rutgers in 1998; I’m sure that some of the same issues discussed then will be debated again next week.

Triton Station

A Blog About the Science and Sociology of Cosmology and Dark Matter