I haven’t written much here of late. This is mostly because I have been busy, but also because I have been actively refraining from venting about some of the sillier things being said in the scientific literature. I went into science to get away from the human proclivity for what is nowadays called “fake news,” but we scientists are human too, and are not immune from the same self-deception one sees so frequently exercised in other venues.

So let’s talk about something positive. Current grad student Pengfei Li recently published a paper on the halo mass function. What is that and why should we care?

One of the fundamental predictions of the current cosmological paradigm, ΛCDM, is that dark matter clumps into halos. Cosmological parameters are known with sufficient precision that we have a very good idea of how many of these halos there ought to be. Their number per unit volume as a function of mass (so many big halos, so many more small halos) is called the halo mass function.

An important test of the paradigm is thus to measure the halo mass function. Does the predicted number match the observed number? This is hard to do, since dark matter halos are invisible! So how do we go about it?

Galaxies are thought to form within dark matter halos. Indeed, that’s kinda the whole point of the ΛCDM galaxy formation paradigm. So by counting galaxies, we should be able to count dark matter halos. Counting galaxies was an obvious task long before we thought there was dark matter, so this should be straightforward: all one needs is the measured galaxy luminosity function – the number density of galaxies as a function of how bright they are, or equivalently, how many stars they are made of (their stellar mass). Unfortunately, this goes tragically wrong.

Galaxy stellar mass function and the predicted halo mass function
Fig. 5 from the review by Bullock & Boylan-Kolchin. The number density of objects is shown as a function of their mass. Colored points are galaxies. The solid line is the predicted number of dark matter halos. The dotted line is what one would expect for galaxies if all the normal matter associated with each dark matter halo turned into stars.

This figure shows a comparison of the observed stellar mass function of galaxies and the predicted halo mass function. It is from a recent review, but it illustrates a problem that goes back as long as I can remember. We extragalactic astronomers spent all of the ’90s obsessing over this problem. [I briefly thought that I had solved this problem, but I was wrong.] The observed luminosity function is nearly flat while the predicted halo mass function is steep. Consequently, there should be lots and lots of faint galaxies for every bright one, but instead there are relatively few. This discrepancy becomes progressively more severe to lower masses, with the predicted number of halos being off by a factor of many thousands for the faintest galaxies. The problem is most severe in the Local Group, where the faintest dwarf galaxies are known. Locally it is called the missing satellite problem, but this is just a special case of a more general problem that pervades the entire universe.

Indeed, the small number of low mass objects is just one part of the problem. There are also too few galaxies at large masses. Even where the observed and predicted numbers come closest, around the scale of the Milky Way, they still miss by a large factor (this being a log-log plot, even small offsets are substantial). If we had assigned “explain the observed galaxy luminosity function” as a homework problem and the students had returned as an answer a line that had the wrong shape at both ends and at no point intersected the data, we would flunk them. This is, in effect, what theorists have been doing for the past thirty years. Rather than entertain the obvious interpretation that the theory is wrong, they offer more elaborate interpretations.

Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everybody gets busy on the proof.

J. K. Galbraith

Theorists persist because this is what CDM predicts, with or without Λ, and we need cold dark matter for independent reasons. If we are unwilling to contemplate that ΛCDM might be wrong, then we are obliged to pound the square peg into the round hole, and bend the halo mass function into the observed luminosity function. This transformation is believed to take place as a result of a variety of complex feedback effects, all of which are real and few of which are likely to have the physical effects that are required to solve this problem. That’s way beyond the scope of this post; all we need to know here is that this is the “physics” behind the transformation that leads to what is currently called Abundance Matching.

Abundance matching boils down to drawing horizontal lines in the above figure, thus matching galaxies with dark matter halos with equal number density (abundance). So, just reading off the graph, a galaxy of stellar mass M* = 108 M resides in a dark matter halo of 1011 M, one like the Milky Way with M* = 5 x 1010 M resides in a 1012 M halo, and a giant galaxy with M* = 1012 M is the “central” galaxy of a cluster of galaxies with a halo mass of several 1014 M. And so on. In effect, we abandon the obvious and long-held assumption that the mass in stars should be simply proportional to that in dark matter, and replace it with a rolling fudge factor that maps what we see to what we predict. The rolling fudge factor that follows from abundance matching is called the stellar mass–halo mass relation. Many of the discussions of feedback effects in the literature amount to a post hoc justification for this multiplication of forms of feedback.

This is a lengthy but insufficient introduction to a complicated subject. We wanted to get away from this, and test the halo mass function more directly. We do so by use of the velocity function rather than the stellar mass function.

The velocity function is the number density of galaxies as a function of how fast they rotate. It is less widely used than the luminosity function, because there is less data: one needs to measure the rotation speed, which is harder to obtain than the luminosity. Nevertheless, it has been done, as with this measurement from the HIPASS survey:

Galaxy velocity function
The number density of galaxies as a function of their rotation speed (Zwaan et al. 2010). The bottom panel shows the raw number of galaxies observed; the top panel shows the velocity function after correcting for the volume over which galaxies can be detected. Faint, slow rotators cannot be seen as far away as bright, fast rotators, so the latter are always over-represented in galaxy catalogs.

The idea here is that the flat rotation speed is the hallmark of a dark matter halo, providing a dynamical constraint on its mass. This should make for a cleaner measurement of the halo mass function. This turns out to be true, but it isn’t as clean as we’d like.

Those of you who are paying attention will note that the velocity function Martin Zwaan measured has the same basic morphology as the stellar mass function: approximately flat at low masses, with a steep cut off at high masses. This looks no more like the halo mass function than the galaxy luminosity function did. So how does this help?

To measure the velocity function, one has to use some readily obtained measure of the rotation speed like the line-width of the 21cm line. This, in itself, is not a very good measurement of the halo mass. So what Pengfei did was to fit dark matter halo models to galaxies of the SPARC sample for which we have good rotation curves. Thanks to the work of Federico Lelli, we also have an empirical relation between line-width and the flat rotation velocity. Together, these provide a connection between the line-width and halo mass:

Halo mass-line width relation
The relation Pengfei found between halo mass (M200) and line-width (W) for the NFW (ΛCDM standard) halo model fit to rotation curves from the SPARC galaxy sample.

Once we have the mass-line width relation, we can assign a halo mass to every galaxy in the HIPASS survey and recompute the distribution function. But now we have not the velocity function, but the halo mass function. We’ve skipped the conversion of light to stellar mass to total mass and used the dynamics to skip straight to the halo mass function:

Empirical halo mass function
The halo mass function. The points are the data; these are well fit by a Schechter function (black line; this is commonly used for the galaxy luminosity function). The red line is the prediction of ΛCDM for dark matter halos.

The observed mass function agrees with the predicted one! Test successful! Well, mostly. Let’s think through the various aspects here.

First, the normalization is about right. It does not have the offset seen in the first figure. As it should not – we’ve gone straight to the halo mass in this exercise, and not used the luminosity as an intermediary proxy. So that is a genuine success. It didn’t have to work out this well, and would not do so in a very different cosmology (like SCDM).

Second, it breaks down at high mass. The data shows the usual Schechter cut-off at high mass, while the predicted number of dark matter halos continues as an unabated power law. This might be OK if high mass dark matter halos contain little neutral hydrogen. If this is the case, they will be invisible to HIPASS, the 21cm survey on which this is based. One expects this, to a certain extent: the most massive galaxies tend to be gas-poor ellipticals. That helps, but only by shifting the turn-down to slightly higher mass. It is still there, so the discrepancy is not entirely cured. At some point, we’re talking about large dark matter halos that are groups or even rich clusters of galaxies, not individual galaxies. Still, those have HI in them, so it is not like they’re invisible. Worse, examining detailed simulations that include feedback effects, there do seem to be more predicted high-mass halos that should have been detected than actually are. This is a potential missing gas-rich galaxy problem at the high mass end where galaxies are easy to detect. However, the simulations currently available to us do not provide the information we need to clearly make this determination. They don’t look right, so far as we can tell, but it isn’t clear enough to make a definitive statement.

Finally, the faint-end slope is about right. That’s amazing. The problem we’ve struggled with for decades is that the observed slope is too flat. Here a steep slope just falls out. It agrees with the ΛCDM down to the lowest mass bin. If there is a missing satellite-type problem here, it is at lower masses than we probe.

That sounds great, and it is. But before we get too excited, I hope you noticed that the velocity function from the same survey is flat like the luminosity function. So why is the halo mass function steep?

When we fit rotation curves, we impose various priors. That’s statistics talk for a way of keeping parameters within reasonable bounds. For example, we have a pretty good idea of what the mass-to-light ratio of a stellar population should be. We can therefore impose as a prior that the fit return something within the bounds of reason.

One of the priors we imposed on the rotation curve fits was that they be consistent with the stellar mass-halo mass relation. Abundance matching is now part and parcel of ΛCDM, so it made sense to apply it as a prior. The total mass of a dark matter halo is an entirely notional quantity; rotation curves (and other tracers) pretty much never extend far enough to measure this. So abundance matching is great for imposing sense on a parameter that is otherwise ill-constrained. In this case, it means that what is driving the slope of the halo mass function is a prior that builds-in the right slope. That’s not wrong, but neither is it an independent test. So while the observationally constrained halo mass function is consistent with the predictions of ΛCDM; we have not corroborated the prediction with independent data. What we really need at low mass is some way to constrain the total mass of small galaxies out to much larger radii that currently available. That will keep us busy for some time to come.

24 thoughts on “The halo mass function

  1. Pretty interesting. I am not sure how to process all this, it involves several aspects of Galactic Astronomy that I am not as familiar with as I would like to be, however I am very glad to see you posting again. You have definitely given me a new topic (sub topic) to read up on.


  2. Fascinating, and although not a physicist I feel that I understand most of this. My favourite bit? “The rolling fudge factor that follows from abundance matching is called the stellar mass–halo mass relation.” Please do keep sharing. I was also amused that you couldn’t help but have another pop at SCDM.


  3. LCDM is the SCDM of today – a theory every one believes must be true, but could prove to be badly misleading. So yes, I always take the opportunity to remind people of this analogy, especially at I notice many younger scientists are blissfully unaware that we’ve been down this path before.


  4. Stacy,

    a straight question: do you believe in Lambda, i.e., in dark energy?

    As you know, several research groups question its reality, as done, e.g., by the various papers of Sarkar, Tsagas, Rameez, or Wilshire. They claim that Lambda is an artefact of either measurements or the peculiar situation of our solar system. (Their claim is vaguely similar to MOND’s claim that dark matter is an artefact of measurements – though the analogy is not really good.)

    Best regards


  5. I believe the evidence that leads us to infer Lambda.

    Lambda is a construct within the framework of General Relativity. It might be what it is, or it might be an indication that the theory is inadequate. In the latter case, one imagines there could be some more general theory in which it becomes clear what it really means… the underlying theory might deviate from GR in a way that “looks like” Lambda. So believing the evidence doesn’t compel belief in dark energy as an entity – that is one possibility, but not the only one.

    If we restrict ourselves to the context of GR, the evidence for Lambda is overwhelming. I do not believe it can be an artifact of measurement error or our particular location. Attempts to dodge it usually involve dodging a single observation (e.g., Type Ia SN) while ignoring a host of other evidence (see https://tritonstation.com/2019/01/28/a-personal-recollection-of-how-we-learned-to-stop-worrying-and-love-the-lambda/).


  6. IF, our universe is not alone in the cosmos. If the cosmos is much bigger / more massive and older than our Big Bang. Could Dark Energy just be the gravitational pull of the cosmos? I know this looks like just another crackpot idea but it is falsifiable and testable. It also resolves baryon asymmetry.
    The universe has always been much bigger and older than we thought.
    Hope I’m not abusing your blog. Keep writing.


  7. You explanations here underscore how hard it can be for a layperson to understand what is really behind arguments for/against dark matter/energy. These statistical operations adjusting data may be well motivated, but it will not be apparent to the layperson that the close match of the “data” to predicted values is due to assumptions relating to the data adjustments.

    I really value the work you put into this blog to give more context and nuance.


    1. The predominant idea for this goes back to the mid-70s, and essentially says that the gas cooling time has to be shorter than the gravitational collapse time. Basically, there has to be time for the gas to collapse. This comes out to about the right upper mass limit for individual galaxies, but is rather crude in many respects – the cooling time is very sensitive to the metallicity of the gas; a spherical cow that ignores hierarchical substructure is assumed, and so on. Still, that’s the best I’ve heard.


  8. I know I should just look in the paper but did you guys do an experiment to see what happens as you relax that abundance matching prior? It would be nice to see a figure that somehow illustrates how the mass function is being “forced” to lcdm. Also, how come this prior is not forcing the high mass end to fit the simulation prediction?


    1. Yes; hope to blog about that next, as Pengfei has a big paper coming out Soon. The short answer is that there is nothing in the kinematics that requires an abundance matching-like behavior, so left to itself, the faint end remains flat. At the bright end, one has to imagine that these very massive halos host groups and clusters, not individual galaxies, so the counting get screwy. Still, there is a problem with just calling the most massive galaxies the “centrals” of big groups, as many manifestly are not.


  9. Stacy,
    thank you for your clear answer. The cosmological constant indeed pops up in the Hilbert action of general relativity. How big is the modern measurement evidence that Lambda was constant over time not only over the last 100 million years, but also over the last 5 or more billion years?

    (There are two backgrounds for the question: first, the theoretical papers on decaying Lambda scattered across the literature; secondly, most microscopic models about Lambda, such as emergent gravity, also suggest a decaying Lambda, with a value that was much higher in the early universe.)

    Thank you for your wonderful blog. Please go on. Best regards, Christoph


  10. Within the standard cosmology, the evidence for Lambda is strong. The evidence for time variable Lambda is not. People have certainly looked for such en effect, usually quantified as deviations of the dark energy equation of state from the nominal value for Lambda (P = -w*p where P is pressure, p energy density, and w = 1 for pure Lambda). The vast majority of such constraints find zero deviation: w = 1.0 plus or minus a small amount (< 0.1).


  11. Thank you Stacy, but wouldn’t a slowly decaying Lambda (such as Lambda proportional to 1/R^2, R being the Hubble radius) still have w=1?


  12. Stacy,

    thanks a lot for the interesting post.

    At the end, what does it mean? Does it solve the missing satellite or dwarf galaxy problem? So the key idea, the rotation curves are not measured to far enough radii and the dark matter halos contain much more mass than so far assumed?



    1. It doesn’t solve the missing satellite problem; it just does not indicate one in the field, to the extent that we can measure. It has long been the problem that rotation curves (and most other tracers) don’t indicate where the edge of the dark matter halo is, so one is free to impose the “right” amount in the choice of priors (“right” in this context meaning what abundance matching wants).


  13. Stacy,

    thanks for the explanation, I am not so sure if I understood you correctly.

    In a more general context, does the new findings/interpretation solves one of the pressing problems of ΛCDM? If it turnes out that the dark matter halos are much bigger than so far assumed, that they contain much more mass/dark matter, is that a new problem because for ΛCDM, since the angular distribution of the microwave background indicates that 20% of the total mass is baryonic? Or more directly, since you allways talk about dark matter halos, are you conviced that dark matter is indeed the cause of dynamics on galaxy scales?



    1. No. Yes. No.
      This work does not solve any problem for LCDM, it only shows that these data are not necessarily in conflict with it at the low mass end.
      It is correct to infer that the required halo masses vastly exceed what they should for the cosmic baryon fraction. This is not a new problem, at least not to me (https://tritonstation.com/2016/07/30/missing-baryons/). In effect, we have to have two forms of dark matter in every galaxy: non-baryonic cold dark matter AND dark baryons.
      I am not at all convinced that dark matter is the cause of galaxy dynamics. But I do believe in objectivity, which involves seeing a problem from all sides. So I can put on a CDM hat and discuss that data in that context. That’s what one has to do to test a hypothesis – one needs to understand what it predicts, and not just knock down straw men.

      Liked by 1 person

  14. Not a scientist but love this blog. Surely it’s possible to directly constrain the halo mass by looking at galaxy mergers. A bigger halo would mean a faster closing velocity so if you looked at a population of mergers vs a model you could test this? Constructing the model sounds tricky for sure.


    1. That is an excellent idea. It was examined by Dubinski et al a couple decades ago: https://arxiv.org/abs/astro-ph/9902217. What they found was that galaxy+halo models that made realistic-looking merger remnants needed to have progenitors with declining rotation curves. Those with realistically flat rotation curves did not make plausible merger remnants. That contradiction was briefly debated around the time, failed to penetrate the haze of cognitive dissonance, and has since been ignored.

      As for closing velocity, yes, that depends on both the halo mass and the law of gravity. There is a limit as to how fast merging objects can fall together as they only have the age of the universe in which to accelerate. So even very massive things can only get up so much speed. The bullet cluster is one of the most massive objects in the universe, and yet its collision speed is considerably higher than the formal speed limit. In contrast, high-speed collisions turn out to be a natural consequence of MOND (https://arxiv.org/abs/0704.0381) – the longer effective range of gravity gets smaller masses up to a higher collision speed.

      So yes, we should be able to test that with a population of merging objects. There are ongoing studies to look at this in galaxy clusters. The preliminary results I have seen suggest that high speed collisions are common, as predicted by MOND, but I rather doubt these will prove conclusive, as there are too many uncertainties and ways to fudge the theories.

      Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s