The Fractal Ratchet

How much of what you see is an artifact of how hard you looked?

A lot of modern discourse boils down to an argument between two methods of seeing the world:

1. Science is real, put up or shut up. Vague assertions of “common sense” without evidence are just cope. How many great scientific discoveries had to deal with a enormous headwind of people saying they’re impossible? Semmelweis was committed to a mental hospital for telling doctors to wash their hands between autopsies and deliveries; that’s the anti-progress you “common sense” folks are fighting for. We’re about “rationality”.

2. Hey, those people up in method 1 come up with some insane horseshit sometimes, huh? The metrics going up seem awfully uncorrelated with my life getting noticeably better. And when I go to the spaces where these guys are speaking freely, it’s about utils and emulation of trillions of beings and methodologies that seem to collapse in the real world. I’m not against figuring stuff out, but it feels like you have to interleave your findings with the common sense you hate so much. We’re about “reasonableness”.

A lot of technically fluent people go on a journey of being more method 1 than the people around them and benefiting from it. But as they drift into social spaces more heavily dominated by method 1 people, they notice a sort of…uneasiness. Suddenly they’re not reaping obvious benefits in their life by measuring things, they’re just getting new numbers they can smash together in increasingly sophisticated ways for decreasingly obvious reasons. Eventually you fall into what David Chapman calls “post-rational nihilism”, and if you manage to climb out of it you end up a lot more method 2 for the trouble. Thus the tech worker to farmer pipeline: in the end you return to the halcyon days of school, where knowledge leads directly to positive outcomes in your environment.

But, well, Semmelweis really was committed to that mental hospital, and some assertions of common sense really are cope. The method 1 folks feel perfectly justified in asking for something more “real” than just “lmao it’s common sense”, and the method 2 people usually don’t have a ready articulation for why they’re ignoring a bad metric. At the same time, the Desystemize archives are full of horror stories about premature systematization.

You can resolve this tension if you find a better metric to use instead, like Ostfield and his ticks. There’s a great timely example from Adam Tooze, who found that his “objective” understanding of inflation clashed with his lived experience:

Chartbook 327 From "anti-core" to "felt inflation": Or how I calmed my populist demons & resolved my cognitive dissonance on inflation, and how the Fed could do more to help.

He’s willing to articulate exactly this struggle between the two methods:

So, what gives? I’ve been feeling torn. My inability to reconcile personal and macro narrative reached something of a crisis point in an Uber a few weeks ago, when discussing the economic situation with the highly sophisticated guy doing the driving. He had NPR on and they were reporting the inflation numbers. As we both guffawed with disbelief, I actually found myself saying: “Yeah, I don’t get it either. There is something wrong with the numbers. They aren’t capturing our reality, are they.”

I could feel the tug. Was I morphing from a sophisticated social constructivist on inflation, to being something closer to a “fakenews” guy? More seriously, what about the rest of the American public? What are citizens to make of such a jarring discrepancy between felt reality and the officially reported, technocratic version?

But he only revealed his struggle because he found an answer, the metric of “anti-core” inflation focusing on food and energy.

If we focus only on food and energy, the price shock of 2021-2 was worse than that in 1973. It is second only to the Iran-crisis shock of 1979, the crisis that put paid to what little chance Jimmy Carter had of reelection in 1980.

Phew. No fake news. Just a familiar story of the way in which the construction of statistical series shapes our view of economic reality, both illuminating and obscuring.

He has relief now that he has an answer - but what should this teach him about the tension? What have we learned about that feeling he had before while talking to his Uber driver: “This all makes objective sense, but doesn’t feel real?” How can he be quicker to notice something like that next time? Before he had access to “anti-core inflation”, his feelings about inflation being a bad metric for his lived experience were just as true. When is common sense the right thing to use, and when isn’t it? And please don’t say the answer to this is “well, use your common sense and decide!”

We don’t have the right metaphorical technology to talk about this yet, so I’m going to step away from the question for a while. We’ve got something else to talk about, a specific phenomenon around detail and comparison that pops up everywhere once you start looking for it. I like to call the “fractal ratchet”. Let me describe it just for the sheer joy of naming patterns into concepts and we’ll try to make it useful at the end. 

The Wikipedia page on the coastline paradox is a good way to see it in action:

An example of the coastline paradox. If the coastline of Great Britain is measured using units 100 km (62 mi) long, then the length of the coastline is approximately 2,800 km (1,700 mi). With 50 km (31 mi) units, the total length is approximately 3,400 km (2,100 mi), approximately 600 km (370 mi) longer.

The smaller ruler can match the coastline more precisely, and in the process, you end up with more kilometers. When you make the rulers half as big you need more than twice as many of them. Since the smaller ruler corresponds better with the coastline, it seems like it’s the better one to use, and in most cases it probably is. But our intuition around better measurements is that they give you a smaller error term: as your measuring device gets more and more precise, you converge more and more on to the true figure. That’s not what’s happening here. With a fractal-like object like a coastline, as your measuring device gets smaller, the amount of coastline you find always goes up. You never converge on a “true value”, you just keep finding more.

This is the fractal ratchet: a store of detail that only turns one way, ever-increasing as you look more in depth. It’s like having a couch you raid for loose change, only under the cushions there are also other, smaller cushions with other, smaller coins underneath them. You can always go for more change as long as you’re willing to squint progressively more and more. This is a good thing to know if you are the leader of a fan club for a particular fjord.¹ If someone tries to talk down your fjord, you can always roll up your sleeves and find a bit more coastline. 

So there’s a meaningful mathematical sense in which you could say the length of every coastline is infinite. But also, of course every coastline isn’t infinite, that’s stupid. If you’re used to cleaning the coastline of your local lake every month, you don’t say “Hey, I may as well clean the coastline of Britain next month. Since it’s the same size — infinite — it should take the same amount of time.” The length of every coastline permits infinite contemplation if you look hard enough, but that doesn’t mean the length of every coastline is the same.

Imagine a world where every fjord was, by convention, measured with the same coarse-grained ruler. You love your darling fjord, so you break with tradition and keep measuring with rulers that are finer…and finer…and finer. Each time you get more precise, you work harder, and you find more. No one else cares about their fjords in the same way, so in the societal league table of fjords sorted by coastline, your pet fjord is steadily climbing. As you start lapping other fjords, their respective fanbases (motivated enough to whine, but not to do any more measuring themselves — typical!) will start to grumble. Your fjord can’t really be bigger than ours, they’ll say. You’ll be prepared to defend every inch of your fjord, and that evidence will really be there. You’re right in the way that you can “prove it”. But they’re right in the way that matters, because their initially larger fjords likely would have grown even more if they were observed to the same level of obsession as yours.

For comparing two coastlines, the solution is easy: just make sure you’re using the same size ruler. But the coastline paradox is the most boring example of the fractal ratchet for the same reason it’s the easiest to understand. Ruler length just so happens to be a numeric value, making it easy to compare two things the same way when you want to, and making it very clear what’s happening when you don’t.

Now, though, I want you to broaden your focus a bit. Instead of specifically thinking of “the length of the ruler” as an absolute, quantitative value of your depth of measurement, think “the level of scrutiny” as a vague description of your depth of measurement. You’re still going over your fjord in passes of increasing detail, but focusing on qualitative observations and not just measuring the coastline. First you’re above it in a glider, noting the broad patterns of the rocks and the greenery; now you’re in a kayak, recording every wave and eddy; now you’re walking and peeking in to every hummock and naming each animal you see; now you’re on your hands and knees with a magnifying glass, keeping an eye on every inch of soil, now you’re bringing samples into a microscopy lab to look for novel pathogens.²

This empirical work is a virtuous thing. More detailed qualitative investigations are what we’re all about at Desystemize. Looking at all these different levels is a lot more socially useful than juicing the length metric of a particular fjord. Maybe you’ll make a more detailed tide chart to let people visit your fjord more easily, or discover the new mouse species Fjordicus minimus hiding in the hummock, or notice interesting signal behavior in the fjord’s rhizome networks. All of these will be real findings that are really there, studiously recorded in the annual omnibus The Great Discoveries of the Fjords. Your fjord is topping the charts with everything you’ve discovered. So, when arguing over how the pan-fjord conservation budget is allocated, you say that your fjord is clearly the most valuable to science, the best place for discovery, the most “interesting”. Is that true?

It certainly isn’t necessarily true. You’ve twisted the fractal ratchet quite a few times: glider-scale, kayak-scale, human-scale, magnifying glass-scale. If the fractal ratchet is still glider-scale for the other fjords, then the relative lack of findings elsewhere doesn’t have much to do with the fjords themselves. But it’s not necessarily false, either. It’s not correct to say that all fjords would yield the same amount of insight if investigated at the same resolution, for the same reason you don’t say that all coastlines are the same size. All fjords are infinitely interesting if you look hard enough, but that doesn’t mean all fjords are equally interesting. 

How would you compare the interestingness of different fjords? You’d need to try to look at all of the different fjords at the same level of focus, but this is a much harder prospect than just using the same size ruler, because “human-scale” is a squisher, harder to define thing. Maybe the Great Fjord Auditor walks around every fjord and counts the diversity of the wildlife, but doesn’t squeeze his head into every hummock and so misses sighting Fjordicus minimus. You protest — your fjord is being robbed! But is there even more life in the hummocks on other fjords that weren’t investigated? Is it in-bounds to bring up the stuff you know is there? Unlike the coastlines, where the rankings of the fjords at one scale at least mostly match the rankings at other scales, you never know when twisting the ratchet will reveal an absurdly deep vein of new detail. (Maybe there are a hundred different mice species in those hummocks!)

The real answer here is to make sure you match your ruler to the reason you care. If you’re looking for the biggest mountains, use the glider. If you’re looking for the cutest mice, crouch down and look for them. You don’t need to define your scale if you can define your outcomes. The right scale will arise naturally, and it might not even be one you expected.

But what about “in general”? What about the league table? What about The Great Discoveries of the Fjords? If you’re supposed to match your comparison to the reason you care, how is it supposed to work when you’re trying to find the overall best without tying to a specific reason? Well…the way out of this paradox is to realize that “in general” is a myth we tell schoolchildren and not something that survives this level of scrutiny. Generally comparing things, without that comparison grounding in a specific intervention, is simply a less principled thing to do than digging into the details about those things.

Importantly: less principled, not unprincipled. Just as the coastline of your local pond and the coastline of England aren’t the same size, you are on solid ground to infer that The Great Discoveries of My Local Pond’s Coastline ought to be shorter than The Great Discoveries of England’s Coastline, even though you could always find more to write about both of them. Why? Because your pond is so much smaller, so much less textured, has so much less room to surprise you. It’s common sense. And what this common sense ultimately cashes out in is something like “Okay, for any of the measurements I can easily think of — how many different animals? How many artifacts of human culture? How varied is the terrain? — there’s just obviously a ton more going on in England than the local pond. Maybe one measure taken to its limit case could find a win for the pond, but when I think of all of the potential in all of the measures, there’s just no way.”

This ensemble of measures isn’t itself a measure. You can’t come up with the “coastline score” by averaging the animals, cultures, terrains and whatever, because the fractal ratchet is twisted unequally among its many components, and someone wanting the ensemble to look a certain way can investigate whichever measure helps their agenda best. The tightness of the ratchet can’t be described outside of a specific context, so you can’t formally hold every measure to the same level of detail. What you can do is let your eyes glaze over a bit and try to aim for the level of tightness “I mean, without thinking too hard, broad ballpark” and say “well, I’m not about to come up with any numbers here, but if there were numbers, would one of them be obviously way bigger than the other one?” This technique won’t give you the same objectivity, precision, or usefulness as an actual measure would, but it will let you partially control the inequality of the ratchet. In a world of finite time and attention, we can’t twist every idea as hard as they can possibly can be, and we inevitably need to exercise some degree of imagination to try and balance the scale between what is known and what would have been known had we gone a different way.

Hey, we’re back at the question!

When is it time to go in for discrete, specific knowledge, and when is it time to hang back and listen to common sense? Well, method 1 discrete knowledge works great for measures. Twist that ratchet to your heart's content. That’s what science is! Prove that fatalities reliably decrease when the hands get washed, and now you understand fatalities more; if they try to take you away, tell ‘em their hands are full of death and their minds are full of cope.

But ensembles of measures aren’t themselves measures. You can prove that washing your hands makes you a better doctor, but you can’t prove how much better of a doctor it makes you. It’s impossible to rank the virtue of handwashing against the virtue of, say, a responsive bedside manner or an attention to diagnostic detail, because the ratchet has been tightened differently for each. If someone says that “Okay I’m a bit lax about the handwashing, but my patients love me”, that’s not a question that can be resolved the same way “I can prove you should wash your hands” should be. And yet, less principled is not unprincipled: if their hands are literally covered in shit and blood while they’re delivering a baby, they’re obviously a terrible doctor no matter how good their bedside manner is, and you are fine to assert this even without a score to back it up. Common sense.

This is the magic of the fractal ratchet: after you’ve climbed out of post-rational nihilism, you’re allowed to keep both your domain specific demand for ultimate detail and your domain-agnostic inclination to reject monomaniacal theories that insist maximizing some given measure value is the One True Way. You can say things like: “Look, no matter how much you show me data that the benefits of patience when listening overrides the state of your hands, if you were serious about being a doctor you would wash your poopy hands. I know this precisely because the benefit is so obvious at a quick glance, and I don’t need to prove it numerically. It’s not possible to pit the two interventions against each other, because it’s not possible to look at each of them exactly as hard. But this one is so easy, and so obviously good, that the lack of a principled way to score interventions doesn’t stop you from having an unacceptably low score.”

Be rational when you are talking about something specific and be reasonable when you are talking about something vague. Assert that free humans are objectively happier than slaves even as you assert that human happiness can’t be objectively measured. If you’re trying to solve a specific problem, keep worrying away at it until it’s fixed and bask in the dignity and purpose of rationality. But if you’re trying to understand a human principle, or a potential future, or a broad ecosystem, then over-tightening one lone measure will lead you to a mistaken conclusion, even if every turn of the ratchet uncovered something real. Study inflation when it’s helpful, but you don’t need to wait for an equally mathematical alternative to notice when it’s not. The apparent tension in these viewpoints is only the tension in your arm, grasping tighter and tighter all the time. Let go of the ratchet, and try to see everything at once. The details will be waiting for you when you’ve figured out your question.

1. For a while I was a stan of the Lysefjord in Norway. It’s one of the top tourist destinations in Norway and — I thought — the origin of my last name, dropping the e and the j. Then I learned that no, we’re from the actual Lysfjord, one letter closer but dramatically smaller. Bummer!

2. If you had to do all those different levels of analysis for your fjord, you probably wouldn’t actually start with the glider. You’d start with the walk, that’s easier. In this extended example, I’m sorting from biggest to smallest because that makes the analogy with the coastline paradox much more clear. But generally, you’d sort from easiest to hardest, jumping something like walk to magnifying glass to kayak to microscope to glider rather than consistently from big to small. And why is the walking scale easier? Because that’s the most in tune with the natural senses and capabilities we possess already. Why are our senses at that level of detail? Because variations at this level of detail are often the ones most immediately applicable to our survival, and natural selection listened to that! So there is a reason to believe that easiest to hardest will often start us with the most fruitful human-level of detail up front. But this is not a certain fact and it’s quite a rabbit hole, so I’ll just hide this idea in a footnote and step away.