I blame not the faith in metrics to produce results, but the faith in the Ivy Leagues to produce graduates who are worth hiring as consultants about anything despite having no experience at anything. The great dysfunction here is the religious nature of the Ivy Leagues, which are the cathedrals of crypto-Calvinist puritanism.
The plural of anecdote is not data, because the anecdotes have always been gathered by people trying to prove one point or another. For instance, we have a vast multitude of anecdotes about faith healings, but no data AFAIK.
I guess both sides of "The plural of anecdote is data" can kind of be misleading depending on your frame of thought when you encounter it. Probably the most-precise way to say it would be something like "Data ought to be the plural of anecdote"; in that the point of data is to accurately describe a wide set of lived experience. But basically the whole thesis of Desystemize is how often that doesn't happen, so flatly asserting "the plural of anecdote is data" is maybe a little risk, hah.
Spent some time thinking about this and I think that the different between anomaly detection and counterfactuals is something like the difference between a scalar and a vector. Because when an anomaly happens, it will cause some sort of different outcome, right? But **how much** anomaly leads to **how much** different outcome? No reason to assume linearity and impossible to say "all else being held equal" when the biggest message an anomaly tells you is "the present is not necessarily in continuity with the past".
Or maybe directionality is a better metaphor for it. It's easy to say that {anomalous situation} leads to {some state}. Counterfactual analysis is dividing {anomalous situation} into {some measurable part of anomalous situation} + {all unmeasured qualities that made anomalous situation come about}. Then you ask, if you perform {intervention that leads measurable part to new value} but still have {all unmeasured qualities that made anomalous situation come about}, could you have avoided reaching {some state}? That's the sort of inference that can't be definitively proven and is more an art than a science - but a lot, a lot, a lot of modern management theory essentially assumes that interventions and states all have clean linear relationships.
Re. "how much your failure actually meant in material terms requires the sort of counterfactual analysis (“What if everything in the world was the same that day except for the average time to ICU bed?”) that’s extremely difficult to do over aggregated numbers.": This is what partial differential equations were invented to do, and they do it very well, without counterfactuals.
Sorry for suddenly appearing on your blog only to disagree with you. I tend not to comment on the parts I agree with, which I know is a fault.
I blame not the faith in metrics to produce results, but the faith in the Ivy Leagues to produce graduates who are worth hiring as consultants about anything despite having no experience at anything. The great dysfunction here is the religious nature of the Ivy Leagues, which are the cathedrals of crypto-Calvinist puritanism.
The plural of anecdote is not data, because the anecdotes have always been gathered by people trying to prove one point or another. For instance, we have a vast multitude of anecdotes about faith healings, but no data AFAIK.
I guess both sides of "The plural of anecdote is data" can kind of be misleading depending on your frame of thought when you encounter it. Probably the most-precise way to say it would be something like "Data ought to be the plural of anecdote"; in that the point of data is to accurately describe a wide set of lived experience. But basically the whole thesis of Desystemize is how often that doesn't happen, so flatly asserting "the plural of anecdote is data" is maybe a little risk, hah.
Is this whole concept of the "counterfactual" just a philosophical version of anomaly detection? https://www.wikiwand.com/en/Anomaly_detection
Spent some time thinking about this and I think that the different between anomaly detection and counterfactuals is something like the difference between a scalar and a vector. Because when an anomaly happens, it will cause some sort of different outcome, right? But **how much** anomaly leads to **how much** different outcome? No reason to assume linearity and impossible to say "all else being held equal" when the biggest message an anomaly tells you is "the present is not necessarily in continuity with the past".
Or maybe directionality is a better metaphor for it. It's easy to say that {anomalous situation} leads to {some state}. Counterfactual analysis is dividing {anomalous situation} into {some measurable part of anomalous situation} + {all unmeasured qualities that made anomalous situation come about}. Then you ask, if you perform {intervention that leads measurable part to new value} but still have {all unmeasured qualities that made anomalous situation come about}, could you have avoided reaching {some state}? That's the sort of inference that can't be definitively proven and is more an art than a science - but a lot, a lot, a lot of modern management theory essentially assumes that interventions and states all have clean linear relationships.
How about, "Data is a set of anecdotes plus the methodology used to collect it"?
"Data is anecdotes plus filters/aggreagators." Also thanks for the free lunch. https://rogersbacon.substack.com/p/eponymous-laws-part-1-laws-of-the
Have you seen this article? It broke down those type of people as systemizers, utilitarians, and entropy injectors (chaos inducers).
https://think-boundless.com/dark-side-strategy-consulting/
Also on "impact" vs "replicability" as borderline antonyms in science (thanks Mr. Kirkegaard):
https://kirkegaard.substack.com/p/higher-impact-factor-journals-have
Re. "how much your failure actually meant in material terms requires the sort of counterfactual analysis (“What if everything in the world was the same that day except for the average time to ICU bed?”) that’s extremely difficult to do over aggregated numbers.": This is what partial differential equations were invented to do, and they do it very well, without counterfactuals.
But I suppose the data needed to construct a system of partial differential equations isn't available for nations or large organizations.