Measure your Assumptions (the Popper way)
Growing Meta #11
Hi folks! Here’s an important idea in the philosophy of science:
Karl Popper thinks researchers should ask questions that can be falsified.
He called this ‘falsifiability.’
For example, if someone said “all ants are poisonous”, you can prove that it’s false if you show “an ant that is not poisonous”. The statement is falsifiable.
Popper says ideas like Freud’s psychoanalysis are not scientific theory (i.e. they are pseudoscience), because you cannot refute them. The nature of this theory does not allow for experimentation or logical criticism. Psychoanalysis is not falsifiable, therefore it is not science.
In 2018, when I was doing my master’s thesis on deep learning models, my hypothesis was along the lines of ‘the model will do better on noise between different objects (like trucks and dogs), because it is resistant to learning different feature representations’. I could easily falsify that by finding evidence that it performs against my assumption.
Use falsifiability as a measure of the quality of your assumption.
I think the notion of falsifiability is a powerful logical check. I think this really applies for product managers, marketers, and anyone working with assumptions. List the clues that can render your assumption invalid, and then begin your research.
This is Popper’s iconic book:
“The Logic of Scientific Discovery”, 1935
Here are some of the main ideas:
How science is done:
You state a hypothesis, then test it (using logic or tangible evidence).
You collect relevant reproducible evidence (if you were an experimenter not a theorist).
The logic of doing science:
Scientific discovery is not inductive logic. In inductive logic, you move from one statement to a “universal truth.” In science, you discover “singular statements”.
To find “inductive inferences”, you need to have a “principle of induction”. This principle has to be falsifiable, amongst other things.
What good science is (according to Popper):
A theory can be tested in four ways:
consistency of internal conclusions,
conformity to the nature of scientific theory,
comparison with known theories or
the validity of its applications. When testing a theory, you can always produce a temporary positive result.
You should look for negative results that refute the hypothesis to show its “falsifiability.”
“Degree of Falsifiability”
I like this bit:
Because theories are not ‘true’ nor ‘false’, good scientists say that a theory is ‘corroborated’ or that they find it ‘not corroborated’ according to current discoveries - instead of implying that there is absolute truth.
This is the elegant, right way of expressing your thoughts on a theory.
There is No End in Sight
Popper also thinks that:
Science “can never claim to have attained truth, or even a substitute for it, such as probability.”
The Case Against Falsifiability
Popper’s thoughts are popular, but there are a lot of arguments against them:
Thomas Kuhn (science historian/philosopher) says that Popper fails to explain how the social and informal process works. Falsifiability isn’t practical, as a theory can never give us absolute truth. Scientists do not discover science this way, they work according to ‘paradigms’.
We don’t need frameworks or methods, science is better off with free-thinkers. - Paul Feyerabend
There are a few more, but Kuhn was the primary ‘opponent’ of Popper’s notions.
(Popper is the one in maroon.)
View the full comic here. : )
We deal with hypotheses in our work.
I think marketers, product managers, and entrepreneurs will find ‘falsifiability’ a useful criteria when thinking about their assumptions.