Which oil is best? Start with something that makes logical sense and go from there. But without testing, the knowledge is less certain. You may be right that brand X is better for our application but there are others who say brand Y is better. Well controlled testing is the best way to find out.
So how do we know anything? Really KNOW?
First, we realize that all knowledge comes in degrees of certainty. The highest certainty comes from using the best evidence. That’s number one for reliable knowledge: quality evidence.
The best evidence is replicable measurements. If it can’t be replicated, it’s not very reliable. If you can’t measure it, we don’t KNOW it. If you can’t measure a claim, the claim does not constitute reliable knowledge. Observations must give measurable results. Initial observations and speculation are great tools–the start of understanding–but they’re just that, a start.
A major tenet of reliable knowledge is verification. Does someone *ELSE* readily get the same result? If not, reliability plummets.
Working against reliable knowledge is bias which is always present, especially when trying to show support for some closely held theory. The more a person or organization is tied to a belief/understanding, the harder it is to eliminate bias and the more we, as information consumers, should be leery of their claims. Humans are not good at data gathering yet this is the crux of science. And science is our most successful tool, by a long shot, for gaining reliable knowledge. Science is, by definition, the set of tools and processes that best uncover reliable knowledge. It requires enormous efforts to reduce bias, especially when dealing with behaviors.
This is why on my reviews I measure everything that I can so it’s less subjective, especially speed. I LOVE it when others either validate my findings or point out errors. For example, I have a formula for converting speeds listed by my measurement to pilots of different weights. That makes a testable prediction and, if the results aren’t accurate, then the formula should be tossed. So far, it has worked quite well.
Looking around at what works and how that’s come to be is quite telling. In short, we know by measuring. Our most reliable knowledge comes from evidence. But what constitutes good evidence? Something that can be measured! If you can’t measure it, then reliability decreases.
Stories and experiences are great beginnings but to really know anything, we have to measure it.
Knowledge Comes In Degrees. The worst type of knowledge, the least reliable, and the most likely to be uncovered by experiment. Further, we need verification to improve certainty. Evidence is king and, until there’s good evidence for any assertion, it should be considered suspect. Intuition is terrible, too. When I first got into paramotoring it only made sense that having the gas tank on top would be more dangerous–an explosion waiting to happen. Guess what? When we measure the accidents where fire was involved, it’s just the opposite. That’s why we must measure!
Another example is saying that Avgas runs cooler. OK, maybe, but did you control for other possibilities? When you tried the Avgas how long did you run it before measuring? What was the elevation? What was the outside temperature, etc. Same with thrust tests. These are quite valid for comparison between motors but not for absolute data. I’ve actually run this experiment and found it pretty inconclusive, with results not far off of the error bars of the test. Which brings us to another important aspect of reliable knowledge: replication. Others should be able to replicate the experiment with similar results. Otherwise the conclusion is questionable. Of course not all experiments are created equal. A well designed and implemented experiment should hold lots more sway than a poorly implemented experiment. You have to read how they did it.
There’s a lot more to gaining reliable knowledge and I’ve been inspired by the following article.
So lets be careful about our claims and try to always rely on a sound, rational appraisal of available evidence. We know in degrees but we should always be striving for the highest degree.