Basically, we measure it. We have to constantly work at reducing bias
which is always present, especially when trying to show support
for some closely held theory. The more a person or organization is tied
to a belief/understanding, the harder it is to eliminate bias and the
more we, as information consumers, should demand it. Humans are not good
at data gathering yet this is the crux
of science. And science is our most successful tool, by a long shot, for
gaining reliable knowledge. But it may require enormous efforts to reduce bias, especially when
dealing with behaviors, but even others areas are difficult.
Another major tenet of reliable knowledge is verification. Can
someone else readily get the same result?
This is why on my reviews I measure everything that I can so it's
less subjective, especially speed. I LOVE it when others either validate
my findings or point out errors. For example, I have a formula for
converting speeds listed by my measurement to pilots of different
weights. That makes a prediction that can be tested and, if the results
aren't accurate, then the formula should be tossed. So far, it has
worked quite well.
Looking around at what works and how that's come to be is quite
telling. In short, we know by measuring. Our most reliable knowledge
comes from evidence. But what constitutes good evidence? Something that
can be measured! If you can't measure it, then reliability decreases.
Stories and experiences are great beginnings but to really know
anything, we have to measure it.
Knowledge Comes In Degrees. The worst type of knowledge, the least reliable, and the most likely
to be uncovered by experiment. Further, we need verification to improve
certainty. Evidence is king and, until there's good evidence for any
assertion, it should be considered suspect. Intuition is terrible, too.
When I first got into paramotoring it only made sense that having the
gas tank on top would be more dangerous--an explosion waiting to happen.
Guess what? When we measure the accidents where fire was involved, it's
just the opposite. That's why we must measure!
Another example is saying that Avgas runs cooler. OK, maybe, but did
you control for other possibilities? When you tried the Avgas how long
did you run it before measuring? What was the elevation? What was the
outside temperature, etc. Same with thrust tests. These are quite valid
for comparison between motors but not for absolute data. I've actually
run this experiment and found it pretty inconclusive, with results not
far off of the error bars of the test. Which brings us to another
important aspect of reliable knowledge: replication. Others should be
able to replicate the experiment with similar results. Otherwise the
conclusion is questionable. Of course not all experiments are created
equal. A well designed and implemented experiment should hold lots more
sway than a poorly implemented experiment. You have to read how they did
it.
There's a lot more to gaining reliable knowledge and I've been
inspired by
the following article.
So lets be careful about our claims and try to always rely on a
sound, rational appraisal of available evidence. We know in degrees but
we should always be striving for the highest degree.
The 15 minute "Baloney Detection Kit"
Here is a great little video that talks about how science ferrets out
reliable truth from our observed world.
Which oil is best? Start with something that makes
logical sense and go from there. But without testing, the knowledge is
less certain. You may be right that brand X is better for our
application but there are others who say brand Y is better. Well
controlled testing is the best way to find out.
