| Tikalon Blog is now in archive mode.
An easily printed and saved version of this article, and a link
to a directory of all articles, can be found below: |
|
This article |
| Directory of all articles |
Science as Data Compression
August 2, 2011
There's an old joke about
timepieces. It's so old that it refers to
analog timepieces, called
wristwatches, that were once worn before everyone carried a
cellphone with built-in clock. Not only that, the joke mentions an action, called "winding," that's probably unknown to the younger generation. The joke is understandable today only if it's rephrased for a digital clock, as follows.
Someone notices that a man's clock is flashing 12:00, and asks why he doesn't set it. He replies that the clock always ran fast, and this way it keeps better time. Perplexed, the other person asks how that could be. The man answers, "Before, the clock never showed the right time. Now it's correct twice each day."
I was reminded of this joke when I read something similar in the draft of a very interesting book by
Daniel Burfoot posted on the
arXiv Preprint Server.[1] Burfoot has a background in
mathematics,
physics and
computer science. The book, entitled "Notes on a New Philosophy of Empirical Science," contains the following
parable.
A physics professor is challenged by a student who claims that he can predict the outcome of physics
experiments by communication with the spirit world. Not only that, he claims that he can make better predictions than she can using her
Newtonian mechanics. As a test of their abilities, they agree to predict the time a projectile will remain in the air when launched by a spring device.
The professor minds her
forces and
angles, and she announces her prediction of two seconds to the class. The student communicates with the spirit world and writes his prediction on a piece of paper. The experiment is performed, and measurement indicates 2.134 seconds. The student claims victory, since his prediction, when revealed, is 1< t < 30. His prediction is correct, and the professor's is wrong by 0.132 seconds.
Daniel Burfoot, author of "Notes on a New Philosophy of Empirical Science", a draft of which is on the arXiv Preprint Server.[1])
The idea of Burfoot's book is that theories are
compression algorithms. They reduce vast amounts of data to an equivalent, but smaller form. Armed with this idea, he considers a refined (but perhaps less efficient) version of the scientific method that proceeds as follows:[2]
• Assemble a large database of measurements of a phenomenon.
• Attempt to develop a theory of the phenomenon, or revise a previous theory.
• Instantiate the new theory as a compression program.
• Invoke the compressor on the database.
• The new theory bests a previous one if it reduces the data to a smaller file size when the length of the compressor program is included.
Every dataset will have at least one "zeroth-order" theory, which would be a
Huffman coding of the data; but a real theory, of the Newtonian mechanics variety, will have a compression ratio many orders of magnitude better. The similarity of this approach to
algorithmic information theory, as championed by
Gregory Chaitin, is apparent.
Francis Bacon.
Bacon was an early proponent of what is now called the scientific method, although his techniques were more empirical than analytical. That's why he's called the father of empiricism, and not the father of the scientific method.
Bacon died of pneumonia while studying the preservation of meat by freezing (chicken, not bacon).
(Via Wikimedia Commons))
One advantage of Burfoot's approach is that
Occam's Razor is an explicit feature. His main argument is that theories developed in this fashion are no different than those developed by physicists; that is, you have a bunch of data, and you reduce it to an equation.
One other feature of Burfoot's method is that it sidesteps the
demarcation problem. This problem, which has been argued for more than a century, is where to draw the line between
scientific problems and other problems. You just apply the method, and get your result. The process makes no distinction as to the suitability of the subject matter, and it can be applied to a wider range of data than the classical scientific method. Quoting from the arXiv abstract,[1]
"In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can compress a standard benchmark database to a small size, taking into account the length of the compressor itself."
In a private communication, Burfoot emphasized that the compression approach is highly effective at debunking
pseudoscience, since pseudoscientific theories cannot achieve compression. In the physics experiment of the professor and her student, we see that the professor's classical mechanics approach will compress the dataset of all projectile problems much better than the student's.
Burfoot also distinguishes the equation F = ma from the hypothesis that generates the equation. Purely mathematical statements cannot achieve compression, but the
hypotheses that they encapsulate can.
One interesting idea, called the veridical simulation principle, is that a good decompressor will generate sensible data from a
random input. The decompressor complement of a good compressor for
English text will generate grammatical, meaningful English sentences when fed random data.
Returning to the topic of wristwatches, I haven't worn a wristwatch in about twenty-five years, but there was an interesting article written by an
engineer about wristwatches when they were still popular.[3] He wore a wristwatch, and whenever he needed to know the time, he would look around to see whether there was another clock in the vicinity. He found that this was nearly always the case, so he concluded he didn't need a wristwatch. He also admitted that he still wore one.
References:
- Daniel Burfoot, "Notes on a New Philosophy of Empirical Science," arXiv Preprint Server, April 28, 2011
- Research page on Dan Burfoot's web site.
- You can read the sentence either way, as meaning when engineers were still popular, or when wristwatches were still popular. Sorry, no reference to the original article.
Permanent Link to this article
Linked Keywords: Timepiece; analog; wristwatch; cellphone; Daniel Burfoot; arXiv Preprint Server; mathematics; physics; computer science; parable; experiment; Newtonian mechanics; force; angle; compression algorithm; Huffman coding; algorithmic information theory; Gregory Chaitin; Francis Bacon; Wikimedia Commons; Occam's Razor; demarcation problem; scientific; pseudoscience; hypotheses; random; English; engineer; Notes on a New Philosophy of Empirical Science.