data mining

Big Data … Why?

If you need a lot of data to spot a useful pattern, you have neglected the small data you had before and you have waited too long. Besides, there is no truth in Big Data, that isn’t already visible in a small data sample.

Imagine if Johannes Kepler had waited to this day, to induce his laws from all the solar and extrasolar planetary behavior and all the other debris that we have since found in the sky. We wouldn’t have known of Uranus, let alone of Pluto and there would be almost no new data, except for that of Tycho Brahe, that Kepler used anyway.

The essence of science (or any other data analyzing) is to do something from almost nothing. And then check this something against the Big Data as well. The previously spotted regularities should hold.

They say that if you have never missed a bus, you are spending too much time on the bus stations. But it is even worse, if you’re still waiting for a Big Bus, to go to the station for the first time.

So, if you have some Data, chop it to smaller chunks if it’s too big and then analyze it like mad. You may test your findings against the Big Data later.