Predictive Analytics in the Real World
After centuries of scientific progress in understanding the structural and periodic patterns of the natural world, as well as our success in developing so many useful technologies that leverage these patterns, most humans are reasonably confident that our future will continue to unfold in a similar fashion. The expectation that we will continue to find and utilize patterns, not only in the natural world but also in the realm of human and man-made system behavior, underlies the current enthusiasm for predictive analytics.
Amazon, Google, and Netflix have clearly demonstrated that predictions based on past user behavior can be profitable; fueling speculation that organizations of all types might benefit by uncovering the hidden patterns in their own data stores, and using these patterns to inform beneficial changes in their processes and systems.
Predictive applications can be used not only to find patterns that will increase sales and improve customer service, but also to improve the performance of complex systems. For example, even tiny changes to automated production processes, or catching anomalies before they turn into major malfunctions, can boost performance and profitability. In short, it just makes sense to think that hidden patterns in the data that are being collected about our behavior, and about the behavior of our devices and systems, will be useful in predicting, influencing, and even changing, future behavior.
The concept is so compelling, and the data so voluminous, that a burgeoning array of new database and analytical software tools are coming to market, joining the existing statistical and decision modeling solutions, to advance the process of prediction. Some of these new tools are based on well-worn statistical paradigms. Others feature algorithms developed in highly specialized realms of scientific research; while still others are being brought into commercial application, from research, into various types of machine learning.
Some of the newest predictive analytic solutions offer bundles of algorithms and automated mechanisms (more algorithms), to evaluate their performance, and to identify the ones that best fit the available data and the desired predictions. These algorithms owe their ability to work efficiently and rapidly to ongoing improvements in the software development frameworks that enable them, and in the ways that compute resources are scheduled and optimized.