IBM Watson and the Dawn of the Cognitive Era
December 6, 2016
Much like we’ve seen from marketing organizations over time, IBM is attaching the word Watson, their primary “cognitive” brand, to just about everything data-related now. As you may recall, Watson was the name given to the computing complex that dominated the Jeopardy game show a few years back. Watson was a compilation of software components, running on a robust Power-based hardware platform that processed natural language and produced high-probability answers very quickly (presented in the form of a question, of course).
Watson’s various components are now commercially available from IBM in a number of different forms. The sales pitch is pretty straightforward—add cognitive to everything that you are doing. Sounds easy. But it is not. In order to get any answers from your data, you have to know the questions to ask. That’s where Data Science comes in.
There are distinct steps in the process of getting to the question. I’ll examine four of them here:
- Exploratory Data Analysis
Exploratory data analysis includes unsupervised learning (turn the machine loose and see if anything interesting pops up), clustering (group the data into bunches that seem to make sense), and basic data summaries (the Business Intelligence sort of stuff we’ve been doing for years). The goal is to get familiar with the data and arrive at one or more hypotheses to be tested later.
Any data set is only a sample. Typically you are attempting to infer what is in the sample to apply it to some larger population. So now you need to take what was found in the exploratory step, apply some of that statistics stuff you forgot about shortly after college, and think about what you might generalize from the sample.
Prediction takes inference a bit further and attempts to guess an outcome. Now the machine learning can kick in (you are starting to know what to look for), along with some other science-ey stuff, to challenge the algorithms (questions) being developed.
- Experimental Design
Experimental design is the process of bringing everything learned in the previous steps together into a form that can be tested and confirmed in a strictly controlled fashion. Is the model producing results that match historical performance? Are there other confirmation mechanisms that will confirm its viability?
After all of that, you finally have the question (actually the entire analysis model) to be applied to any data set going forward. Tools in the Watson bag of tricks have made this process easier and shorter, but it still is a process.
I’ve been impressed with how IBM is marketing the idea of cognitive and the Watson brand. In fact, the stuffed dog that randomly lurks around our office has acquired the name.
The message is getting out.
Submitted by Randy Dufault
Director of Solution Development