MINING is an arduous, time-consuming business. Sometimes, tons of material must be excavated to uncover ounces of precious metals or gems. The computational equivalent of old-fashioned, down-in-the-dirt mining is data mining. Whether the search is for metals or information, the task is similar. In data mining, trillions of bytes of data must be sifted to find a handful of precious numbers or images.![]() ![]() ![]() |
Preprocessing Techniques
Sampling: Selecting a subset of data items. Sampling is a widely accepted technique to reduce the size of the data set and make it easier to handle. However, in some cases, such as when looking for something that appears infrequently in the set, sampling may not be viable.
Multiresolution analysis: Another technique to reduce the size of the data set. With multiresolution analysis, data at a fine resolution can be "coarsened," which shrinks the data set by removing some of the detail. By preserving the detail, the transformation can be reversed.
Denoising: A technique that removes "noise" in images or data. It can be used to sharpen a fuzzy picture or aid in character recognition (differentiating a "6" from a "b" in text, for instance).
Feature extraction: A technique to extract relevant features from the raw data set. In credit card fraud, for instance, an important feature might be the location where a card is used. Thus, if a credit card is suddenly used in a country where it's never been used before, fraudulent use seems likely. Dimension reduction: Reducing the number of features used to mine data, so only the features best at discriminating among the data items are retained.
Normalization: A technique used to level the playing field when looking at features that widely vary in size as a result of the units selected for representation.
Data Mining Step by Step![]() ![]() ![]() ![]() |
Mining bent Doubles![]() ![]() ![]() ![]() |
Gems Uncovered![]() ![]() ![]() —Ann Parker |
Key Words: Accelerated Strategic Computing Initiative (ASCI), data mining, Faint Images of the Radio Sky at Twenty Centimeters (FIRST), pattern recognition, Sapphire.
For more information contact Chandrika Kamath (925) 423-3768 (kamath2@llnl.gov).
More information about Sapphire can be found at http://www.llnl.gov/casc/sapphire/sapphire_home.html.