Contact your data providers and instruct them to send test files through for enhancement. Measure the turnaround time and customer service, as well as the documentation provided with the returned data and your company contacts' level of knowledge. This is especially important if you are looking to establish a long-term contract with a data provider.
When the results arrive, send the enhanced files to a phone-survey company for verification of the results (accuracy testing). One thousand completed surveys should be sufficient for any test. You can count on 15-20 attempted surveys for every completed survey at a cost of about a dollar per survey attempt. Thus, for a 1,000 record file a cost of $15,000-$20,000 is normal.
Below is an example test for a fictional company to further illustrate the process.
Example Case: HomeBank
The following example for HomeBank should provide you with a good foundation for designing your own data test. Notice throughout the case that HomeBank follows the proper testing procedures and defines its data needs before testing.
Company Information: HomeBank is a regional bank based in Dallas, Texas. Like most banks, it issues credit cards in addition to offering standard banking services such as home loans, checking accounts and investment options. HomeBank would like to enhance its current customer list for internal analysis and target market upscale services to its prospect list.
Current Data: HomeBank currently has about three million records including prospects and current customers. It is interested in obtaining data about its current customers' household information, wealth indicators, real property information, education levels, automotive information and some lifestyle segmentations. It will use this data for segmentation6 and to create scores7 for its prospect file.
Data Suppliers: HomeBank analysts have identified three companies that can provide all of the elements described above. To ensure no biases in the tests, they are known simply as Company A, Company B and Company C.
Data Needs: HomeBank has identified the elements it desires along with acceptable ranges. For relative importance, a weighting system has been devised. A "3" represents data that HomeBank absolutely must have for its models and marketing. A "2" represents data that is very important to have but will not preclude a data provider from being used by HomeBank if it fails to supply the element. A "1" represents data that would be nice to have but is not imperative. A "0" is given for companies who do not provide any data for an element. HomeBank's testing framework is shown in Figure 1.
Testing Procedures
Following the testing procedures outlined above, HomeBank has created a fair and valid test as follows:
1. 100,000 records were chosen at random from the 3 million records on file using a random number generator and following the steps listed in this paper.
2. HomeBank sent the file to each of the three data suppliers for enhancement.
3. After the files were enhanced, HomeBank analysts judged each supplier on customer service parameters and called each with problems to judge responsiveness. The results of these separate tests were ranked as listed on the previous page (pg. 8) and compiled into the final test results.
4. To test accuracy, HomeBank used an outside survey company. Calls were made at random until 1,000 completed surveys were received.
The results from the tests are listed and ranked on the next two pages. The total scores represent the sum of the weighted scores multiplied by the rankings. The largest total score reflects the overall highest ranked data supplier.
See Figures Two and Three below
Interpreting The Results
First, HomeBank reviewed all of the data returned to see if each company met all the required elements.
