The Art and Science of Business-to-Business Merge/Purge
DMA Business-to-Business Journal, Summer 1998 issue

By Joe Balaban,
Vice President Business Development,
Anchor Computer, Inc.

Business-to-Business data presents many challenges to the data processor. With the right planning, knowledge of the marketplace and the data you are dealing with it can become less of a challenge. Although this is titled merge purge, the techniques described are equally applicable when consolidating business data for a database.

This article presents you with quality control and formatting techniques that should be part of any business to business processing. It applies to data sources, matching techniques and points out what you should be aware of as a marketer to business or as a processor of business data.

• Know your data sources – The more knowledgeable you are about how your input data is constructed the more accurate you can be when reformatting and standardizing the data prior to matching or consolidation. If you are drawing on internal sources demand consistency, if in doubt about format or meaning of data ask the person who owns it what it means. Making the wrong assumption can cause delays, increase the cost and have disastrous results on your decision process. I.e., all data expressions should follow the same scheme; all addressing should contain consistent elements and whenever possible follow United States Postal Service guidelines.
   
• When you find consistent data entry or formatting errors, take corrective action, inform the source and requests that they put corrective measures in place.
   
• Dump and examine samples from the beginning, middle and end of the input file run frequency distribution reports on all critical data elements to make certain that you receive what you expect. I.e., titles appear as expected, in the appropriate fields, address elements are correct, SKU information is on the file and in the correct format and contact names (if required) appear on the file. Be particularly alert to the potential for errors when moving numeric fields. Different sources frequently will have varying character lengths or justification. Phone numbers should be checked by verifying with a few phone calls or by comparing them to a reference file.
   
• Use United States Postal Service CASS certified processing to standardize addresses.
   
• Use NCOA prior to merge purge, many mailers are afraid to use NCOA for Business addresses because the file contained invalid moves in the past. The United States Postal Service has put a number of checks in place to minimize the occurrence of this. Keep in mind that you do not have to apply all NCOA address changes and you still get the benefit of address standardization that will improve matching and delivery!
   
• If you have recent customer activity, you most likely do not want to apply NCOA but when dealing with older prospect records it makes sense to apply the new address. If the address change is from street to PO Box address, you probably do not want to make the change. When applying address changes, populate them back to the source so you do not have to change them next time you use the same source.
   
• Use appropriate edit tables as a basic screen for all data elements before matching or consolidating data. I.e. ZIP code, city name, product codes, acceptable dollar ranges.
   
• Convert titles to title codes (or add them to the record), this will make it easier to run counts, make selections for output and analyze response. You should still address the individual by full title!
   
• The key is to establish a balance between data processing requirements, costs and customer requirements.
   
• Cross match fields, especially company names vs. individual name. Many times they look alike and can help identify duplicates that might not appear the same if you are only matching on a field by field basis. i.e. Jersey, Smith & Balaban Assoc., F. Smith esq., 123 Main Street . . . match to (Blank company name), J. Balaban, 123 Main Street . . .
• This is a must if you market to doctors or lawyers as these are frequent occurrences with professionals.
   
• Use an SCF match as many businesses have PO Box and Street address that will have different ZIP Codes, but generally are in the same SCF.
   
• Carry both street and PO Box address for matching and mailing but make sure that you also carry a correct ZIP code for each address. Most businesses have at least one of each address type. The United States Postal Service prefers that you defer to PO Box, but your customer may prefer that you use the street address.
   
• If the output is also going to populate a sales or contact database you need to have the street address since your salespeople can’t make calls on PO Boxes!
   
• DO NOT use telephone number to create the match as they can be reassigned in high-density areas within a few weeks or even days in the case of a move. Most businesses have multiple numbers and you have the issue of switchboard numbers vs. direct dial numbers so individuals at a site may have different numbers than those listed for their companies. You can use them as a tiebreaker or as part of the match logic if other elements are equal.
   
• When applying phone numbers to your database from different files call a few to make certain that the correct numeric fields are captured as many 10-digit field look like phone numbers.
   
• Build table of acceptable variances based on your data and industry. If input varies from the norm, question it! I.e. number of PO Box addresses, titles that may be inappropriate for your business, indicators of a government agency.
   
• Consider using DSF² to identify business vs. consumer addresses on prospect files and drop them. You can determine the value of this by back testing against previous efforts.
   
• Most important make certain that the software you use in-house or service bureau you use if outsourcing your business to business processing has the flexibility to handle your business to business data.
   

These are not a panacea nor are they a substitute for your knowledge of your market and marketplace data. However, if you apply these procedures to your in-house processing or choose a service bureau that uses these procedures you will be rewarded with the benefits from a more accurate identification of duplicates.

Back to Feature Articles