Best Practices for Creating and Maintaining a Clean Database

An Experian QAS White Paper

Best Practices for Creating and Maintaining a Clean Database

It’s not just about having a database; it’s about what you do with itA contact database is an essential component of effective sales and marketing initiatives. Failure to establish a process and institute a consistent approach to maintaining contact data quality can negate the overall usefulness of your database. Further, the costs associated with bad data can significantly diminish the value of your database. It is essential to have processes in place that enable your organization to maintain high-quality data. Even the largest and most expensive databases will be virtually useless if the contact information contained within it is inaccurate or incomplete.

Six steps for cleansing and maintaining a databaseA continuous, systematic approach to maintaining a database requires

consistent follow up. The following 6-step process is designed to help you better understand your database and how to keep it clean.

Step 1: Understand Your Data Learning what is in your database will provide you with better insight into the condition of your data. It’s important to be aware of your data quality processes in order to better understand how contact information is used downstream.

Step 2: Clean Existing Data Start with a data cleansing project that will provide you with the greatest benefit. Starting with a “clean slate” for future data quality initiatives is most effective.

Step 3: Remove Duplicate Records Removing duplicate records is an important step in the cleansing

process that provides a better sense of your true customer base.

Step 4: Enhance and Update Data Understand the complete view of your customer or prospect with enhanced data. This step will allow you to fine tune your message for your target audience.

Step 5: Verify Data at Capture Points Reduce the need for “after the fact” cleansing by capturing accurate information at the point of entry. Look for tools that will help you ensure that data is accurate, valid and standardized before the information ever enters your database.

Step 6: Enhance, Update, and Learn Since data is constantly changing, implement a continuous process that is performed regularly to clean, augment and update data.

Data cleansing and maintenance are critical to a successful contact data management strategy. An uncleansed, poorly maintained database can rob your business of profits, limit communication efforts and damage ROI. Storing bad data inhibits your ability to market and cross-sell products and delays the bill collection process. Accurate street addresses, telephone numbers, and emails are critical components of customer communications. A cleansed and enhanced contact database eliminates unnecessary mailings, cuts expensive printing and postage costs, and improves call center and back-office staff efficiency. Ultimately, ensuring the accuracy of contact data improves customer and prospect relationships- a positive result for any type of organization.

Step 1: Understand Your DataThe first step in cleaning and maintaining your contact data is to truly understand what information is contained in the database, as well as how that information is currently maintained. There are several key questions that should be asked during this process;

1. Questions related to data entry: Where does the data come from? Who enters it? Is it entered by customers or by staff in a call center or other department? Are they motivated to enter accurate data? Do they understand the cost of improper entries and incomplete records?

2. Questions related to what data is stored and how: What type of technology do you use to store your customer data records? How is data formatted? Do you use Excel files, a SQL Server database, or another format? What fields are required to constitute a full record?

3. Questions around data quality processes: Do you have controls in place to ensure the quality of records being entered into the database? Does your organization provide data entry training to ensure employees understand the importance of accurately entered data? Do you audit newly entered information?

As you learn more about your data acquisition and entry practices, it may be helpful to organize the data flow in a diagram such as Figure 1 below.

Figure 1. Typical contact data inputs to a multi-channel retailer database and the business processes impacted by the data

The illustration uses a multi-channel retailer model to demonstrate how records may be entered into a database and how contact information is used. This method of mapping data inputs and outputs can also be applied to other industries. The left side of the diagram shows data sources that enter contact information into the retailer’s database. These sources include the online, call center, mail or fax and point-of-sale channels. The right side of the diagram illustrates how data is used. For example, fulfillment of an order is complete when a package is delivered to the correct address. Poor contact data can impact the successful fulfillment of an order. The success of marketing campaigns depends on marketers’ ability to

understand their customers’ needs and reach them with relevant offers. Customer service departments may use customer data to send correspondence and order-related documentation.

Understanding the Challenges of Data CaptureAccording to Experian QAS research, corporate databases double in size every six months. This exponential growth can be attributed to numerous data capture points such as the Internet, call centers, and mailings. The high number of potential input channels increases the likelihood of data errors due to inaccurate record entry. Understanding where bad data originates will help organizations to modify current processes and flag and correct bad data up front.

Web/OnlineIt is a common misconception that information entered into a database by customers is accurate and complete because the customer has directly provided the information. In fact, close to 20% of information entered online is incorrect or incomplete and requires cleansing. Individuals entering information online are frequently distracted while they enter data, and typos and missing information are common.

Call CenterCall centers are controlled environments where representatives are trained in collecting and entering

information. A challenge inherent with call centers is the misinterpretation of what a caller is saying, perhaps because the representative is unfamiliar with the caller’s accent. This can be particularly challenging in an international organization where callers may be dialing in from a variety of countries. “Fat fingering,” or pressing more than one key at a time, can also introduce problems as a representative may inadvertently press a key next to the intended keystroke. Finally, call center representatives are typically short on time, adding to the challenges around ensuring high levels of accuracy.

Mail/Fax (Manual Entry)Challenges with mailing and faxing can stem from poor handwriting, typing and/or incomplete information. This raises the likelihood of misinterpretation by the recipient of the forms, who may ultimately enter details into the database system.

StoresThe challenge inherent in stores (or branches) is that this channel includes face-to-face interaction in a high-pressure environment. At times, a customer may be rushed to provide information or they may be unwilling to provide contact details. A sales associate may be impatient and incorrectly enter contact details due to lack of time or understanding about the importance of collecting accurate data.

Organizations should go through this exercise of identifying and mapping sources of contact data, thinking through potential areas contributing errors, and evaluating downstream business impacts of inaccurate and incomplete contact information. Once this exercise is completed, Step 2 follows.

Step 2: Clean Existing DataOnce you have an understanding of the sources of the records in your database, how errors are entered and how poor data quality affects your business processes, you are ready to begin cleansing existing data. This task is typically accomplished within organizations by Database Analysts or IT Analysts.

Review Your DataA key step in this process is the review of existing records and patterns. Manually reviewing records will help to uncover data inconsistencies. This provides a better sense of what your data actually looks like.

Audit CompletenessAn audit can determine whether fields need to be filled in. When appending data, you should pay careful attention to where and when fields may be incomplete. Re-evaluating your required fields helps define how you will deal with incomplete information in the future. Clean and StandardizeLeverage a 3rd party data source to clean and standardize your data. A host of companies exist that can perform this task and verify the records in your database. It is recommended that you speak with an industry expert to learn more about the data with which your data is being compared. This ensures that the 3rd party vendor is using the best quality data available.

Don’t Underestimate AddressesOne of the most valuable pieces of data you have is a customer address. Just as important as email address and telephone number, the address record is the core of customer contact data. Physical addresses can be matched against postal authorities to ensure that they are deliverable and accurate.

Step 3: Remove Duplicate RecordsIn the de-duplication process, duplicate records are merged, leaving only one copy of the record to be stored, along with references to the unique copy of data. Duplicates occur when there is no structured or controlled process for monitoring records entered into your database. Records are commonly entered through multiple distribution channels or through sales channels. Without proper controls on the front end, eventually merging records into one source is necessary. Records

merged into a centralized master file provide a singular view of the data, allowing you to better understand, segment and communicate with your customers. When embarking on a de-duplication project, consider the following options.

Decide what elements to match onMatching elements determine where duplicates can be found. Think about the level of matching you would like to accomplish, as well as the tolerance level for what is considered a duplicate record in your organization.Use a tool with fuzzy or flex matching. “Fuzzy matching” refers to matches that may be less than 100% perfect when identifying correspondences between segments of a text and entries in a database. This allows for a greater match rate and helps to locate duplicate records that may not have been identified with a rigid or exact matching process. Some examples of de-duplication are shown below:

Figure 2 – Examples of “fuzzy matching” Step 4: Enhance and Update DataData enhancements quite simply complement the information that your organization has already been able to gather. Enhanced data allows you to segment your database, stand out from spammers and provide targeted messaging.

The first way that organizations can enhance data is to perform NCOALink® (National Change of Address) processing. The USPS® maintains nearly four years of Change of Address (COA) information. According to the USPS®, over 43 million permanent COA orders are processed each year. This correlates to approximately 15% of Americans and 19% of businesses moving annually. Internal studies conducted at Experian QAS indicate that moving customers are most likely not reaching out to change address information, so it is important to perform NCOALink® processing regularly. Moreover, NCOALink® processing is a requirement if the mailer is looking to qualify for postal discounts.

Data can also be enhanced by appending contact information with geo-demographic information, like latitude and longitude coordinates, since street-level geocoding in the United States became available in

Match Type ExampleAddress/Household Physical + Name or EmailPhonetic Dougherty = DortyAcronym National Broadcasting Company = NBCCharacter Occurrence Wilson = WislonTable-based William = Bill = Will = BillyElement Matching Mr. J. Smith = John Smith = Smith JohnCustom Field Shoe Size, SSN, Customer Number

the early 1990’s. This method provides specific socio-demographic information and a view of who your customers are and where they are located. By segmenting your database, you can increase your ability to target relevant customers.

Step 5: Verify Data during Capture ProcessesYou can improve the overall quality of your database by verifying each record as it is entered at every capture point, rather than randomly cleansing the entire database. An important part of this process is to understand the types of records that are being entered and where they are being collected. Whether it’s originating through a point-of-sale system, a web portal, a call center or paper forms, all data should be verified on the front end.

Set expectations by having a standardized process in place so that all information is being captured in the same way. Sometimes this can be accomplished by using system rules. For example, at Experian QAS, when an address is entered into the database, it is verified in real time using the company’s front-end address verification tool. It may also be useful to implement a training program to help your staff understand the value of verifying data correctly at all capture points, and tie performance metrics to the level of accuracy of the data captured.The illustrations below demonstrate the effects of verifying data at various capture points.

Figure 3 – Illustration of data in/out of a database with no front-end verification

The figure above shows the impact of records entered into a database from a call center, a data entry operator and through a website. Note that there is no verification prior to entry and no verification from within the database. The resulting data used by marketing, billing and shipping is of poor quality.

Figure 4 – Illustration of data in/out of a database with some front-end verification

Figure 4 shows the impact of records entered into a database from a call center, a data entry operator and through a website. In this diagram, the records entered by a data entry operator are verified prior to entry into the database and also within the database. The resulting data used by marketing, billing and shipping has increased in quality.

Figure 5 – Illustration of data in/out of a database with front-end verification at all points of entry

The figure above shows that the records entered by a data entry operator, call center and website are verified prior to entry into the database and also within the database. The resulting data used by marketing, billing and shipping is “cleansed” and much more efficient.

Step 6: Continue to Enhance and Clean

Once there are processes in place to enhance and cleanse data, your focus can be shifted to regular “checkups” and occasional snapshots with more focus on maintenance, enhancement, and appending and/or refreshing older data. More of your time will be spent gaining knowledge and making business decisions based on your data. It is important to consider the use of performance metrics to monitor the health of your database on a regular basis. Sharing internal data quality reports that grade the accuracy of recently entered records will raise awareness among stakeholders, driving overall accountability of your data initiatives.

Continuous database maintenance allows for a better assessment of the quality of data after each strategy is implemented, and also ensures that older data is refreshed and continues to perform well in future initiatives.

SummaryMost organizations do not have a complete data quality process in place to create and maintain a clean database. To ensure that the database is in fact useful, an organization should familiarize all stakeholders with data capture best practices and take the necessary steps to ensure data accuracy before entering any new information. It is important to proactively check new elements in a database, as well as to continuously review and enhance existing data with updates and appends. Organizations in industries such as higher education, retail, insurance, financial services and government all

Experian QAS125 Summer St Ste 1910Boston, MA 02110-1615

T: 1 888 322 6201info@qas.com

www.qas.com

© 2011 QAS Limited.

QAS Limited. Registered in England. No. 2582055.Talbot House,Talbot Street, Nottingham NG801TH.

The words ‘Experian’ and ‘QAS’ are registered trade marks in the EU and other countries and are owned by

Experian Limited and/or its associated companies.

All rights reserved.

Experian QAS is QAS Ltd and exists in our own right.

leverage contact data for various purposes. Ultimately, all organizations that collect customer data with the intention of driving business value will benefit from a systematic and well-thought-out strategy of database cleansing and maintenance.

Experian QAS Products and ServicesExperian QAS provides software and services to capture, validate, cleanse, standardize and enrich customer contact information. We are dedicated to helping our 11,000 customers worldwide improve the quality of their databases. For more information visit our website at www.qas.com or call us at 1-888-322-6201.