How do I improve my data quality

This article tells you how to clean, inspect, and benchmark data.

Clean Data:

Inconsistent or incorrect data leads to false conclusions. Just like in the for-profit world, in not-for-profit organizations, incorrect data can be costly. Many organizations use customer information databases that record data like client and contact information, addresses, and preferences. If the addresses are inconsistent, the organization could potentially experience higher costs because of having to resend mail or in a worst-case scenario, they could even potentially lose clients.

Remember the old saying “Garbage in equals Garbage out”? Well, the better the data you enter into the system and have, the better results you will get. Therefore, the way in which data is cleaned and how well someone understands the data they have, has a high impact on the quality of the results. The following provides some guidelines on how to work with your data to ensure the data you are working with is in best “shape” as possible.

To ensure you have “clean” data, when you enter data, your data must…

• conform to your business rules (for example, if a field requires a number, you shouldn’t be entering text),
• be accurate (for example, instead of saying someone’s mailing address is Rochester, NY, you should be more precise and add the street),
• be complete (for example, if data is missing, it should be entered),
• be consistent (for example, information you enter on different pages shouldn’t contradict each other or be entered on some and missing on others),
• and be uniform (for example, data should all be the same unit…like all weights should be in fractional pounds.

A good practice to follow to clean the data already in your imPowr instance, is as follows:

• Inspect your data to look for unexpected, incorrect, and inconsistent data.
• Clean your data to fix or remove the issues which were found.
• Recheck your data after cleaning, to verify you have corrected the issues
• Benchmark your data by reporting the changes made and the quality of the data stored, so you now have a starting point of when your data was at its best.

To help you improve data quality, imPowr offers you some tools to help you inspect and clean your data…

• Login to your imPowr website.
• Click on Data Cleanup in the Data section of the Performance menu:

Inspecting your Existing Data:

In the Data Cleanup page, there are numerous options for cleaning data:

• Select from one of the various data cleansing methods by selecting an Account Data Check from the drop-down list. The checks are self-explanatory, such as accounts which have no account type, or accounts who share an email address with other accounts, etc.

• If you choose the Manual Options: Manual search (use dropdowns on filter), after clicking on the refresh button, select the missing items to filter the displayed list on, from the drop-down list of Missing items:

Until you select the Manual Search option, the list of possible missing items does not display on the screen.

• Other kinds of searches can also be done by selecting Dates of Birth (DOB), Account Types, Case Managers, Program, or Added By from their drop-down lists:

• And lastly, you can optionally choose to hide duplicates from your results, when specific items are unique:

• All of the above can be selected independently or in combination with other selections.

Once you have finished making your search selections, click on the Refresh button to display your results:

Your results will be displayed:

Now that you have completed your inspection of this Account Data type, there are various things you can do to clean your data.

Cleaning Your Data:

There are a multitude of data cleaning methods to choose from (and new ones are added all the time).

Cleaning your data can be done manually, by looking through each entry and adding in the missing data or correcting spellings or inaccurate data.

When your cleaning involves looking for information which is in one account and is similar to information in another account, looking at each one independently and then choosing which to keep and which to get rid of, could be a challenge…especially if there are more than 2 entries involved. To help you with this task, imPowr offers a feature to allow you to compare entries and merge them into a single entry.

Optionally, to get a more detailed view of the entries, click on the Related () icon, found on the very right-hand side of one of the entries:

A more detailed comparison will be displayed:

By selecting from the various Related Account checkboxes at the top of the screen, you can view the data in each of those columns:

• From here, you can either click on Compare & Merge or Close to return to the previous screen. Select the entries you wish to compare, by checking the checkbox to the left of the entry and clicking on the Compare & Merge:

Or

• A Compare & Merge screen will be displayed:

Note the account header which indicates which account will be kept and which will be removed.

Note: If you would rather the 2nd is the account to keep, select that account and click on Swap.

That will cause the 2nd account to be the one which will be retained.

• Select the items (via the radio buttons) that you wish to keep from each account by clicking on the radio button.

Example:

• Select the Merge Options which best apply:

Also note the items which will automatically be merged:

• When you are ready, click on Merge to merge your accounts into one account:

If there is an Account Type mismatch, you will receive a warning, asking you if you wish to proceed:

If you wish to stop, click on Cancel. This will bring you back to the previous screen.

If you wish to proceed, click on Proceed.

Your accounts will be merged (one account will be retained with the merged data and the other will be eliminated from the instance), and you will be brought to a display of the appropriate profile (if that was selected as an option).

• Click on Save.

Note: in some cases, you will need to manually clean the data. Depending on the Account Data Check selected, you may need to enter in missing data or correct inaccurate data.

For example: Clients with no date of birth:

For these, you will want to click on each entry and enter in the missing data.

Re-inspecting Your Data:

Be sure to rerun your Account Data Checks both immediately after cleaning your data (to make sure you captured the issues), and on occasion to keep your data “clean”.

The more data checks you run, the more you run different types of checks, and the more things you do to clean your data, the better your data quality will be and the more reliable will your results be.

Benchmark Your Data:

… as a starting point for the next time.

As they say in this age of the coronavirus pandemic… It is better to guard yourself against a disease instead of spending the time and effort to remedy it. Similarly, investing the time upfront, when you enter your data, will minimize the time you spend cleaning it up later on!