Duplicate Record Management

Overview

When two or more records exist for the same person, household, or organization, the records should be merged so that each person/household/organization is represented by a single record in Veracross. The system automatically creates “dupe groups” to help with duplicate record management. Read more about deduping categories and the exceptions surrounding them here.

Records can be merged in two ways: first, by flagging them for merging, which is covered in this article, and second, by using the built-in record merge feature in Axiom. Read more about merging records in Axiom. This article also covers the basic logic used by the system to determine which record’s information to keep when merging.

NOTE: There is no way for Veracross clients to administratively “un-merge” records. If records have been merged in error, the only way to reverse the process is for the client to manually create the records that have been lost, or under certain circumstances, and typically for extra cost, Veracross may be able to restore the records.

Flagging Records for Merging

Flagging records for review via the “Duplicate People” reports

When you find any two person records that should be merged, in a query Result use the “Dupe Group” field, and set it to an identical value for each record that you want to merge. Tip: Set the value as “00.USER.{value}” for BOTH the two records. For instance, if there’s two records representing “John Smith” that should be merged, enter the value “00.USER.JohnSmith” into the Dupe Group field for both John Smith records. By doing so, these dupes will later show in your “Duplicate people” report, making it really easy for you (or for some other staff at your school) to actually perform the merge.

Setting records for automated merging

Persons, households, and organizations can be flagged for automated merging after your school’s overnight processes have been enabled. Determine which record to keep by reviewing each record to see which has the most current data. Note that the nightly scripts take the “Auto Merge” system parameters into account.

1. Note the IDs for the record to keep and the record to delete.
2. Open the record for the duplicate record that should be deleted and navigate to the “Other” tab. Note: the tab will look different for each type of record (person/household/organization).
3. Locate the Surviving Record field.
4. Click the magnifying glass icon  to search for the record that you want to keep. When you choose the surviving record, the name of the person/household/organization to keep will appear in the “Surviving Record” field.
5. Click Update .

No further action is required; Veracross will merge these flagged records within 48 hours assuming (a) overnight processes have been enabled and (b) the auto-merge parameters are configured to allow it, deleting the old record and updating the surviving record. See below for logic on which information is kept.

Preventing Records From Merging

To protect a record from being merged during the deduping process, enter the Surviving Record to be itself. The record will be removed from the dupe group in the overnight process, or by forcing the deduping list to refresh using the “Find Duplicate…” procedure on the System homepage. This will also remove the record from its Dupe Group on the Potential Duplicates reports.

For example, if there are two John Smiths who are indeed different people and so should not be merged, select John Smith (#1) as the surviving record on John Smith’s (#1) record and select John Smith (#2) as the surviving record on John Smith’s (#2) record. Then either force the dupe group to refresh by running the “Find Duplicate People” procedure on the System homepage, or wait for the overnight process to run. The two John Smiths will not be merged together and will be removed from any person dupe groups.

Information Retained When Merging

When the merge process runs, the system applies logic to to determine which information is retained. Note: the term “surviving record” is used to indicate the record that is kept.

• Broadly speaking, when there is conflicting data, the survivor’s data will be retained.
  • Example: Suppose there are two organizations, “Apple, Inc.” and “Apple, Incorporated.” If they are merged, whichever record is selected as the survivor will be the name of the surviving organization.
• Broadly speaking, when there isn’t conflicting data, existing data will be retained.
  • Example: Suppose two person records are being merged, only one of which (doesn’t matter which one) contains a “preferred name.” That preferred name will be retained in the surviving record.
• In the case of phone numbers and email addresses, data from the more recent record is retained in the merge, based on the assumption that a more recent email or phone number is probably more accurate.
• Name prefixes and suffixes retain the “higher” value.
  • Example: “Reverend” would be retained over “Mr.” in the surviving record.
• In the case of admissions data, the more recent record is retained, except for notes (see next point).
• Admission and person notes are combined when merged.
• Emails are moved from the victim record to the survivng record only if the victim record is greater than 14 days old.
• Not all information entered in an inquiry form is considered admissions data — for instance, a birthdate, while entered on an inquiry form, is not considered admissions data. This statement more refers to data such as a student’s Grade Applying For value.
• Address information is retained uses the following logic:
  • When merging households’ primary addresses, the separate fields comprising the address (Address 1, Address 2, City, State, Postal Code) will not be combined. Addresses are a concrete entity and partial addresses will not be merged with other partial addresses. Note that this does not apply to the two secondary addresses.
  • When merging organizations’ addresses, the separate fields comprising the address will not be combined, unless the surviving address has no address information, in which case the old address will populate on the surviving record.

Note: Some of the above logic operates differently when a school is implementation mode. For instance, the selected surviving (not most recent) record for contact information will be kept, since presumably the school is selecting the intended record to keep when doing the initial “clean up” of the data. Schools should talk to their Account Manager to clarify steps of the deduping process during implementation.

Find Duplicate… Action Menu Items

On the Action menu of the System homepage are three procedures, “Find Duplicate Households / People / Organizations.” Click to force refresh the dupe group category and dupe group number on records.

It is worth noting that if the insert or last update date on a Person or Household record is over 45 days of being flagged, the system will automatically unflag the record as a duplicate. Organizations will be unflagged after 90 days. If you do not want the system to automatically unflag duplicate records by erasing the dupe group from records that haven’t been touched in 45/90 days, you can set the "Find All Duplicates (Not Only Recently Updated)"  system parameter to 1 and it will prevent the system from erasing the dupe groups.