In Dynamics 365 we capture data and data can easily become duplicate for multiple reasons such as bulk record import.

Microsoft Dynamics 365 provides OOB merge functionality which is quite helpful in deduplication of the records and cleaning up the data.

Merging in Dynamics 365

In Dynamics 365 we can merge Account, Contact, Leads or Incidents.

Merging process in Dynamics 365 takes two records of same the entity, the Master record and the Subordinate record, and you specify which is the master record out of two.

On merging the subordinate record gets deactivated and linked to the master record any related records from subordinate record moves to the master record.

Let’s explore more about the merging process and extend the merge tracking in Dynamics 365

In this blog we will cover:

• Merge process in Dynamic 365 for accounts, contacts and leads.
• Merge process in Dynamics 365 for incidents.
• Merge triggers in Dynamics 365.
  • Manual selection of records in grid
  • Duplicate detection rule configuration
  • Calling MergeRequet SDK Message
  • Calling Web API Merge Action
• The new enhanced merging experience and options in Dynamics 365.
  • New options
    • Merge records by choosing fields with data
    • View fields with conflicting data
    • Enable Parent Check
    • Select all fields in this section
• Limitations of merging in Dynamics 365.
• Hidden merge tracking fields in Dynamics 365.
• Extending merge tracking functionality in Dynamics 365.
  • Create a system view to show Master and Subordinate record in a view.
  • Create visualization for merged records.
  • Record merged fields and subordinate record reference on master record for tracking.
• Security considerations for Merging in Dynamics 365.

Merge process in Dynamic 365 for account, contacts and leads.

When you choose to merge Accounts, Contacts or Leads you are presented with following screen.

On Merging screen you specify which is the Master (primary) record, and the other one becomes the Subordinate record. Optionally you can specify any subordinate records fields to override the master records fields, by default all the master records fields will be selected.

On the merge screen you are also presented with following options which we will discuss later in this blog.

• Merge records by choosing fields with data
• View fields with conflicting data
• Enable Parent Check
• Select all fields in this section

When you have chosen the master record and fields, you can click on Ok to merge.

Merge process in Dynamics 365 for incidents.

Merging process for Incidents is a bit different from Account, Contact and Leads.

For merging incidents you notice few differences such as:

• You have to select mater record from the grid to which other record will merge into.
• You don’t get option to choose fields from the subordinate record to move onto the master record.
• There are no new option as they are available for account, contact and lead merging.

Different merge triggers in Dynamics 365

There are multiple triggers to initiate merging process such as:

• Manual selection of records in grid
• Duplicate detection rule configuration
• Calling MergeRequet SDK Message
• Calling Web API Merge Action

Manual selection of records in grid

In manual merging you select two records in grid and then click on Merge button to launch merge screen.

Duplicate detection rule configuration

We can set up duplicate detection rule, which will launch the merge screen if duplicate records are detected as per the rule.

Calling MergeRequet SDK Message

You can merge records programmatically by calling MergeRequet message available in the SDK.

The following example in C# shows how we can merge two record using MergeRequest message.

// Create the target for the request.
              var target = new EntityReference();
// Id is the GUID of the account that is being merged into.
      // LogicalName is the type of the entity being merged to, as a string
             target.Id = _account1Id;
             target.LogicalName = Account.EntityLogicalName;
// Create the request.
      var merge = new MergeRequest();
      // SubordinateId is the GUID of the account merging.
      merge.SubordinateId = _account2Id;
      merge.Target = target;
      merge.PerformParentingChecks = false;
Console.WriteLine("\nMerging account2 into account1 and adding " + "\"test\" as Address 1 Line 1");
// Create another account to hold new data to merge into the entity.
      // If you use the subordinate account object, its data will be merged.
      var updateContent = new Account();
      updateContent.Address1_Line1 = "test";
// Set the content you want updated on the merged account
      merge.UpdateContent = updateContent;
// Execute the request.
      var merged = (MergeResponse)svc.Execute(merge);

Note: UpdateContent property will not be applicable to incidents and will be ignored.

Calling Merge Action using Web API

The following example in TypeScript shows how we can call the Merge Action available in the Web API to merge records.

    export async function ContactMerge() {
        debugger;
        const targetId: string= "71a17064-1ae7-e611-80f4-e0071b661f01"; // replace with target Id
        const subordinateId: string= "73a17064-1ae7-e611-80f4-e0071b661f01"; // replace with subordinate Id
        const contactMergeRequest: any = {};
        contactMergeRequest.Target = {
            entityType: "contact",
            id: targetId
        }
        contactMergeRequest.Subordinate = {
            entityType: "contact",
            id: subordinateId
        }
        contactMergeRequest.UpdateContent = {
            jobtitle: "{Upated Job title}",
             "@odata.type": "Microsoft.Dynamics.CRM.contact"
        }
        contactMergeRequest.PerformParentingChecks = false;
        contactMergeRequest.getMetadata = function () {
            return {
                boundParameter: null,
                parameterTypes: {
                    "Target": {
                        "typeName": "mscrm.contact",
                        "structuralProperty": 5
                    },
                    "Subordinate": {
                        "typeName": "mscrm.contact",
                        "structuralProperty": 5
                    },
                    "PerformParentingChecks":{
                        "typeName": "Edm.Boolean",
                        "structuralProperty": 1
                    },
                    "UpdateContent":{
                        "typeName": "mscrm.contact",
                        "structuralProperty": 5
                    }
                },
                operationType: 0,
                operationName: "Merge"
            }
        }
        const response = await Xrm.WebApi.online.execute(contactMergeRequest)
        console.log(response);
    }

The new enhanced merging experience and options in Dynamics 365.

New Merge screen

The merge screen provides few new options, and provides opportunity to choose fields from master or subordinate records which will be saved finally on master record.

When you click Ok, merging begins.

and on completion success message appears as following.

The selected records are merged and subordinate record is deactivated.

As you can see the new enhanced merging experience provides four new options as following:

• Merge records by choosing fields with data
• View fields with conflicting data 
• Select all fields with conflicting data
• Enable Parent Check

Let’s discuss each of them.

Merge records by choosing fields with data

If you select this option, fields from master or subordinate which has the data in same field is selected, if both the records have the data in same field then master record field is selected.

View fields with conflicting data 

If you select this option then only fields with different data will be shown. Fields with same data will be hidden.

Select all fields in this section

If you select this, then all the fields under this section will get selected. This is helpful if you have many fields under a section to select.

Enable Parent Check

This is interesting.

If this option is checked and the records have different parent Accounts, then on merging following error will be thrown.

Error

Unable to merge because sub-entity will be parented differently. You can disable the parent check prior to execution as part of Merge dialog.

Limitations of merge functionality in Dynamics 365.

In Microsoft Dynamics 365 merging feature is great but has some limitations too.

Let’s check some of the important merging limitations you should be aware of.

• Merging is available only for Account, Contact and Lead system entities.
• Mering is not available for custom entities yet, there are some suggestion though to Microsoft, to support merging on custom entities.
• Once records are merged you don’t know which data was merged from subordinate to master record, unless you have enabled auditing on master, secondary and related entities.

Hidden merge tracking fields in Dynamics 365.

After merging, subordinate record shows the notifications similar to the following.

The record was merged with {record name}, and the deactivated.

The notification on the subordinate record provides a link to the master record.

But on master record there is no such link to subordinate record.

So there’s no quick way to navigate to subordinate record from the master record.

How merged notification is displayed on subordinate record?

Entities contain two hidden fields for merge tracking as shown in following screenshot:

• Merged: Boolean, Shows whether the account has been merged with a master contact.
• Master ID: Lookup, Unique identifier of the master record.

For internal merge tracking Dynamics 365 updates values of these two fields on subordinate record. These fields are then used to display notification on the subordinate record.

Please note, these fields will not be updated on the master record and because merge is not available on other entities you will not find these fields on entities other than Account, Contact, Lead and Case.

When merge happens, fields on subordinate record will be updated:

Merged: Will be set to true, to indicate this record has been merged with another record.

Master ID: Will be assigned reference to the master record.

Querying merged records

Let’s try now to find the merged records from the advanced find.

As you can see these fields are not available in advanced find, so we cannot filter records to find merged records through advanced find.

These fields are not available even through Add Columns in view, so we cannot see the fields them in grid.

We know if fields are not available through advance find, they may have searchable option set to false in field editor. Let’s check that.

Enable Searchable option on fields

Searchable option on fields is used to show/hide fields from advanced find, if searchable is false then field will not appear in advanced find.

Merged field, searchable is disabled in the classic field editor.

MasterI ID field, searchable is disabled in the classic field editor.

Let’s check through Power Apps portal, as some features are available there.

Merged field

Merged field is editable so checked and saved.

Master ID field

Master ID field is also enabled, so checked and saved.

In Power Apps portal both the fields were editable, so now we should be able to query in advanced find. Let’s check again in advanced find.

Hmmmm….

Merged and MasterId fields are still not available in advanced find, why?

Because Dataverse didn’t saved those fields. Let’s verify.

So, that means we cannot find merged records using views with filter on merged field.

Visualisations options

Since we don’t have views with merged field filter we cannot create visualization for the merged records.

Let’s try to add fields on Form.

In classic editor

We cannot find merged and masteid in classic form editor.

let’s check the modern editor.

You can see fields are not available even in modern forms to add.

Querying Dataverse for Merged and MasterId

Let’s try now to query Dataverse.

Query by FetchXML

Let’s see if we can query the Dataverse using following FetchXML.

FetchXML is filtering records based on merged field.

<fetch top="50" >
  <entity name="contact" >
    <attribute name="fullname" />
    <attribute name="masterid" />
    <attribute name="merged" />
    <filter>
      <condition attribute="merged" operator="eq" value="1" />
    </filter>
  </entity>
</fetch>

Result

Yes, we are able to query through FetchXML, and we can see

Merged is true for subordinate record.

MasterId is set to the Id of the master record.

Note, above result is showing subordinate record only and not the master record. That means merged field is true only on subordinate records.

If you need to find master record, you can do joins on masterid field.

Query by Web API

Let’s try the Web API now.

Xrm.WebApi.online.retrieveMultipleRecords("contact", "?$select=fullname,_masterid_value,merged&$filter=merged eq true").then(
    function success(results) {
        for (var i = 0; i < results.entities.length; i++) {
            var fullname = results.entities[i]["fullname"];
            var _masterid_value = results.entities[i]["_masterid_value"];
            var _masterid_value_formatted = results.entities[i]["_masterid_value@OData.Community.Display.V1.FormattedValue"];
            var _masterid_value_lookuplogicalname = results.entities[i]["_masterid_value@Microsoft.Dynamics.CRM.lookuplogicalname"];
            var merged = results.entities[i]["merged"];
            var merged_formatted = results.entities[i]["merged@OData.Community.Display.V1.FormattedValue"];
        }
    },
    function(error) {
        Xrm.Utility.alertDialog(error.message);
    }
);

Web API Query Result

So we are able to get the results with Web API as well.

Extending merge tracking functionality in Dynamics 365

So far we have learned OOB merge behaviour and some of the limitations of merging in Dynamics 365, let’s try to extend the merging functionality using some customizations.

We will try the following to extend the merging usability:

• Create view for Merged Contacts by updating view FetchXML and Layout XML.
• Create visualizations for merged contacts.
• Adding fields on form by updating FormXML.
• Subordinate Lookup and Change content tracking on master Contact record.
  • Add custom fields on Contact record.
  • Write plugin on merge to populate Subordinate lookup field on master record.

Create view for Merged Contacts by updating view FetchXML and Layout XML

As we don’t have OOB way of creating view for merged records, let’s try to create by editing FetchXML.

• Create a View for example Merged Contacts in a solution and export as the unmanaged solution.
• extract the files and update the FetchXML filter in the customizations.xml file as following.

              <fetch version="1.0" output-format="xml-platform" mapping="logical">
                <entity name="contact">
                  <attribute name="fullname" />
                  <attribute name="masterid" />
                  <attribute name="contactid" />
                  <filter>
                    <condition attribute="merged" operator="eq" value="1" />
                  </filter>
                </entity>
              </fetch>

• Update Layout XML too to add masterid cell as following.

            <layoutxml>
              <grid name="resultset" jump="fullname" select="1" icon="1" preview="1">
                <row name="result" id="contactid">
                  <cell name="fullname" width="200" />
                  <cell name="masterid" width="200" />
                </row>
              </grid>
            </layoutxml>

• Zip the files again.
• Import the solution back in environment and publish.

And you will see the view with merged records.

From view we can observe the following:

• We have a view now showing merged records, because of our applied filter in view fetchxml.
• Both Subordinate and Master Lookups are available in same view which enables us to find which records were merged together.

Activating inactive subordinate records

Let’s try to activate the inactive subordinate record in above view and let’s see what happens.

Subordinate record is activated again, let’s go to Merged view.

And the activated record has gone from the view.

Subordinate record is removed from the Merged Contacts view because it’s no longer Subordinate and has following effects on Merged and MasterId fields on activation.

• Merged field: It is reset back to false.
• MasterId field: This field is set to null.

Visualizations for merged contacts

Because we have Merged Contacts view available now, we can create nice visualization using that.

Adding fields on form by updating FormXML

I was not able to successfully display Merged and MasterId on forms, If you are able to please share.

Subordinate Lookup and Update Content tracking on master Contact record

As we have already seen subordinate record contains reference to the master record in MasterId field, but there is no reference on the Master record for the Subordinate record.

Let’s do the custom implementation for this.

Add Custom fields on Contact

I have added following two fields on contact, you may try as per your need:

• Subordinate: Contact Lookup for Subordinate reference.
• Merge Update Content: String Field to store selected fields content from Subordinate record, to track what values were migrated from subordinate record to the master record.

Write plugin on merge to populate Subordinate field on Master record.

Write and Register following plugin on Post Operation of Merge Message.

using Microsoft.Xrm.Sdk;
using System;
using System.Collections.Generic;
namespace SureshMaurya.Merging.Plugins
{
    public class PostMergeOperation : IPlugin
    {
        public void Execute(IServiceProvider serviceProvider)
        {
            var context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
            var tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
            var organizationServiceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
            var organizationService = organizationServiceFactory.CreateOrganizationService(context.UserId);
            var primaryEntityReference = ((EntityReference)context.InputParameters["Target"]);
            var subordinateId = (Guid)context.InputParameters["SubordinateId"];
            var updateContentEntity = (Entity)context.InputParameters["UpdateContent"];
            tracingService.Trace($"Primary Record Id {primaryEntityReference.Id}");
            tracingService.Trace($"Subordinate Record Id {subordinateId}");
            UpdateContentOnPrimaryRecord(organizationService, primaryEntityReference, subordinateId, updateContentEntity);
        }
        private void UpdateContentOnPrimaryRecord(IOrganizationService organizationService, EntityReference primaryEntityReference, Guid subordinateId, Entity updateContentEntity)
        {
            //Prepare Update Content String
            List<string> updateContentCollection = new List<string>();
            foreach (var attribute in updateContentEntity.Attributes)
            {
                updateContentCollection.Add($"{attribute.Key}: {GetAttributeValue(updateContentEntity, attribute.Key)}");
            }
            var updateContentString = string.Join("\n", updateContentCollection);
            //Update Primary Record with Update Content
            Entity entity = new Entity(primaryEntityReference.LogicalName, primaryEntityReference.Id);
            entity["msdc_subordinate"] = new EntityReference() { Id = subordinateId, LogicalName = primaryEntityReference.LogicalName};
            entity["msdc_mergeupdatecontent"] = updateContentString;
            organizationService.Update(entity);
        }
        private string GetAttributeValue(Entity entity, string key)
        {
            if (entity.Attributes.ContainsKey(key))
            {
                if (entity.Attributes[key] is EntityReference)
                {
                    return $"{{Id: {((EntityReference)entity[key]).Id}, Name: {{{((EntityReference)entity[key]).Name}}}}}"; ;
                }
                if (entity.Attributes[key] is OptionSetValue)
                {
                    return ((OptionSetValue)entity[key]).Value.ToString();
                }
                //Handle more field data types.
                return entity[key] as string;
            }
            return string.Empty;
        }
    }
}

When successfully plugin registered try to merge two contact records and select few fields from the subordinate records.

After merging open contact record and observe Subordinate and Merge update Content field. Should be similar to following.

Subordinate Lookup contains reference to the Subordinate record.

Merge Update Content field contains what field data is moved to master from the subordinate record.

Security considerations for Merging in Dynamics 365.

Merging in Dynamics 365 is very powerful but there might be unintended consequences, there are a few security consideration you should keep in mind while merging as explained in Microsoft docs:

• Scenario #1: Merge records that are owned by users
• Scenario #2: Merge records that are shared to users
• Scenario #3: Merge records that are shared to access team members
• Scenario #4: Merge records that are owned by teams

Summary

In this blog we learned a lot about merging in Microsoft Dynamics 365. We started with what merging is in Dynamics 365 and what are the different triggers for merging. Then we explored the new enhanced experience for mering in Dynamics 365.

Merging is tracked by two hidden fields in Dynamics 365, we explored what are those two fields and how we can surface then into a view so that those fields are easily accessible and facilitates in creating the visualisation on top of that.

We went ahead and wrote a plugin to keep track of merged record and fields onto the master record.

I hope you liked this blog post about merging and how we can leverage the hidden features of merging, please share your thoughts or anything to improve.