How and what to check when your emails are not being deleted or archived properly in Exchange Online

I thought about writing this article about how to troubleshoot MRM/Archive issues in Exchange Online, as we often see in support, situations like these. 

IMPORTANT:  Before I go ahead and expand on this subject, please keep in mind that all the information found here, is based on my own troubleshooting experience and nothing else.

IMPORTANT 2: If you are troubleshooting archiving issues in a hybrid deployment, where the primary user mailbox is located on premises and the archive is in Exchange Online, you can still apply the same troubleshooting steps from below, but just do this on the on premise Exchange Management Shell, since it is the on premise server applying the archive policies and hosting the primary user mailbox.

To start my article, I will begin by noting down the possible causes of MRM/Archive not working/slow processing for a mailbox:

-        RetentionHoldEnabled attribute on the mailbox is set to True – property on the output of Get-Mailbox command

-        ElcProcessingDisabled is set to True - property on the output of Get-Mailbox command

-        We have a retention tag applied on the mailbox that is disabled (never move to archive” or “never delete” actions are applied)

-        The mailbox to be processed has a lot of items and has a big size or in the case of archiving, the Dumpster of the primary mailbox has a big size. Items in this case will be processed slower.

-        The policy applied on the mailbox contains only personal tags, which if they are not manually applied by the user, will result in a behavior where MRM doesn't seem to process the mailbox.

Keep in mind that the Managed Folder Assistant has an SLA in Exchange Online of up to 7 days until it will process a given mailbox. This means that even if it usually processes mailboxes every day, it can sometimes take up to 7 days. If you would like to try and force the process, you can always run "Start-ManagedFolderAssistant <mailbox ID>"

Another important thing to note here, is that MRM will no process mailboxes that have less than 10MB in size.

Going further with more details about the possible causes of MRM/Archive not processing a mailbox:

1. First, please make sure that the attribute RetentionHoldEnabled on the mailbox, is not set to True (typical for migrations done via PST Import Service).

2. Another important thing, verify that the attribute ElcProcessingDisabled is set to False. This parameter tells us if MRM is going to process a mailbox or ignore it completely.

3. If we have a Default Archive/Default Retention tag (applied on the entire mailbox), we need to make sure:

  •  we have no personal archive/retention tag set to "never move to archive/never delete" previously applied on folders
  •  we have no disabled or default archive/retention tag applied on the entire mailbox 
  •  with MFC MAPI, we need to check the property called PR_ROAMING_XMLSTREAM, which contains all the retention policies applied on a mailbox; we need to make sure that in the list of retention policy tags found in this property, we have the Default Archive tag present (or any other policy tag that we just applied); if we cannot find it, delete the IPM.Configuration.MRM message that contains the PR_ROAMING_XMLSTREAM property and run a Start-ManagedFolderAssistant for the affected mailbox, this will regenerate the IPM.Configuration.MRM message which should also update the PR_ROAMING_XMLSTREAM with the new policy tag.
  •  to locate the PR_ROAMING_XMLSTREAM and the IPM.Configuration.MRM message, please follow these steps:


- setup the affected mailbox in Outlook

- download the correct version (must match the installed version of Office of 32 or 64 bit) of MFC MAPI from here: https://github.com/stephenegriffin/mfcmapi/releases

- open MFC MAPI and go to Tools/Options and check these 2 flags: MAPI_NO_CACHE and MDB_ONLINE

- click on Session/Logon and then select the Outlook profile that contains the affected mailbox and open it

- double click on the affected mailbox in MFC MAPI and look after the Top of Information Store after you expand the Root Container

under the Top of Information Store, look after Inbox and right click on it, then select Open Associated Contents Table

- expand the current window and look after the message class IPM.Configuration.MRM and highlight it

- after highlighting the message, below you should see all of its properties, look after the one called PR_ROAMING_XMLSTREAM

- double click the property and copy paste the XML inside it, in a notepad then save it as .xml

- open the xml with a browser to be able to read it easier, it will show you what are the actual retention policy tags that are applied on the mailbox

This property contains all the retention policies that govern the mailbox. 

If we have personal archive/retention tags, please check if the personal tag is correctly applied on the folders by using MFC MAPI. Simply open the Outlook profile of the affected mailbox in MFC MAPI as described above and check the properties of the folder for archive tags or policy tags.

For example, when dealing with a Default Archive Policy (that applies on the entire mailbox), you will not be able to see any archive policy properties on the mailbox with MFC MAPI, such as: PR_ARCHIVE_TAG, PR_ARCHIVE_PERIOD, PR_ARCHIVE_DATE. There properties can be seen only when having a personal archive tag applied.

4. Collect the output of the following commands:

Get-MailboxFolderStatistics alias | fl Name,ContainerClass,DeletePolicy,ArchivePolicy,FolderType - this will show if personal tags have been applied on the mailbox folders

Get-MailboxFolderStatistics alias -IncludeOldestAndNewestItems | fl - see what are the newest and oldest items in the primary mailbox and correlate with the archive/retention tag period (for the Recoverable Items folder and its sub-folders, it is the oldest last modified date that is important, not the oldest received date)

Get-MailboxFolderStatistics alias -Archive -IncludeOldestAndNewestItems | fl - see what are the newest and oldest items in the archive mailbox, to verify if emails have been moved based on the archiving period (for the Recoverable Items folder and its sub-folders, it is the oldest last modified date that is important, not the oldest received date)

Get-MailboxStatistics alias | fl ItemCount,DeletedItemCount,TotalItemSize,TotalDeletedItemSize - follow the item count and total item size to see if the mailbox size is modified (we should see a decrease in the TotalDeletedItemSize and DeletedItemCount if we have an archive enabled in which we move Dumpster items from the primary mailbox)

Get-MailboxStatistics alias -Archive | fl ItemCount,DeletedItemCount,TotalItemSize,TotalDeletedItemSize - follow the item count and total item size to see if the archive mailbox size is modified (means that items are being moved in and the size/count should increase)

Export-MailboxDiagnosticLogs *** Email address is removed for privacy *** -ComponentName MRM - this should show us if errors have occurred when the Managed Folder Assistant processed the mailbox; beware of the date when the last error occurred to see if it is relevant for your issue; "resource unhealthy" errors are throttling messages, which means that MRM is processing the mailbox, but at a very slow rate because of the size and the number of items in the mailbox. Throttling cannot be avoided when dealing with big mailboxes.

Once again it is very important to pay a great deal of attention when checking the MRM logs, to their creation/generation date so that you know they are relevant to your current issue.

IMPORTANT: If you are not seeing any logs (error message on red font saying, “no logs were found”), this means that MRM processed the mailbox without any kind of errors. You can confirm that the Managed Folder Assistant ran across the mailbox, by checking the attributes ElcLastRunTotalProcessingTime and ElcLastSuccessTimestamp (see below).

The ExtendedProperties flag of the command Export-MailboxDiagnosticLogs, will produce a big XML output, out of which we are interested to see only the Elc properties (Email Life Cycle), so we need to parse the XML with below set of 3 commands in Powershell:

$logProps = Export-MailboxDiagnosticLogs *** Email address is removed for privacy *** -ExtendedProperties

$xmlprops = [xml]($logProps.MailboxLog)

$xmlprops.Properties.MailboxTable.Property | ? {$_.Name -like "ELC*"}

ElcLastRunTotalProcessingTime

ElcLastRunSubAssistantProcessingTime

ElcLastRunUpdatedFolderCount           

ElcLastRunTaggedFolderCount             

ElcLastRunUpdatedItemCount              

ElcLastRunTaggedWithArchiveItemCount    

ElcLastRunTaggedWithExpiryItemCount

ElcLastRunDeletedFromRootItemCount  - items from Deleted Items folder that expire and should be moved to Recoverable Items automatically    

ElcLastRunDeletedFromDumpsterItemCount – items from Recoverable Items folder that are purged

ElcLastRunArchivedFromRootItemCount - items that are being moved from the primary mailbox Inbox or Top of Information Store, into the archive's Inbox or Top of Information Store

ElcLastRunArchivedFromDumpsterItemCount  - items that are being moved from the primary mailbox Recoverable Items folder into the archive mailbox Recoverable Items folder

ElcLastSuccessTimestamp - the last time when MRM processed the mailbox without encountering any errors; in case of MRM throttling, these errors can be temporary, which means that items will continue to be moved/deleted but at a slower rate than usual.

5. To check the retention policies and tags, we can run below commands. Pay attention to those policy tags that are disabled or have the actions set to “never move to archive” or “never delete”.

Get-RetentionPolicyTag -Mailbox *** Email address is removed for privacy *** | fl

Get-RetentionPolicyTag -Mailbox *** Email address is removed for privacy *** -IncludeSystemTags

Get-Mailbox *** Email address is removed for privacy *** | fl *Retention* - see what we have on RetentionHoldEnabled and 

RetentionPolicy

Get-RetentionPolicy | fl

Get-RetentionPolicy | select -ExpandProperty RetentionPolicyTagLinks – see what policy tags are added in the MRM policy; this command assumes we have only the Default MRM Policy

Get-RetentionPolicyTag | fl 

We always take into consideration the tag that has the biggest period of time set on it. This is why I said above to check for tags that have “never move to archive” or “never delete” because these are the biggest periods of time theoretically and they will have precedence before any other tags that may be applied.

You can find more details about what is the meaning of the above command switches (such as IncludeSystemTags) or properties that can be seen when running those commands, here: https://docs.microsoft.com/en-us/powershell/module/exchange/policy-and-compliance-retention/Get-RetentionPolicyTag?view=exchange-ps 

Remember that we usually process the Dumpster first and then the Inbox or the folders under the Top of Information Store, but on the same level with the Inbox. This can cause some confusion and make the user think that retention and archive policies are not working on his mailbox.

The Dumpster of the primary mailbox, should not be at the maximum quota (30GB for regular mailbox or 100GB for a Litigation Hold enabled mailbox). In this situation, MRM will no longer move items in the archive.

With the above in mind, one should consider enabling the archive mailbox for an account, right after it has been placed on Litigation Hold (especially if the user has a high amount of email traffic), to avoid having the Recoverable Items folder becoming full and the user being unable to further delete items from his primary mailbox. Additionally, they should enable Auto-expand archives depending on their Office 365 licenses (more details can be found here Mailbox Storage Limits in Exchange Online.

Useful resources:

How retention age is calculated

Retention tags and retention policies

Default Retention Policy in Exchange Online and Exchange Server 

Default folders that support Retention Policy Tags

Setup and archive and deletion policy for mailboxes in your organization

Thank you,

Alex

Tags: MRM,Archive,Exchange,Online,Office365,Retention,Policies

Back to Exchange Online Support Corner

Looking for something like this for 6 months and after 3 support tickets with Microsoft.  Appreciate this very much!
 
 

Forum Article Info


Last updated October 19, 2020 Views 9,455 Applies to: