This blog posts reflects our recent experience with a variety of large clients in the development of Proofs of Concept (PoC) that use Microsoft 365 (M365) and Microsoft Purview (Purview) to achieve enterprise records management (RM) at scale. These clients are all using E5 tenants with at least the E5 Information Protection and Governance (IP&G) add-on to enable Purview RM. Our experience is that it is much easier to implement a Purview RM PoC in a consistent E5 environment than a mixed E5/E3 environment.
Most of the organizations we work with are at least $5B companies, many much larger. They have had records management programs for decades, including both electronic and physical RM. In many of these organizations, the decision to commit to an E5 level of licensing was driven by concerns related to privacy, security, identity, and perimeter protection, was driven by the CIO or CISO, and was unrelated to RM. By and large, these organizations understand that Microsoft is going to require them to use Purview to govern their content in M365. They are skeptical of Purview RM based on what they have heard from vendors and consultants, and they don’t want to lose the records functionality they are comfortable with in their current RM systems. Their objective is to prove Microsoft Purview RM can satisfy their needs based on their current RM processes in a safe environment and, therefore, they launch a Purview RM PoC initiative.
Our clients have had a variety of motivations and questions that they want to address in their development of a Purview RM PoC:
- Is Microsoft Purview ready?
- Can we govern M365 content without breaking the bank or burying our users with too much effort to classify and manage records?
- Can we develop a governance model that meets the requirements of our leadership, our RM team, our legal team, and the business?
- How should we enable and test the integrations that were included in ECM systems we are replacing?
- Do we believe Azure Purview will address the challenges of managing file shares with RM capabilities and immutability in a potential move to Azure files?
- Finally, many organizations are on a forced march to M365, so it is just time to get on board.
Alternative PoC Approaches
Many organizations start with a Purview RM PoC using out-of-the-box capabilities. They set up Purview and learn to use it. The good news with this approach is that Purview works as advertised. The bad news is that it doesn’t do RM the way most Records Managers expect to do it. It is a fine IT solution but is somewhat cumbersome and time-consuming for Records Managers to use to manage traditional RM and disposition processes. It is even more cumbersome for large-scale disposition management because of the Purview presentation of each record for separate disposition review. Frequently, these organizations consider whether their challenges and RM requirements need to be addressed by extensions to Purview.
First, a little background. Microsoft Purview RM is an E5 product that is clearly the future of Microsoft 365 Records Mgmt. All of the content in Microsoft 365 is contained in the Microsoft Substrate, which is independent of the M365 workloads including SharePoint Online, Exchange Online, Teams, OneDrive, etc. Purview is the solution Microsoft uses to manage tags on content in the Microsoft Substrate, including retention labels, sensitivity labels, and many other label types. It includes retention and disposition via the Purview Retention Engine. All integration with Purview is achieved through the Graph API’s and metadata combination and reporting. No customization of Purview is possible. This is a huge difference from prior Information Governance Solutions. Tools that integrated with the M365 workloads independently to manage content and records are scrambling to be relevant in the M365 governance world.
In 2022, Gartner published an excellent review[1] of the requirements and alternatives for enterprise RM in Microsoft 365. It concluded that an E3 license was insufficient, but that E5 would be sufficient for enterprise RM if a variety of requirements and challenges were addressed. Here is a list of Gartner’s overall requirements for Enterprise Records Management:
- Manage and retain content stored in-place within the system (1)
- Declare an item as an immutable record (1)
Manage and retain content stored outside the system (1) - Management of physical record metadata (6)
- Ability to apply multiple life cycle rules to a single piece of content
- Long-term audit log retention matching record life-span (2)
- Defensible destruction (2)
- Manage record termscentrally with a file plan (7)
- Allow for multistage disposition review (4)
- Support multiple retention triggers (5)
Guarantee immediate destruction of data once retention ends - Provide analytics and reporting (3)
The following paragraphs provide some insights to the numbered requirements. What we have seen in our clients is that with some straightforward extensions to Purview, it is possible to address many of the limitations of Purview that were identified by Gartner and have frustrated some Records Managers. These extensions are not an add-on solution. Instead, they add the capture of some additional audit log data related to record events in an Azure SQL table and enable additional reporting and data aggregation based on the audit log. Other extensions include support for consistent site provisioning with governance built in, the integration of retention schedules, an event disposition framework, and support for using Purview to manage Azure files and blobs. There are reasonable solutions to the non-numbered requirements, but our clients have not seen them as equally important.
1. Manage and retain content stored in-place
M365 does an excellent job of managing the retention of content stored in M365. Extensions enable Purview to manage the retention and immutability of content stored in Azure files and blobs. This enables network file shares to be moved to Azure with classification metadata applied and the immutability of records managed by Azure Purview while retaining the familiar Explorer interface.
2. Audit Log retention for life of record
Extensions can capture a wide variety of audit log information into Sentinel or an Azure SQL table for an indefinite
period (=life of the record). While Purview has limits on how long metadata is stored, an extension would not. This is useful for long-lived records, archival, and to provide disposition and eDiscovery evidence.
3. Dashboards, Analytics & Reporting
3. Dashboards, Analytics & Reporting
Extensions provide extensive analytics and reporting capabilities through dashboards and reports that combine information from across the M365 and Purview tenant. These dashboards and reports leverage the additional audit log information and the Power Automate and Power BI frameworks. Organizations can easily customize and extend these dashboards and reports using these tools.
4. Disposition Review & Workflow
Extend Microsoft Purview disposition process to enable user-friendly disposition and reporting capabilities:
- Aggregation of records to enable the disposition of thousands of records simultaneously.
- Filter records to identify subsets of records for disposition or to extend the disposition timeframe.
- Enable Disposition Certificates in bulk operations.
5. Retention Triggers and Events
Extension provides for the orchestration of events and leverages Purview to trigger integrated events in M365, typically as Document Sets. Events still need to be raised in LOB systems such as SAP, Dynamics, etc. This extension streamlines the management and disposition of records in Purview based on events.
6. Manage Physical Records
M365 provides no capabilities to manage physical records. A simple Purview extension can apply retention labels to Microsoft 365 list items representing physical records. We typically integrate to Gimmal Physical or other physical RM providers for more sophisticated physical RM capabilities. The integration is usually based on integrating to a shared retention schedule and presenting related records via search.
7. Manage Retention Schedules
Extension integrates with external retention providers (currently, Access Corp.’s Virgo and Iron Mountain’s Policy Center) to synchronize retention schedules with Microsoft Purview.
Purview extensions work because Purview is the same platform in every M365 tenant. Long gone are the days when RM solutions had to be aware of the customizations in the M365 workloads to apply governance. Microsoft may eventually add many of the features and capabilities of these extensions. An additional benefit of this approach is that because the extensions don’t change any information or configurations in Purview, there is an almost zero cost for migration when the new Purview features and capabilities are available. Just stop using the extensions.
There are a variety of other important questions that often need to be addressed in a PoC:
How to create a manageable disposition lifecycle?
- How to execute retention lifecycle in manner that records administrators like?
- How to execute retention lifecycle in PoC time frame?
- Is fully automated disposition management possible? (i.e., Build a consensus focused on policy and proving execution. Then, let disposition processes execute automatically)
How to enable the manageable application of retention labels to content
- How to minimize user involvement in the application of retention labels?
- What is the role of adaptive scopes and auto-classification?
- What about events?
Migration from SharePoint On-Premises
- Should we retain current Site Structures and Navigation?
- What about the migration of content and records?
- How to move SharePoint Record Centers to a Modern M365 experience?
Migration from ECM Platform
- What are the Integrations that need to be migrated?
- How to migrate ECM Data Models?
Migration from Network File Shares
- What is the Business Case for migrating from File Shares?
- Is it primarily cost or governance?
- What discovery tools to use?
- There are many, including Syntex & Viva Topics, Automated Intelligence, EncompaaS, Active Navigation, eventually Purview Copilot, and lots of others.
- Azure Files and Blobs: How to integrate with Purview?
What about your Retention Schedule?
- How to publish Records Retention Schedule as retention labels into Purview to enable data lifecycle management.
- How to import Content Type associations into Purview to support the auto-classification of records.
Information Architecture and Information Lifecycle Definitions
- How to incorporate retention taxonomies into Content Gallery and Term Store, especially for metadata inheritance?
- How to incorporate retention schedule and content type mappings into auto-classification?
Build out taxonomies in Microsoft Content Type Gallery and Term Store that are connected to retention policies and labels to enable automatic labelling:
- Configure each document library with an organizational RM content type based on standard RM classification columns customized to the client organization.
- Expose RM global columns (Information Type, Information Status) in default library views and set defaults at library and folder level for documents to minimize the need for manual entry of metadata.
- The Information Type property within each RM content type is used to organize documents to facilitate selection/tagging and align the document types with your retention schedule.
- We frequently inherit the RM content type information into other content types in the Content Type Gallery and publish them through the Managed Metadata Service.
- We work to adapt this taxonomy approach to your classification and records requirements, including automatic labeling.
Site provisioning
- How to establish manageable provisioning of sites with retention policies baked in.
- How to incorporate retention schedule and content type mappings into site provisioning process.
Things to also consider while conducting a PoC:
- Establish a current state assessment of Compliance and Governance
- Define an overall RM Process
- Establish an M365 information lifecycle strategy
- Establish a technology roadmap for the deployment of M365 Purview
o SharePoint Online
o Teams
o Exchange Online
o Azure
o Multi-Cloud?
o Microsoft 365 Archiving
- Establish a program approach for the governance and records management of information in Microsoft 365 and Azure leveraging Microsoft Purview
o Project Portfolio
o Business Case
Where to Host the PoC?
We have seen a variety of scenarios in terms of where our PoC’s are conducted. The simplest alternative is often to host the PoC in the partner/integrator’s tenant. There is no need to include any real documents or data in the early stages of the PoC. However, many large organizations are not comfortable with hosting PoC’s outside of their infrastructure. Conducting a PoC in the organization’s test tenant is an attractive alternative as long as the organization has the capacity and capabilities to do this. We have seen several PoC’s delayed because their IT departments weren’t ready to host a PoC.
Where to get help?
The learning curve for Purview is long and steep. As you address the gaps and challenges in a Purview PoC, try to find partners who have done it before:
- Experts in Records Management
- Experts in Purview RM
- Have done Purview RM at scale, multiple times
- If the partners come with pre-built Purview extensions, so much the better. This might save half of the work in a PoC. Extensions to Purview that have worked in one Purview tenant will generally work in other similarly-licensed M365 tenants.
What to Include in your PoC?
Everyone’s Purview PoC is going to be different. It is important to review the objectives of the PoC. Is the objective of the PoC to understand “the viability of Purview for RM” or to evaluate “migrating to M365 from other platforms with integrations”. Testing the viability of Purview for RM requires a very different approach to planning than the migration to M365 of a variety of ECM integrations with Line of Business or ERP solutions.
Summary
In this blog post, we have made the case that a variety of extensions to Purview along with addressing some additional requirements could make Purview RM much more useful to your organization. Which requirements make the PoC cut? This can be based on the target RM processes, the appetite of the organization for extending Purview, the available budget and other constraints on the business. Just do it.
Do you want to learn more, contact us. We’re happy to help.
