Using SharePoint PnP and Sharegate to Clone SharePoint Online sites

So you’ve been given the task to clone a production Intranet site running on SharePoint Online to a clean site collection. It could be for testing, other development work, or training, and the question is how can we do this on the Office 365 platform?

If it were an on-prem instance of SharePoint, no problem; fire up PowerShell, Backup-SPSite > Restore-SPSite, done. Easy. (Storage capacity issues aside…)

But there’s no such option with SharePoint Online, so what are the alternatives?

Sharegate is probably one of the first tools that springs to mind for most SharePoint Admins. But what if our Intranet requires certain site objects to remain identical — as an example property bag entries, content types and site column IDs? Sharegate will bring most of these across, but may generate new IDs for some of them in the process, so you find your Intranet just doesn’t work the same way on the new site (or not at all).

Maybe some automation tools were used to build your site initially, so you could dust off the code and fire it at your new site collection? But your production site has been in active use for a while now, are you sure the code will replicate Production (I doubt it)?

This is where the SharePoint PnP project comes to the rescue!

An aim of the project and its provisioning engine is it can extract the configuration and structure of existing SharePoint sites and save them to XML templates, which can then be applied to new (and existing) sites.

Now we’re probably pushing the boundaries of what the provisioning engine was intended for, but when paired up with Sharegate, the two work really well together to get everything across the line — the overall process looks something like the following…

Install the SharePoint PnP PowerShell Module (if you don’t have it already):

Install-Module SharePointPnPPowerShellOnline

Connect to your production SP Online site:

Connect-PnPOnline — Url https://YourSPOnlineTenant.sharepoint.com

Extract the configuration and structure from the site and save to an XML template:

Get-PnPProvisioningTemplate -Out YourSiteTemplate.xml

Now there are some quirks with the way PnP builds this XML and applies it to a new site, so you may need to get your hands a little dirty at this point and open up the XML to slice out a few elements — some of the common ones I’ve found are:

• Site security groups
• Microfeed list instances
• Some Navigation elements

Not a problem to exclude these of course, as Sharegate will pick them up afterwards

Now we can connect to the new site collection (PnP can provision this too, which is great):

Connect-PnPOnline — Url https://YourSPOnlineTenant.sharepoint.com/sites/NewSite

Then we apply the XML template to the new site:

Apply-PnPProvisioningTemplate -Path YourSiteTemplate.xml

Depending on the complexity of your production site, you may run into some odd errors when applying the new template. A couple of PnP features can help to troubleshoot these —

Turn on trace logging:

Set-PnPTraceLog -On -Level Debug

And/or exclude some sections when processing, to focus on a particular area, with the “ExcludeHandlers” parameter:

Apply-PnPProvisioningTemplate -Path YourSiteTemplate.xml

-ExcludeHandlers SiteSecurity,TermGroups,Navigation

If you find the source of your error is something in the XML template, you can go back and revise it or just cut / comment it out and run the Apply-PnPProvisioningTemplate Cmdlet again.

Now you have an exact copy of your production site structure and configuration, you can then fire up Sharegate to finish the job. If you use the “Merge” option when copying the site objects and content with Sharegate, it will match up the Content Types and Site Column IDs etc created by PnP, and use them instead of creating new ones. !

Some handy references for the SharePoint PnP PowerShell module:

Summary of the project:

https://github.com/SharePoint/PnP-PowerShell

Cmdlet reference:

https://github.com/SharePoint/PnP-PowerShell/blob/master/Documentation/readme.md

Cloning site collections in SharePoint Online is definitely more complex and time consuming compared to SharePoint on-prem, however, it’s both possible and a necessary. This is now a process we are using with all our customers.

How are you cloning your SharePoint Online site collections? Do you need help creating your test, dev or training site collections? I’m interested in hearing all about your problems and experience.

If you have any additional questions about cloning or SharePoint in general, please get in touch with
our local team of SharePoint experts today.