At the end of last year, we pulled together a detailed site migration guide supporting those brands that might be considering taking the leap in 2021. Each week we will focus on a new stage to support you with your migration plans and help you maintain your SEO performance throughout.
You can skip to the relevant sections using the links below:
- Part 1 – Planning & Scope
- Part 2 – Pre-Migration Planning
- Part 3 – In the Test Environment Part 1 – Preparation
Site migrations can provide a fantastic opportunity to improve the functionality of your website and fix all those frustrating CMS quirks that prevent you from getting the most of your current setup. Conversely, you might find yourself in a situation where a migration becomes unavoidable. Maybe a new piece of back-end software won’t work with your current setup, maybe you’re (finally) switching the site over to HTTPS or maybe you’re going through a complete re-brand where your entire domain name needs to change.
Either way, a migration can be a fantastic opportunity to re-build your site as you’d always wanted it and fix all those long-standing issues that you’ve come to despise over the years. But migrations also carry a SEO huge risk, and can, if badly implemented, lead to a complete loss of visibility in search engines, organic traffic and revenue.
This is a worst-case scenario and can be avoided with some planning and careful implementation of a clear migration strategy. But there can be a lot to consider when making such large-scale changes to your site, so we’ve utilised our Tech SEO team’s comprehensive experience in site migrations to create this in-depth guide to help steer you through the process.
Part 1: Planning & Scope
Vital to any migration success is the planning and scope stage to fully understand the objective, timing and key actions or milestones of the project. It is essential that this is well communicated across all parties involved, and centrally managed to keep the migration on track.
In addition, it is worth noting that SEO requirements for a migration can often be left to near the end of a migration plan, by which point it’s often too late to adequately implement the required functionality. Many of the issues impacting SEO, such as page weight, site structure and domain consolidation need to be addressed at the very earliest planning stages so it is important that SEO is considered at the very start of the process.
1.1: Timings & Potential impact
From an SEO perspective, it is important that the site migration takes place at a time of relatively low site traffic to minimise the impact of the migration.
Typically, a site migration will change many aspects of a website, with the site’s design, templates, coding, content and entire URL structure potentially being altered.
When this happens, search engines need time to understand the changes that have taken place, and there may be a period of reduced organic traffic while, for example, they crawl and index redirected URLs.
For this reason, we’d advise agreeing on a go-live date that allows at least 2 months of recovery before your site’s busiest traffic or trading times.
1.2: What’s in Scope?
As mentioned above, many elements can be changed as part of a site migration. It’s important to understand the full scope of changes that will take place during the migration to allow whoever’s responsible for your SEO to plan work, estimate the impact of the migration and reduce the impact on organic visibility.
Examples of elements that may be changing as part of the migration include:
• Design of the site, including page templates
• Site URL structure
• Site domain (i.e., www.example.com may change to www.better-example.co.uk)
• Site Content Management System (CMS)
• Site code base (i.e. incorporating more JS elements into pages)
• Site content
• Site security protocol (i.e., changing from http://www.example.com to https://www.example.com)
• Domain consolidation (combining one or more domains, moving sub-domain content to the main domain)
• Internationalisation (i.e. adding content to target different international markets)
• Mobile responsive design
Any, or all, of the above, maybe changed during a migration, and each element requires time for search engines to understand the changes.
1.3: Staggered Migrations & HTTPS
Once you’ve worked out the scope of your migration project, you need to agree on whether everything is going to be implemented at once or opt for a staggered migration. While project timings and budgets don’t always make this possible, we’d recommend that if a lot of on site elements are changing, a staggered migration is considered. This means that some elements are changed separately, their impact assessed, and further changes are then made at a later date.
For example, we’d suggest that content changes are made separately to a migration to a new CMS so that the same content is ‘lifted and shifted’ to the new platform where possible. This is partly to reduce the number of site changes search engines need to process in one go, and also to make it easier to measure the impact of each major change that’s made as part of the migration.
Similarly, Google’s advice on HTTP to HTTPS migrations is that they should be undertaken separately from other large-scale site changes as it makes it easier for Google to understand that the new secure version of the site is the same as the old non-secure version and quickly re-index pages under the new protocol.
We’ll be looking closer at advice on HTTPS in the upcoming weeks as we recommend sites use this version of the protocol (and because many will be on HTTPS already and this needs to be tested as part of the migration), but we’d suggest that any change from HTTP to HTTPS should be carried out as a separate migration.
1.4: Traffic Alternatives
As the migration is likely to result in a short-term loss of organic traffic, it’s worth looking at alternative sources of traffic to the site during the process to make up the shortfall. This may include increasing PPC budget on certain high-value search terms that may lose organic visibility through the migration or increasing the frequency of email campaigns during the migration process.
Once a date for migration has been agreed, it’s important to scope out these additional marketing activities and understand what will need to be increased, and over what period.
1.5: Required Functionality
While making large-scale changes to the site does carry risk to organic visibility, it also provides a great opportunity to fix current issues, update site functionality and ensure current SEO best practice is being delivered.
For this reason, we suggest SEO requirements are fully mapped out and considered when a new CMS or site design is implemented. We often find these requirements are best communicated through user stories using the format:
“As a <type of user>, I want <functionality example> so that <benefit for SEO>.”
This can help to explain exactly why you’re pushing for specific changes to be made, and which stakeholders will benefit from the functionality. For example:
“As a content editor, I want the ability to redirect pages in the CMS so that I can ensure equity from out-of-date pages is passed to a relevant up to date URL.”
We can then rank these requirements by importance to help with prioritisation, stating which functionality is:
b. highly recommended
1.6: Site Speed, Usability & Mobile Responsiveness
Site speed, usability and accessibility via mobile devices are very important elements in how Google assesses websites, and how users interact with your brand online. It’s important that these issues are taken into account during the development build of a new site to ensure it works well on mobile devices and that pages load as quickly as possible.
For this reason, we’d recommend considerations for both are included as part of the scoping process, particularly where site redesigns are concerned.
Site speed can be benchmarked through Google’s own Page Speed Insights and Lighthouse tools, which will give your site a grade out of 100, as well as suggestions for improvements. This insight can be used to understand current performance and also provide recommendations on ways to improve on this score in the new build.
In terms of mobile accessibility, we’d strongly suggest any migration that deals with a redesign of the site’s templates is built from a mobile-first perspective to ensure mobile users have the best experience possible, and to help ensure you’re not serving different content to desktop and mobile users. This is increasingly important from both a user’s perspective and to ensure the site is optimised for Google’s mobile-first index.
Part 2: Pre-Migration Planning
In order to understand the potential impact of your migration, you’ll need to benchmark current site performance, including:
- Traffic – from Google Analytics
- Keyword rankings – using the rank checking software of your choice, we generally use SEO Monitor as it has daily rank checking which can be super helpful in the aftermath of a migration
- Site speed – from Google’s Page Speed Insights and Lighthouse tools, Web Page Test and Google Analytics
- Overall site visibility – using tools like Systrix and Semrush
- Domain Authority – from Moz
- Number of indexed pages – from Google Search Console or using a Google search
Take measurements for each of these elements pre-migration and document them for comparison post-live. You’ll also need to take a full crawl of the current site so you can easily compare new and old sites once the migration is complete. We’d recommend Screaming Frog for this, though other crawling programs are available.
If possible, it is also worth making and keeping a backup of the current site in case of emergencies, so the migration can be ‘rolled back’ to a previous version of the website if anything goes wrong.
2.2: Migration Roadmap
Once a go-live date has been agreed on, it’s important to create a full roadmap for the migration process with the help of all key stakeholders. Stakeholders should have an idea of how long elements within their area of expertise will take to implement and should be consulted in order to ensure enough time is scheduled to complete necessary tasks, and for robust testing of the new site before it goes live.
As part of this process, it’s important to set a date for a content freeze across the site after which only absolutely essential content changes should be made. This is to ensure there is a ‘final’ list of current site content to be moved to the new site, that won’t be altered pre-migration.
Migration timings can often slip and the go-live date pushed back as various developments and dependencies are delayed. It’s worth being a little pessimistic in terms of timings for each migration stage to make sure your go-live date isn’t pushed back to the extent that it jeopardises a busy period of traffic.
It’s also a good idea to communicate this to relevant stakeholders in the business to ensure they’re aware that the initial dates given for the migration are subject to change – it’s never a good idea to be tied to a completion date that’s not attainable and trying to force through a site that’s not ready to launch.
2.3: Rationalising Current Site Content
One of the first steps in a migration is generating a list of current site pages, and agreeing what will be migrated to the new site.
Some migrations might not require content to change at all, for example, if you’re only:
- Changing the protocol from http to https
- Moving the entire site, unchanged, to a new CMS
- Changing the design/templates of a site
In these instances, you may want to keep all pages intact and simply ‘lift-and-shift’ every current page to the new site. In this case, you’ll still want to create a list of current site pages as a way of checking that all content is moved to the new site.
2.4: Pulling a list of pages
If possible, wait until after the content freeze to pull this list. Otherwise, you’ll need to re-do or add pages to the list.
The list of pages can be created through:
- A crawl of the current site – taken from Screaming Frog or similar
- A list of URLs taken from Google Analytics
- A list of all pages with an external link pointing to it – taken from a link analysis tool such as Ahrefs, Google Search Console (free) or any other link crawling software
- A list of pages shared on social platforms – taken from Buzzsumo or similar
- A list of pages indexed in search – taken from Scrapebox or similar
Combine all these lists into one Excel sheet and de-dupe them. You may need to amend the formatting for some of these to ensure they’re in the same format (ie so either all URLs are using the full domain path, or none of them are).
Once you have this full list, it’s worth running it all through Screaming Frog’s List Mode or similar to pull out the status code of each page. Anything listed as a 404 or 301/302 redirect does not need to be carried forward but should be added to your redirect list – see chapter 5.
If you’re happy keeping all site pages the same and simply migrating everything ‘as is’, you can skip the next section and simply use this list to check all content has been moved over successfully.
However, site migrations provide a great opportunity to re-assess content on your site and determine what should be kept, what should be removed and how you can improve content for greater SEO visibility, user experience and customer interaction.
2.5: Auditing Current Site Content
Poor quality content can impact how that content ranks in search and in some cases can also impact the visibility of the entire site, so it’s important to periodically audit your site’s content to ensure quality is being maintained throughout the site and pages are performing well in search.
There are numerous guides to conducting an SEO content audit (a couple of examples are listed below), but the basic premise is to generate a list of all pages for your site, pull in some ranking, traffic and engagement metrics to find out how content is performing, and determine which pages should be kept, improved (or consolidated) and removed.
Pulling in page metrics
Once you have your list of current site pages, it’s worth pulling in some traffic and engagement metrics from Google Analytics. To do this:
- Go to Google analytics and pull organic traffic by landing page for as many URLs as you can
- By default you’re limited to exporting 5,000 rows at a time, so if your site has significantly more key landing pages than this, you’ll need to do several exports of this data and combine them, or use a tool like Supermetrics or a Google Sheets add on to pull more than 5,000 rows of data.
- The GA data should contain organic landing page visits as well as metrics like Bounce Rate, Time on Page, Average Pages Per Session etc
- Make sure the URLs in the GA export are in the same format as your full list of URLs (ie all using the full domain path etc)
- Use a VLOOKUP in Excel to pull the traffic and engagement metric data from your GA sheet into the pages on your full URL list
- You can also repeat this process to pull in things like social shares (from Buzzsumo, etc), external links (from Ahrefs, Majestic or similar link analyser), number of important keywords pointing to the URL (from your rank checking software) and Page Authority (from Moz).
Once you have this data you can use it to order your pages based on current performance. One of the key metrics to use here is organic traffic, as generally speaking, pages that are not generating many visits from organic search are either not of good enough quality to rank, or are not serving a user intent.
However, it’s worth sense-checking why these pages are not performing as there may be a simple technical reason they’re not ranking, despite containing fantastic, useful content. You may find that improving internal linking to the page, or expanding the scope of the content is enough to get the page ranking and delivering traffic on your new site.
You can also supplement this traffic data with additional metrics to get a better idea of performance. For example, you may want to look at all key landing pages with a higher click-through rate than your site average or review all pages with no social shares as a priority.
Assess this prioritised list with the following rules:
- Keep – page is performing well, content is good and meets user needs. The page should be moved to the new site pretty much as-is.
- Improve /Consolidate – page is not performing as well as it could be or contains content that is weak, thin, inaccurate, out of date, duplicated (or very close to) other pages on your site or on other sites. However, the page has promise/is about a topic central to the site. In this case, the page needs to be improved, re-written or combined with similar pages to create one page of strong content.
- Remove – as above, but there is little reason for the page to exist. It may be unrelated to the current site proposition, a duplicate of an existing page or a page with no content worth saving. In this instance, the page should be marked for removal with the URL redirected to a new page on site.
This is a brief overview of the process involved, but it’s worth consulting with a technical SEO specialist (like us 😉 ) or reading through some of the numerous guides available online for more information:
2.6: New Site IA
Once you’ve agreed to a list of pages that will be moved over to the new site, the next step is to create an Information Architecture document for the new site, outlining the pages it will contain, where these pages sit on-site and how they relate to each other.
We recommend creating two documents for this that both need checking and signing off by all relevant stakeholders:
- A visual IA document showing the site structure and hierarchy
- A confirmed list of new site pages, along with their URLs – see Chapter 3
We’d recommend assessing the new IA to make sure that all key pages are surfaced within a couple of clicks of the home page.
Maintaining SEO equity
There will be many factors that determine the final list of pages that will be kept on the new site, particularly in relation to business/stakeholder needs. However, it’s important that key SEO pages are kept as part of the migration, or failing that, the content is incorporated into other pages that will be strong enough to maintain visibility for important keywords.
This is where your list of current site pages comes in handy as one of your metrics should include keyword ranking data that you can use to make sure all URLs currently ranking for important terms are moved over to the new site.
Mapping keywords to pages
Once the list of new site pages has been agreed upon, it’s useful to map your current list of high-value keywords to your new list of pages. In many cases, there will be a direct and easily identifiable page on the new site that directly corresponds to a page that’s ranking on the current site, especially if you’re doing a simple ‘lift-and-shift’ of content.
However, in some cases, particularly where you are auditing site content, a page that was optimised for important terms may have been removed, and you’ll need to optimise a different page on the new site for them instead.
We’ll cover this keyword mapping in detail in Chapter 3, but at this stage, it’s important to be aware of the pages on your site that are ranking for important terms, and ensuring you have a plan to migrate this content, in some form, over to the new site.
3.0: In the Test Environment Part 1 – Preparation
Most migrations require a test environment to be set up where the new site content can be added and amended until it is ready to launch. In some cases, the current site will be largely replicated in the test environment and used as a template for the new site. In others, the new site will be built from the ground up in the test environment and completely re-designed and re-coded as part of the migration process. In either case, the test environment serves as an area where content and on-page elements can be uploaded and optimised until it is ready to take the place of the old website.
3.1: Blocking Indexation
You need to ensure the test site is not indexable by search engines to make sure no work-in-progress content is seen by the public, and to prevent confusion over which version of the site Google should serve to users.
This can be done by:
- Password protecting the test environment
- Adding a site-wide no-index tag to every page on-site to prevent pages from being indexed
- Adding a rule in the site’s robots.txt file that blocks pages from being crawled
- A combination of the above
Once this is done, it’s worth periodically checking that pages are not being indexed by using the ‘site:’ search operator in Google, ie: “site:test.example.com”.
We’d strongly suggest password protecting the site as the primary method of blocking Google access to the site. The other methods can be employed as a backup/double-protection but ideally should not be used on their own, as they both have weaknesses that could see pages indexed by accident.
So ideally, we’d recommend password protecting the site, then adding the noindex tag to every page on the site as a back-up.
Once this is done, it’s worth periodically checking that pages are not being indexed by using the ‘site:’ search operator in Google, ie: “site:www.your-test-domain.com”.
3.2: URLs & Page Equity
Depending on the type of migration you’re carrying out, you may need to create a new URL structure for your site. Even if every page on the site is staying the same, and you’re taking a ‘lift-and-shift’ approach to content, you may decide to change the site’s URL structure as part of the migration to make URLs easier to read for users.
However, be advised that changing the URL of a page on the new site means the new URL is essentially starting from scratch in the eyes of a search engine.
The pages on your site build up value over time in the eyes of search engines that help them to rank for key words and phrases. This value, or search equity is tied to the specific URL of that piece of content. So, if you use a different URL for the piece of content when you migrate the site, this value is essentially lost and the new URL has to begin accruing equity from scratch. Some pages on your site may have built up a lot of equity over time from high quality links from other sites, so it’s essential to preserve this if you want the pages on your new site to rank well.
If you want to maintain this search equity from the old URL to the new one, a redirect must be put in place.
So, if you have a current site page: http://www.example.com/a-really-great-page and the new site URL is going to be: http://www.example.com/great-page, you must redirect the current URL to the new one.
This applies to anything that changes the URL string, including:
- Changing the site’s domain
Old Site: http://www.example.com/a-page
New site: http://www.example-site.com/a-page
- Adding/removing the www. prefix
Old Site: www.example.com/a-page
New site: example.com/a-page
- Changing the security protocol from http to https (this should be a separate migration)
Old Site: http://www.example.com/a-page
New site: https://www.example.com/a-page
- Adding or removing a trailing slash on all URLs
Old Site: https://www.example.com/a-page
New site: https://www.example.com/a-page/
- Changing the casing of the URL
Old Site: https://www.example.com/a-page
New site: https://www.example.com/A-Page
- Changing how words in the URL are separated
Old Site: https://www.example.com/a-page
New site: https://www.example.com/a_page
So, any change to the URLs between the old and new site needs a redirect to be applied if you want to maintain search equity and have your content continue to rank post-migration.
Bear in mind that even when using redirects, you may see a short-term drop in search visibility. This is covered in more depth in the redirects section to follow.
Which type of redirect to use?
There are a number of different redirect types you could employ to send users and search engines from your old URLs to your new ones. In 2016, Google announced that all types of 3xx redirects will pass search equity between pages, but we’d recommend sticking to using 301 redirects as they denote a permeant change in URL. Using 301s should mean your old pages will be removed from the search index once Google crawls the redirect.
3.3: Writing new URLs
Creating new URLs for pages can be very useful if your current URLs are confusing for users, hard to read, or don’t fully reflect what the page is about. You may also need to re-write your URL structure based on your new IA as pages may have been added, removed or changed and the current structure may not fit. In any case, a site migration is often a good time to re-assess your pages and create a new simple, easy to read URL structure.
As a general rule, we’d recommend that URLs are static rather than dynamic, are easy to read, use words rather than random character and number strings, and are consistent across the site. For this reason, it’s useful to create a set of rules for your URLs that ensure consistency and readability are maintained.
We recommend URLs are:
- All lower case
- All use a trailing slash
- All with www. prefix
- All use hyphens to separate words
- All use the same security protocol
- Kept short and easy to read
Once you’ve decided on a set of rules, you can write new URLs for your pages and save them so they can be used in page creation within the test environment.
It’s essential that these URLs are signed off and adhered to by content creators and editors throughout the migration process.
If a new site URL has to change for some reason, it’s important that this change is communicated to all stakeholders and recorded, as these new site URLs are crucial to the site redirects and is something that is often overlooked.
3.4: Mapping of Keywords to New Site URLs
Once you have a full list of new site URLs, you can start the process of mapping keywords to each page of the site. You may already have a clear idea of which pages you want to rank for important keywords for your business, based on how your site ranks at present. You may be in a situation where you want to change which pages rank for certain terms, or you may be removing/amalgamating pages but want to maintain rankings.
Either way, you probably already have a list of terms that you want your site to rank for, and these can be mapped to the pages of your new site. If you don’t have this already, or think there may be additional terms you want to be visible for in search, you’ll need to carry out extensive keyword research. You can speak to your SEO agency about this, or use one of the many in-depth guides available online:
Backlinko’s Definitive Guide to Keyword Research
Ahrefs’ Keyword Research blog
Kissmetric’s Keyword Research the Smart way
Once you have an extensive list of keywords, it’s a good idea to track their ranking positions over time using a keyword ranking tool. You can use this to assess how the migration has impacted keyword rankings, and also check which pages are ranking for which terms pre and post-migration. We’d recommend keywords are tracked on a weekly or two-weekly basis, and if possible, a subset of keywords should be changed to daily ranking over the period of the new site going live to better understand changes during the switch.
How to map keywords to your new site pages
A good way to start with this is to pull the data for which pages on your current site rank for each term:
Take your list of keywords, put them into a rank checking program (we use SEO Monitor – other rank checking tools are available) and pull the results into a spreadsheet:
You should have the keyword, its position, and the current site URL ranking. You should also have the monthly search volume for each keyword as part of your keyword research. If you don’t have this, you can get it through Google’s Keyword Planner tool.
You can then organise this data by URL to see which current pages are ranking for which keywords:
Once you have this, you’ll need to work out whether your current ranking pages have an alternative in your list of new URLs. In some cases this might be pretty easy, the URLs may be the same, or very similar across both URL sets. However, if you’re changing URL structure or adding/removing pages as part of the migration, it might require a bit more work.
Once you’ve worked out the equivalent new site page for each of your current ranking URLs, you should be left with something like this. Any instances where there is no direct new site replacement for a current URL can be marked with NA.
You’ll now need to run through this list and check that all keywords mapped to pages make sense. If you find keywords that you feel are mapped to the wrong pages, feel free to change the new site URL you want them mapped to.
You’ll also need to go through all keywords that no longer have a page assigned and map them to a new site URL. If you find keywords that can’t logically be mapped to a new site URL you’ll need to consider whether you can afford to lose visibility for these terms, and if not, consider adding a new page to your new site plan.
Again, bear in mind, any newly added pages need to be communicated to key stakeholders and reordered.
You can now reverse-engineer this list of URLs and keywords so that you’re left with a list of new site URLs, the keywords that should be mapped to them, and the current site version of each URL:
This list will likely contain most of your important new site pages, but may not be complete. So you’ll now need to add in the rest of your new site pages, to give you a complete list. The easiest way to do this is to copy and paste your full list of URLs into column A of your KW mapping spreadsheet and then de-dupe the list. This should leave you with all your new site URLs, with many of them having KWs mapped to them.
You can now go through and map keywords to any additional pages that do not have keywords assigned.
Make sure you keep the ‘Current URL’ column in this list. It will make things easier when you come to start writing the redirects.
3.5: Keeping Track of Everything
You should now have a couple of documents detailing the URLs for your new site. One will simply be a list of pages you and their URLs you agreed on in section 3.3, and one will be this list of pages with keywords assigned, and for some pages, a reference to the corresponding current site URL.
To allow for ease of communication, it’s useful to assign a reference number to each page being created for the new site. This way, if the URL of a page changes, the reference number can still be used in communication so everyone knows which page people are referring to. The reference numbers can follow the structure of the new site, like this:
Chapter 4 will be on its way shortly, but if you need support with your own site migration don’t hesitate to get in touch with our team of specialist Technical SEO Consultants.