October 22, 2013

Publishing Your XML Sitemaps to Your Content Delivery Servers

By: Craig Taylor
October 22, 2013

Publishing Your XML Sitemaps to Your Content Delivery Servers
My most-recent post talked about how you can utilize the Sitecore Sitemap XML module in order to automatically build robots.txt and XML sitemap files in a mulit-site, hardened environment so that the search engines can see your sitemap(s).  This works great, but a new issue arises if you have separate Content Managment (CM) and Content Delivery (CD) servers.

The Problem

When your content is published, the Sitemap XML module waits for the "publish:end" event to start building the robots.txt and XML sitemap(s).  It does so and stores the files in the file system.  The issue with this is that the processing and creation of files all happens on the CM server, which is behind a firewall.  The CD servers are the web-facing servers that will be serving the sitemap and robots.txt files.  We need the files on the CD servers.

The Solution

I went through a number of iterations in my head about how I could get the robots.txt and XML sitemap files to the CD servers.  I thought about scheduled jobs in Sitecore.  I thought about using an external application that is used for file replication.  I thought about a custom application inside of Sitecore so that a content author could manually kick off a deploy.

After running some of these ideas by a fellow Arke employee and Sitecore MVP, Andy Uzick, I came to realize that I was over thinking my solution by far.  This sort of problem had been solved long ago and is *WAY* simpler than I had originally thought.

The solution is to change the "SitemapXML.config" to subscribe to the "publish:end:remote" event instead of the "publish:end" event.  This event fires after a successful publish and let's remote servers perform the actions specified.  In our case, this lets the SitemapXML module know to build the robots.txt and sitemap(s) files.  Just ensure that you have the SitemapXML.DLL and SitemapXML.config files present on all your CD servers and you're good-to-go!

Very simple solution and now I don't have to manually copy files between servers!

11 comments:

  1. I struggled with this one for a couple of hours. Your solution works great. Thank you Craig and Andy for this solution and for taking the time to document it. - Craig Nelson

    ReplyDelete
  2. the only inconsistency with it (or maybe I'm doing something wrong) is that for some servers the sitemap.xml gets generated with the server IP in the URL (for ex: instead of putting http://uat.testing.com/Our-places.aspx puts http://127.0.0.1/Our-places.aspx , ignoring what I entered in the serverURL, while in others it takes the value from there or if I leave it blank it takes the correct DNS name.

    Do we know why and how to solve that? am I missing a setting somewhere?

    ReplyDelete
    Replies
    1. Hi Carlos. I've seen this before as well but do not know a solution for it. I'd recommend posting the issue to the developers of the module (not me): http://marketplace.sitecore.net/en/Modules/Sitemap_XML

      Delete
  3. It’s working great. Thank you for posting this.

    ReplyDelete
  4. Do you make the change to the config files on the CD side, the CM side, or both? Tried just the CD side and it didn't work.

    ReplyDelete
    Replies
    1. Rachel: You should be able to just make the change on the CD side. If you don't update the CM, it will just create the sitemaps on your CM server as well, which doesn't hurt anything. I'd suggest to examine the Sitecore logs on CM and CD to see if you can see what might be causing it to not work.

      Delete
  5. Quick question -- did you have to install the Sitemap XML module on your delivery servers? If so how did you do this? They aren't supposed to be capable of running the Sitecore client...

    ReplyDelete
    Replies
    1. No, you don't have to run through the 'full' install via the Sitecore interface. Just manually copy up the Sitemap XML DLL and Sitemap XML config file and this should work.

      Delete