March 26, 2016

Solr in Production for Sitecore - SwitchOnRebuildSolrSearchIndex

March 26, 2016

Solr in Production for Sitecore - SwitchOnRebuildSolrSearchIndex

The configuration for running Solr in your local dev environment is likely going to be different than running it in the production environment.  Oftentimes, your local indexes will be configured as a type of "Sitecore.ContentSearch.SolrProvider.SolrSearchIndex" while using the "syncMaster" indexing strategy.  This strategy rebuilds the index after data changes are saved in the master database.  It works well locally if you want to avoid having to publish to the Web database to see your indexes update, but this doesn't work in a production environment where your Content Delivery (CD) servers don't have access to the master database.  Not to mention, you don't want to be indexing content that hasn't been published yet.

In a production environment, it is much more common to use the "SwitchOnRebuildSolrSearchIndex" index type while using the "onPublishEndAsync" indexing strategy.  This type keeps your indexes up and running even when they are rebuilding.  This is accomplished by building to a secondary index and when the rebuild is complete, Sitecore uses that index as the primary index.  The rebuild is triggered when items are published to the 'Web' database.

Note: I should also note here that the CD servers should actually have their indexing strategies set to "manual" due to the Content Management (CM) server controlling when the central Solr indexes are being rebuilt .  More on this below.

Since I have different strategies defined for the different environments and have different build configurations in my Visual Studio solution, I utilized config transforms to allow the correct type and strategy to be used in the different environments.  I'll detail these transforms in a future blog post.

For this example, my local development instance has a custom index used for indexing products defined as follows:

          <index id="product_search_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider">
            <param desc="name">$(id)</param>
            <param desc="core">$(id)</param>
            <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
            <configuration ref="contentSearch/indexConfigurations/productSearchIndexConfiguration" />
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/syncMaster" />
            </strategies>
            <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
              <policies hint="list:AddCommitPolicy">
                <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
              </policies>
            </commitPolicyExecutor>
            <locations hint="list:AddCrawler">
              <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
                <Database>web</Database>
                <Root>/sitecore/Content/Products</Root>
              </crawler>
            </locations>
          </index>

My local dev instance is a CM/CD combined instance and as you can see in my config, it uses the "Sitecore.ContentSearch.SolrProvider.SolrSearchIndex" type, with the "syncMaster" strategy on the "master" database.  I can now update items and have them immediately available in my index without having to go through the trouble of publishing.

When deploying to a dev, stage or prod CM server, I want to ensure that my products index is always available and only contains data that has been published.  We can accomplish this with the "SwitchOnRebuildSolrSearchIndex" index type and the "onPublishEndAsync" indexing strategy.  I have the following defined for a dedicated CM server:

          <index id="product_search_index" type="Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSorlSearchIndex, Sitecore.ContentSearch">
            <param desc="name">$(id)</param>
            <param desc="core">$(id)</param>
            <param desc="rebuildcore">$(id)_sec</param>
            <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
            <configuration ref="contentSearch/indexConfigurations/productSearchIndexConfiguration" />
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/onPublishEndAsync" />
            </strategies>
            <commitPolicyExecutor type="Sitecore.ContentSearch.CommitPolicyExecutor, Sitecore.ContentSearch">
              <policies hint="list:AddCommitPolicy">
                <policy type="Sitecore.ContentSearch.TimeIntervalCommitPolicy, Sitecore.ContentSearch" />
              </policies>
            </commitPolicyExecutor>
            <locations hint="list:AddCrawler">
              <crawler type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
                <Database>web</Database>
                <Root>/sitecore/Content/Products</Root>
              </crawler>
            </locations>
          </index>

Note: These two index configurations are shown to illustrate the process for both switching and utilizing the onPublishEndAsync strategy.  In an actual production environment, you should separate your indexes so that you have one for the master database and one for the web database.

The changes of note here are:
  1. The index type was changed to "Sitecore.ContentSearch.SolrProvider.SwitchOnRebuildSorlSearchIndex"
  2. We added a new parameter named "rebuildcore".  This tells Sitecore the name of the secondary core to utilize while building.  Note that you need to have an additional core created to facilitate this.
  3. The strategy has been changed to "onPublishEndAsync"
  4. The "Database" we are indexing is the "Web" database.
This is a configuration for a CM server.  In order to keep the CD servers from kicking off a rebuild of the indexes, you should set their index strategy type to "contentSearch/indexConfigurations/indexUpdateStrategies/manual".

Now, each time you publish an item, the index will be automatically updated and you don't have to worry about the rebuild time taking it offline as Sitecore will be using the secondary core you defined.

Additional Reading

1 comment:

  1. Awesome Post, m8! Exactly what I was looking for! Cheers

    ReplyDelete