You probably don't need a custom index - Modifying the ExternalIndex in Examine and Umbraco

Additional indexes means more storage and more places for index corruption, so I prefer to keep my indexes to a minimum. Umbraco comes with an ExternalIndex out the box - this is designed to be used for end-user site search, and doesn't contain any unpublished nodes (unlike the InternalIndex which is intended to be used by the backoffice).

Defining fields in an existing index

To add our own fields to an index or to change the type of a field, we need to create an IConfigureNamedOptions<LuceneDirectoryIndexOptions>. This class will need registering in a composer (see below). The important lines are:

options.FieldDefinitions.AddOrUpdate(new FieldDefinition("searchCategories", FieldDefinitionTypes.Raw));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition("searchDate", FieldDefinitionTypes.DateMonth));

The definition name is a magic string we'll use to reference the field when setting the values or searching. It can be anything so long as it is unique (avoid calling it the same thing as one of your properties!) and is conventionally camelCased.

Select the most relevant field definition type from the Examine Value Types documentation. In this example, I've used Raw (for exact matches) and DateMonth (for storing dates as Ticks to the precision of the month).

My full class looks like this:

public class ConfigureExternalIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
    private readonly IOptions<IndexCreatorSettings> _settings;

    public ConfigureExternalIndexOptions(IOptions<IndexCreatorSettings> settings)
        => _settings = settings;

    public void Configure(string? name, LuceneDirectoryIndexOptions options)
    {
        if (name?.Equals(Constants.UmbracoIndexes.ExternalIndexName) is false)
        {
            return;
        }

        options.FieldDefinitions.AddOrUpdate(new FieldDefinition("searchCategories", FieldDefinitionTypes.Raw));
        options.FieldDefinitions.AddOrUpdate(new FieldDefinition("searchDate", FieldDefinitionTypes.DateMonth));
    }

    // not used
    public void Configure(LuceneDirectoryIndexOptions options) => throw new NotImplementedException();
}

Adding or updating values in an existing index

If you only want to modify the value of a field or have created the new field, as above, we can start populating the fields in the index.

To do this, we have to copy the existing values to a dictionary, add or modify the values and save them back using the SetValues method.

Each field can have multiple values, as shown in the case of searchCategories, but this is often an array of one item, like with searchDate.

I've also commented out an example of modifying a value which is the example given in my blog post on Tag-style exact-matching with Examine. In this case, I could have done the same thing with categories, but have chosen to index the sanitized values as a new field in this example.

public class ExternalIndexValueTransformationComponent : IComponent
{
    private readonly IExamineManager _examineManager;
    private readonly IShortStringHelper _shortStringHelper;

    public ExternalIndexValueTransformationComponent(IExamineManager examineManager, IShortStringHelper shortStringHelper)
    {
        _examineManager = examineManager;
        _shortStringHelper = shortStringHelper;
    }
    public void Initialize()
    {
        if (!_examineManager.TryGetIndex(Constants.UmbracoIndexes.ExternalIndexName,
                out var index))
        {
            return;
        }

        if (!(index is BaseIndexProvider indexProvider))
        {
            return;
        }

        indexProvider.TransformingIndexValues += ExternalIndex_TransformingIndexValues;
    }


    private void ExternalIndex_TransformingIndexValues(object? sender, IndexingItemEventArgs e)
    {
        var values = e.ValueSet.Values.ToDictionary(x => x.Key, x => (IEnumerable<object>)x.Value);

        // Insert new values

        if (e.ValueSet.Values.ContainsKey("displayDate") && DateTime.TryParse(e.ValueSet.GetValue("displayDate").ToString(), out DateTime date))
        {
            var searchDate = date.ToString("MM-yyyy");
            values.Add("searchDate", new [] { searchDate });
        }

        if (e.ValueSet.Values.ContainsKey("categories"))
        {
            var categories = e.ValueSet.GetValues("categories")
                .Select(x => x.ToString()?.ToCleanString(_shortStringHelper, CleanStringType.UrlSegment))
                .WhereNotNull().ToArray();
            if (categories?.Any() ?? false)
            {
                values.Add("searchCategories", categories);
            }
        }

        // Modify existing values
        //foreach (var value in e.ValueSet.Values)
        //{
        //    if (value.Key == "myKey")
        //    {
        //        var values = value.Value.FirstOrDefault().ToString().Split(Environment.NewLine);
        //        updatedValues["myKey"] = values.Cast<object>().ToList();
        //    }
        //}

        e.SetValues(values.ToDictionary(x => x.Key, x => x.Value));

    }

    public void Terminate()
    {
        if (_examineManager.TryGetIndex(UmbracoIndexes.ExternalIndexName, out IIndex index)
            && index is BaseIndexProvider externalIndex)
        {
            externalIndex.TransformingIndexValues -= ExternalIndex_TransformingIndexValues;
        }
    }
}

Registering the configuraiton

Both of these need adding to a composer's compose method

public class Composer : IComposer
{
    public void Compose(IUmbracoBuilder builder)
    {
        builder.Services.ConfigureOptions<ConfigureExternalIndexOptions>();
        builder.Components().Append<ExternalIndexValueTransformationComponent>();
    }
}

Rebuild the indexes

You can debug the methods above either by saving a content node or by rebuilding the index with a breakpoint in the above code. You'll also need to rebuild the index to get the new/modified values for all content.

To rebuild the index, navigate to Settings > "Examine Management" tab > ExternalIndex > "Rebuild index" button.

Searching with the modified index

Here's an example of a blog search functionality. This could live in a controller or service. This makes use of the Search Extensions package, which I generally recommend, for some of the extension methods and path format.

if (_examineManager.TryGetIndex(UmbracoIndexes.ExternalIndexName, out IIndex index) == false)
{
    throw new Exception($"Failed to find {UmbracoIndexes.ExternalIndexName}");
}

var type = typeof(BlogPost).Name.ToLower();

var query = index.Searcher.CreateQuery("content").NodeTypeAlias(type);

query.And().Field("path", blogRoot.Id.ToString());

if (!string.IsNullOrEmpty(category))
{
    query.And().Field("searchCategories", category);
}

if (!string.IsNullOrEmpty(month))
{
    if (DateTime.TryParseExact(month, "MM-yyyy", null, DateTimeStyles.None, out var date))
    {
        query.And().Field("searchDate", date);
    }
}

if (sortAscending == true)
{
    query.OrderBy(new SortableField("searchDate", SortType.Long));
}
else
{
    query.OrderByDescending(new SortableField("searchDate", SortType.Long));
}

var searchResults = query.Execute();

return searchResults.GetResults<BlogPost>();

Further Reading

• Tag-style exact matching