Skip to main content

How to sort items by a custom date in an Umbraco v13+ Examine Index

Briefly Examine and Lucene

Examine is a powerful tool that simplifies data indexing and searching. It was developed by Shannon Deminick and operates on top of the Lucene.Net Search Engine Library, which is a high-performance search library designed for .NET applications.

 

Examine is highly extensible, allowing users to configure multiple indexes, each with its own settings. By default, Examine provides a Lucene-based index implementation along with a Fluent API for easy data searching.

 

Additionally, Umbraco offers a layer known as UmbracoExamine, which provides Umbraco-specific APIs for indexing and searching content and media. The Umbraco backoffice utilizes the same search engine to return results for content, media, and various Umbraco settings, including Document Types, Data Types, and Dictionary Items.


Umbraco stores all values in Examine as Strings


By default, Umbraco stores all search index values as strings. This creates a significant issue when you need to sort your data by a custom date field. Since the values are stored as strings instead of datetime, you cannot order items by date properly.

Updating an Examine Index and adding a Custom Date Field

If you want to order your search items by a custom date, you should update your index and add your custom field that stores the date as a "long" type. Below is the full implementation for adding this custom date field and ordering the results by this custom date field.

New Sortable Date Field Constant

namespace ProjectName.Common.Constants
{
    public static class Search
    {
        public static class ExamineFieldDefinition
        {
            public const string SortableArticlePublishedDate = "sortableArticlePublishedDate";
        }
    }
}

Examine Component

using Examine;
using ProjectName.Common.Constants;
using ProjectName.Web.Models.ModelsBuilder;
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.Web;
using Umbraco.Extensions;

namespace ProjectName.Web.Components
{
    // Add Examine Component functionality on start-up
    public class ExamineComponent : IComponent
    {
        private readonly IExamineManager _examineManager;
        private readonly IUmbracoContextFactory _umbracoContextFactory;

        public ExamineComponent(IExamineManager examineManager, IUmbracoContextFactory umbracoContextFactory)
        {
            _examineManager = examineManager;
            _umbracoContextFactory = umbracoContextFactory;
        }

        public void Initialize()
        {
            if (!_examineManager.TryGetIndex(Umbraco.Cms.Core.Constants.UmbracoIndexes.ExternalIndexName, out IIndex index))
            {
                throw new InvalidOperationException(
                    $"No index found by the name {Umbraco.Cms.Core.Constants.UmbracoIndexes.ExternalIndexName}");
            }

            if (!(index is BaseIndexProvider indexProvider))
                throw new InvalidOperationException("Could not cast)");

            indexProvider.TransformingIndexValues += IndexProviderTransformingIndexValues;

        }

        private void IndexProviderTransformingIndexValues(object? sender, IndexingItemEventArgs e)
        {
            var values = e.ValueSet.Values.ToDictionary(x => x.Key, x => (IEnumerable<object>)x.Value);

            if (int.TryParse(e.ValueSet.Id, out var nodeId))
            {
                using (var umbracoContext = _umbracoContextFactory.EnsureUmbracoContext())
                {
                    var contentNode = umbracoContext?.UmbracoContext?.Content?.GetById(nodeId);
                    if (contentNode != null)
                    {
                        var val = e.ValueSet;

                        switch (val.ItemType)
                        {
                            case NewsArticle.ModelTypeAlias:

                                // Get create date
                                var createDate = val.GetValue(GetUmbracoFriendlyCamelCaseKey(nameof(NewsArticle.CreateDate)));

                                // Get update date
                                var updateDate = val.GetValue(GetUmbracoFriendlyCamelCaseKey(nameof(NewsArticle.UpdateDate)));

                                // Get publication date
                                var publicationDate = val.GetValue(GetUmbracoFriendlyCamelCaseKey(nameof(NewsArticle.PublishedDate)));

                                // Use the create date by default (not all articles will have a update date or publication date)
                                var finalDate = createDate;

                                if (DateTime.TryParse(publicationDate?.ToString() ?? string.Empty, out DateTime publicationDateValue))
                                {
                                    if (publicationDateValue > DateTime.MinValue)
                                    {
                                        // Use the publication date as it was set
                                        finalDate = publicationDate;
                                    }
                                }
                                else if (DateTime.TryParse(updateDate?.ToString() ?? string.Empty, out DateTime updateDateValue))
                                {
                                    if (updateDateValue > DateTime.MinValue)
                                    {
                                        // Publication date was not set but the item has been published. Use the update date
                                        finalDate = updateDate;
                                    }
                                }

                                if (DateTime.TryParse(finalDate?.ToString(), out DateTime finalDateTime))
                                {
                                    // Add the new key to the dictionary
                                    values.TryAdd(Search.ExamineFieldDefinition.SortableArticlePublishedDate, new List<object> { finalDateTime.Ticks });
                                }

                                break;
                        }
                    }
                }
            }
            e.SetValues(values);
        }
       
        public void Terminate()
        {
            // do not do anything
        }

        /// <summary>
        /// Returns Umbraco-friendly camel case key value from the property name
        /// </summary>
        /// <param name="propertyName"></param>
        /// <returns></returns>
        private string GetUmbracoFriendlyCamelCaseKey(string propertyName)
        {
            if (!string.IsNullOrEmpty(propertyName))
            {
                return $"{propertyName.Substring(0, 1).ToLower()}{propertyName[1..]}";
            }

            return propertyName;
        }
    }
}

Examine Composer

using Microsoft.Extensions.DependencyInjection;
using ProjectName.Web.Components;
using ProjectName.Web.CustomIndexing;
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.DependencyInjection;

namespace ProjectName.Web.Composers
{
    public class ExamineComposer : IComposer
    {
        // Customise the behaviour of the Umbraco application at start up by composing the following functionalities into the pipeline
        public void Compose(IUmbracoBuilder builder)
        {
            // Register ConfigureExternalIndexOptions in a composer so that it can configure our indexes with custom index fields.
            builder.Services.ConfigureOptions<ConfigureExternalIndexOptions>();
            builder.Components().Append<ExamineComponent>();

        }
    }
}

Adding the new custom date field to the Examine Index

using Examine;
using Examine.Lucene;
using Microsoft.Extensions.Options;
using ProjectName.Common.Constants;
using Umbraco.Cms.Core;
using Umbraco.Cms.Infrastructure.Examine;

namespace ProjectName.Web.CustomIndexing
{
    public class ConfigureExternalIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
    {
        public void Configure(string name, LuceneDirectoryIndexOptions options)
        {
            if (name.Equals(Constants.UmbracoIndexes.ExternalIndexName))
            {
                options.Validator = new ContentValueSetValidator(true, true, null, null);

                options.FieldDefinitions.TryAdd(new Examine.FieldDefinition(Search.ExamineFieldDefinition.SortableArticlePublishedDate, FieldDefinitionTypes.Long));
            }
        }

        public void Configure(LuceneDirectoryIndexOptions options)
        {
            //  // Part of the interface, but does not need to be implemented for this.
            throw new NotImplementedException();
        }
    }
}

New Examine Search Service

using ProjectName.Web.Models.ModelsBuilder;
using Umbraco.Cms.Core.Models.PublishedContent;

namespace ProjectName.Web.Services.Interfaces
{
    public interface IExamineSearchService
    {
        IEnumerable<NewsArticle> GetOrderedNewsArticles(int parentId);
    }
}

using Examine;
using ProjectName.Common.Constants;
using ProjectName.Web.Models.ModelsBuilder;
using ProjectName.Web.Services.Interfaces;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Models.PublishedContent;
using Umbraco.Cms.Core.Security;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Infrastructure.Examine;
using Umbraco.Extensions;

namespace ProjectName.Web.Services.Implementations
{
    public class ExamineSearchService : IExamineSearchService
    {
        private readonly IExamineManager _examineManager;
        private readonly IPublishedContentQuery _publishedContentQuery;

        public ExamineSearchService(IExamineManager examineManager, IPublishedContentQuery publishedContentQuery)
        {
            _examineManager = examineManager;
            _publishedContentQuery = publishedContentQuery;
        }

        public IEnumerable<NewsArticle> GetOrderedNewsArticles(int parentId)
        {
            try
            {
                var index = GetIndex();

                var query = index.Searcher.CreateQuery(IndexTypes.Content);
                var queryExecutor = query.ParentId(parentId)
                    .And()
                    .NodeTypeAlias(NewsArticle.ModelTypeAlias)
                    .OrderByDescending(new Examine.Search.SortableField(Search.ExamineFieldDefinition.SortableArticlePublishedDate, Examine.Search.SortType.Long))
                    .Execute();

                var newsArticleIds = queryExecutor.Select(x => x.Id);

                var results = _publishedContentQuery.Content(newsArticleIds).OfType<NewsArticle>();

                return results;
            }
            catch (Exception ex)
            {
                return Enumerable.Empty<NewsArticle>();
            }
        }

        #region Private methods

        private IIndex GetIndex()
        {
            if (!_examineManager.TryGetIndex("ExternalIndex", out var index))
            {
                throw new InvalidOperationException($"No index found by name ExternalIndex or is not of type {typeof(IUmbracoIndex)}.");
            }
            return index;
        }

        #endregion Private methods
    }
}

Adding the new Examine Search Service to the Pipeline

using Microsoft.Extensions.DependencyInjection;
using ProjectName.Web.Services.Implementations;
using ProjectName.Web.Services.Interfaces;

namespace ProjectName.Web.Extensions
{
    public static class ServiceCollectionExtensions
    {
        public static void AddServices(this IServiceCollection source)
        {
            source.AddScoped<IExamineSearchService, ExamineSearchService>();
        }
    }
}

Sample Razor View to get the ordered search results by custom date and display them

@using Microsoft.AspNetCore.Http.Extensions
@using Microsoft.Extensions.Primitives
@using ProjectName.Web.Models.ModelsBuilder
@using Umbraco.Cms.Core.Models.Blocks
@using Microsoft.AspNetCore.Http
@inherits UmbracoViewPage<BlockGridItem<CardsCollection>>
@inject IExamineSearchService ExamineSearchService

@if (Model != null)
{
    var page = Umbraco.AssignedContentItem;

    var items = ExamineSearchService.GetOrderedNewsArticles(page.Id); // Getting the ordered items

    var query = Context.Request.Query;
    int pageSize = 9;
    int pageNumber = 1;
    int.TryParse(query["page"], out pageNumber);

    if (Context.Request.Query.TryGetValue("category", out StringValues category))
    {
        string? catName = category.FirstOrDefault();
        if (!string.IsNullOrEmpty(catName))
        {
            items = items?.Where(x => x.Tags != null && x.Tags.Contains(catName));
        }
    }

    var queryBuilder = new QueryBuilder(Context.Request.Query.Where(x => x.Key != "page"));
    bool hasQueryString = queryBuilder.Any();

    var totalPages = (int)Math.Ceiling((double)items!.Count() / (double)pageSize);

    <section class="query">
        <div class="container">

            @if (pageNumber > totalPages)
            {
                pageNumber = totalPages;
            }
            @if (pageNumber < 1)
            {
                pageNumber = 1;
            }

            @if (totalPages > 1)
            {
                <div class="pagination">
                    <ul>
                        @if (pageNumber > 1)
                        {
                            <li><a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@(pageNumber-1)">Prev</a></li>
                        }
                        @for (int p = 1; p < totalPages + 1; p++)
                        {
                            <li class="@(p == pageNumber ? "active" : string.Empty)">
                                <a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@p">@p</a>
                            </li>
                        }
                        @if (pageNumber < totalPages)
                        {
                            <li>
                                <a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@(pageNumber+1)">Next</a>
                            </li>
                        }
                    </ul>
                </div>

            }

            <div class="event-grid__boxs">
                @foreach (var item in items!.Skip((pageNumber - 1) * pageSize).Take(pageSize))
                {
                    <div class="box">
                        <a href="@item?.Url()" class="image">
                            <img src="@item?.ThumbnailImage?.LocalCrops?.Src" alt="@item?.ThumbnailImage?.Name">
                        </a>
                        <div class="box__info">
                            <div>
                                <div class="row" style="float:right;">
                                    <h5>@item?.PublishedDate.ToString("dd/MM/yy")</h5>
                                </div>
                            </div>
                            <a href="@item?.Url()" class="text-decoration-none">
                                <h2>@item?.Name</h2>
                            </a>
                            <div>
                                @if (item?.TimeToRead > 0)
                                {
                                    @Html.Raw(item.TimeToRead + " min")
                                }

                                <ul class="news-tags">
                                    @if (item!.Tags!.Any())
                                    {
                                        @foreach (var tag in item?.Tags!)
                                        {
                                            <li>
                                                <a href="/news/?category=@tag">@tag</a>
                                            </li>
                                        }
                                    }
                                </ul>
                            </div>
                        </div>
                    </div>
                }
            </div>

            @if (totalPages > 1)
            {
                <div class="pagination">
                    <ul>
                        @if (pageNumber > 1)
                        {
                            <li><a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@(pageNumber-1)">Prev</a></li>
                        }
                        @for (int p = 1; p < totalPages + 1; p++)
                        {
                            <li class="@(p == pageNumber ? "active" : string.Empty)">
                                <a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@p">@p</a>
                            </li>
                        }
                        @if (pageNumber < totalPages)
                        {
                            <li><a href="@(queryBuilder?.ToQueryString())@(hasQueryString ? "&" : "?")page=@(pageNumber+1)">Next</a></li>
                        }
                    </ul>
                </div>

            }
        </div>

    </section>

}

The new sortable date field and final note

If you have reached this point, you should have a new custom date field that you can use to sort your items by that date as displayed below. 
















Comments

Popular posts from this blog

How to fix Git push error: "RPC failed; curl 56 HTTP/2 stream 7 was reset send-pack: unexpected disconnect while reading sideband packet fatal: the remote end hung up unexpectedly"

Problem Today I saw the following problem when I tried to push my changes to a Git server after doing some work for upgrading an Umbraco v7 project to v8.18.8.  Possible reasons After some investigations, it seems like this could be because of the following reasons; Git is not happy with the amount of changes that are being pushed into the server.  There are possible limitations on the server about the size/amount of files that you can push. Your internet connection is not good and stable enough. Your Git client's version is old. Solution options For me, the easiest option was connecting to another Wifi and trying again. Apparently, this option helped quite a few people, so it is worth giving it a try. Unfortunately, it didn't work for me. A bad internet connection wasn't an option for me either, as my internet is pretty fast (500 Mbps). Similarly, my Git client version was the latest version (git version 2.41.0.windows.3).  On StackOverflow, there were a lot of recommendat

How to use JQuery Ajax Methods for Async ASP.NET MVC Action Methods

Making repeatedly calls to async methods can be a nightmare. In this case, it makes sense to use 2 ajax methods, instead of one. Here is a simple solution to overcome this problem. See that  ajaxcalls   is emptied after the success response for the first ajax call and then the second ajax method is used to make one single call to the async action method. Hope it helps. View: @section Scripts{     < script type ="text/javascript">         var smartDebitObject = new Object();         smartDebitObject.MembershipNumber = $( "#MembershipNumber" ).val();         smartDebitObject.ProfileId = $( "#ProfileId" ).val();         smartDebitObject.FirstName = $( "#FirstName" ).val();         smartDebitObject.LastName = $( "#LastName" ).val();         smartDebitObject.AddressLine1 = $( "#AddressLine1" ).val();         smartDebitObject.Postcode = $( "#Postcode" ).val();         smartDebitObject

How to fix "Microsoft SQL Error SQL71564: Error validating element [USERNAME]: The element [USERNAME] has been orphaned from its login and cannot be deployed."

I needed to export a database in BACPAC format today in order to restore it somewhere else, and I encountered the following error. To resolve this issue, I deleted all of the users mentioned in the error log. After successfully creating the BACPAC file, I used it to create a new database with no problems. Error: TITLE: Microsoft SQL Server Management Studio ------------------------------ One or more unsupported elements were found in the schema used as part of a data package. Error SQL71564: Error validating element [USER1]: The element [USER1] has been orphaned from its login and cannot be deployed. Error SQL71564: Error validating element [USER2]: The element [USER2] has been orphaned from its login and cannot be deployed. Error SQL71564: Error validating element [USER3]: The element [USER3] has been orphaned from its login and cannot be deployed. Error SQL71564: Error validating element [USER4]: The element [USER4] has been orphaned from its login and cannot be deployed. Error SQL71