Error Handling

The sad truth is that Jasper will not unfrequently hit exceptions as it processes messages. In all cases, Jasper will first log the exception using the standard ASP.Net Core ILogger abstraction. After that, it walks through the configured error handling policies to determine what to do next with the message. In the absence of any configured error handling policies, Jasper will move any message that causes an exception into the error queue for the transport that the message arrived from on the first attempt.

Today, Jasper has the ability to:

• Enforce a maximum number of attempts to short circuit retries, with the default number being 1
• Selectively apply remediation actions for specific Exception types or matching conditions on Exception types
• Choose to re-execute the message immediately
• Choose to re-execute the message later
• Re-queue the message for later execution at the back of the line
• Just bail out and move the message out to the error queues
• Discard the message
• Script out how an error is handled on various attempts
• Apply error handling policies globally, by configured policies, or explicitly by chain
• Use custom error handling to do whatever you want utilizing Jasper's IContinuation interface

Configuring Global Error Handling Rules

To establish global error handling policies that apply to all message types, use the syntax as shown below:

public class GlobalRetryApp : JasperOptions
{
    public GlobalRetryApp()
    {
        Handlers.OnException<TimeoutException>().RetryLater(5.Seconds());
        Handlers.OnException<SecurityException>().MoveToErrorQueue();

        // You can also apply an additional filter on the
        // exception type for finer grained policies
        Handlers
            .OnException<SocketException>(ex => ex.Message.Contains("not responding"))
            .RetryLater(5.Seconds());
    }
}

In all cases, the global error handling is executed after any message type specific error handling.

Explicit Chain Configuration

To configure specific error handling polices for a certain message (or closely related messages), you can either use some in the box attributes on the message handler methods as shown below:

public class AttributeUsingHandler
{
    [RetryLater(typeof(IOException), 5)]
    [RetryNow(typeof(SqlException))]
    [RequeueOn(typeof(InvalidOperationException))]
    [MoveToErrorQueueOn(typeof(DivideByZeroException))]
    [MaximumAttempts(2)]
    public void Handle(InvoiceCreated created)
    {
        // handle the invoice created message
    }
}

If you prefer -- or have a use case that isn't supported by the attributes, you can take advantage of Jasper's Configure(HandlerChain) convention to do it programmatically. To opt into this, add a static method with the signature public static void Configure(HandlerChain) to your handler class as shown below:

public class MyErrorCausingHandler
{
    // This method signature is meaningful
    public static void Configure(HandlerChain chain)
    {
        // Requeue on IOException for a maximum
        // of 3 attempts
        chain.OnException<IOException>()
            .Requeue(3);
    }


    public void Handle(InvoiceCreated created)
    {
        // handle the invoice created message
    }

    public void Handle(InvoiceApproved approved)
    {
        // handle the invoice approved message
    }
}

Do note that if a message handler class handles multiple message types, this method is applied to each message type chain separately.

Configuring through Policies

If you want to apply error handling to chains via some kind of policy, you can use an IHandlerPolicy like the one shown below:

// This error policy will apply to all message types in the namespace
// 'MyApp.Messages', and add a "requeue on SqlException" to all of these
// message handlers
public class ErrorHandlingPolicy : IHandlerPolicy
{
    public void Apply(HandlerGraph graph, GenerationRules rules, IContainer container)
    {
        var matchingChains = graph
            .Chains
            .Where(x => x.MessageType.IsInNamespace("MyApp.Messages"));

        foreach (var chain in matchingChains)
        {
            chain.OnException<SqlException>().Requeue(2);
        }
    }
}

To apply this policy, use this syntax in your JasperOptions:

public class MyApp : JasperOptions
{
    public MyApp()
    {
        Handlers.GlobalPolicy<ErrorHandlingPolicy>();
    }
}

Filtering on Exceptions

To selectively respond to a certain exception type, you have access to all of Polly's exception filtering mechanisms as shown below:

public class FilteredApp : JasperOptions
{
    public FilteredApp()
    {

        Handlers
            // You have all the available exception matching capabilities of Polly
            .OnException<SqlException>()
            .Or<InvalidOperationException>(ex => ex.Message.Contains("Intermittent message of some kind"))
            .OrInner<BadImageFormatException>()

            // And apply the "continuation" action to take if the filters match
            .Requeue();

        // Use different actions for different exception types
        Handlers.OnException<InvalidOperationException>().RetryNow();
    }
}

Built in Error Handling Actions

The most common exception handling actions are shown below:

public class ContinuationTypes : JasperOptions
{
    public ContinuationTypes()
    {

        // Try to execute the message again without going
        // back through the queue with a maximum number of attempts
        // The default is 3
        // The message will be dead lettered if it exceeds the maximum
        // number of attemts
        Handlers.OnException<SqlException>().RetryNow(5);


        // Retry the message again, but wait for the specified time
        // The message will be dead lettered if it exhausts the delay
        // attempts
        Handlers
            .OnException<SqlException>()
            .RetryLater(3.Seconds(), 10.Seconds(), 20.Seconds());

        // Put the message back into the queue where it will be
        // attempted again
        // The message will be dead lettered if it exceeds the maximum number
        // of attempts
        Handlers.OnException<SqlException>().Requeue(5);

        // Immediately move the message into the error queue for this transport
        Handlers.OnException<SqlException>().MoveToErrorQueue();
    }
}

The RetryLater() function uses Scheduled Message Delivery and Execution.

See also Dead Letter Envelopes for more information.

Scripting Error Handling by Attempt

Using the TakeActions() method, you can script out fine-grained retry/requeue/discard policies for an exception by attempt number as shown below:

Unknown topic key 'AppWithScriptedErrorHandling' -- CTRL+SHIFT+R to force refresh the topic tree

Exponential Backoff Policies

By integrating Polly for our retry policies, Jasper gets exponential backoff retry scheduling nearly for free.

To reschedule a message to be retried later at increasingly longer wait times, use this syntax:

public class AppWithErrorHandling : JasperOptions
{
    public AppWithErrorHandling()
    {
        // On a SqlException, reschedule the message to be retried
        // at 3 seconds, then 15, then 30 seconds later
        Handlers.OnException<SqlException>()
            .RetryLater(3.Seconds(), 15.Seconds(), 30.Seconds());

        // This is another equivalent option
        Handlers.OnException<TimeoutException>()
            .TakeActions(x =>
            {
                x.RetryLater(3.Seconds());
                x.RetryLater(15.Seconds());
                x.RetryLater(30.Seconds());

                // Jasper will automatically move the
                // message to the dead letter queue
                // after a 4th failure
            });
    }
}

Custom Actions with IContinuation

If you want to write a custom response to failed message handling, you may need to write a custom IContinuation that just tells Jasper "what do I do now with this message?":

public interface IContinuation
{
    Task Execute(IMessagingRoot root, IChannelCallback channel, Envelope envelope,
        IQueuedOutgoingMessages messages, DateTime utcNow);
}

Internally, Jasper has built in IContinuation strategies for retrying messages, moving messages to the error queue, and requeueing messages among others.

As an example, let's say that on a certain exception type, you want to reschedule the failed message for an hour but also raise some kind of alert event for the support team to know what just happened. A custom continuation class might look like this:

public class RaiseAlert : IContinuation
{
    private readonly Exception _ex;

    public RaiseAlert(Exception ex)
    {
        _ex = ex;
    }

    public async Task Execute(IMessagingRoot root, IChannelCallback channel, Envelope envelope,
        IQueuedOutgoingMessages messages,
        DateTime utcNow)
    {
        // Raise a separate "alert" event message
        var session = root.NewContext();
        await session.Schedule(envelope, utcNow.AddHours(1));
        await session.Send(new RescheduledAlert()
        {
            Id = envelope.Id,
            ExceptionText = _ex.ToString()

        });


    }
}

Then in usage, we can apply the continuation usage like this:

public class AppWithCustomContinuation : JasperOptions
{
    public AppWithCustomContinuation()
    {
        Handlers.OnException<UnauthorizedAccessException>()

            // The With() function takes a lambda factory for
            // custom IContinuation objects
            .With((envelope, exception) => new RaiseAlert(exception));
    }
}