Web Api MessagePack Formatter

Download the solution here.

There are multiple ways to achieve performance in designing our web applications and while some of them include optimizations at an architectural or infrastructure level, there is also the case of the size of data which is delivered over the wire. We do have the tendency to use what is trending and common and here I am talking about Json format which comes by default with asp.

MessagePack is “like Json but smaller”. It’s an efficient binary format serializer which reduces the size of the message sent. You can find more details about this over https://msgpack.org/. The following chart (which i extracted from the official git repository of MessagePack by neuecc:

In this article I am setting up an Asp dotnetcore 3 api application that uses a custom MediaTypeFormatter which serializes data with MessagePack

The solution consists in 5 projects, one of which is a test project:

Solution Tree

The solution has the following dependencies:

The Microsoft.AspNetCore.Mvc.Core provides the classes InputFormatter and OutputFormatter.

The Microsoft.AspNet.WebApi.Client (old  System.Net.Http.Formatting) provides the MediaTypeFormatter and the ReadAsAsync extension method we use in the client solution.

The files in the solution are:

AcceptStrictAttribute

While Asp provides ProduceAttribute and ConsumesAttribute, it does not have an action which actually checks and forces the Accept header of a request.

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class AcceptStrictAttribute : Attribute, IActionFilter
{
    private const string AcceptKey = "Accept";

    public AcceptStrictAttribute(params string[] acceptValues)
    {
        AcceptValues = acceptValues;
    }

    public string[] AcceptValues { get; }

    public void OnActionExecuted(ActionExecutedContext context)
    {
    }

    public void OnActionExecuting(ActionExecutingContext context)
    {
        if (!context.HttpContext.Request.Headers.ContainsKey(AcceptKey))
        {

            context.Result = new ObjectResult("The request does not contain the header Accept.")
            {
                StatusCode = (int)HttpStatusCode.NotAcceptable
            };

            return;
        }

        List values = context.HttpContext.Request.Headers[AcceptKey].ToList();

        bool found = false;

        foreach (string acceptV in AcceptValues)
        {
            if (values.Contains(acceptV))
            {
                found = true;
                break;
            }
        }

        if (!found)
        {
            context.Result = new ObjectResult($"Could not find a valid Accept Header value. Valid : [{string.Join(",", AcceptValues)}]")
            {
                StatusCode = (int)HttpStatusCode.NotAcceptable
            };
        }
    }
}

MessagePackInputFormatter

This formatter is used in the api solution and handles deserializing incoming models

public class MessagePackInputFormatter : InputFormatter
{
    public MessagePackInputFormatter(params string[] mediaTypes)
    {
        foreach (string mt in mediaTypes)
        {
            SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse(mt));
        }
    }
    protected override bool CanReadType(Type type)
    {
        return true;
    }

    public override async Task ReadRequestBodyAsync(InputFormatterContext context)
    {
         try
            {
                MemoryStream memoryStream = new MemoryStream();
                await context.HttpContext.Request.Body.CopyToAsync(memoryStream).ConfigureAwait(false);
                memoryStream.Seek(0, SeekOrigin.Begin);

                if (memoryStream.Length == 0)
                {
                    return InputFormatterResult.NoValue();
                }

                object model = await MessagePackSerializer
                    .DeserializeAsync(context.ModelType, memoryStream)
                    .ConfigureAwait(false);

                return InputFormatterResult.Success(model);
            }
            catch (Exception )
            {
                //TODO : Log
                return InputFormatterResult.Failure();
            }
    }
}

MessagePackOutputFormatter

This formatter is used in the api solution and handles serializing outgoing models

  
public sealed class MessagePackOutputFormatter : OutputFormatter
{
    public MessagePackOutputFormatter(params string[] mediaTypes)
    {
        foreach (string mt in mediaTypes)
        {
            SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse(mt));
        }
    }

    protected override bool CanWriteType(Type type)
    {
        return true;
    }

    
    public override async Task WriteResponseBodyAsync(OutputFormatterWriteContext context)
    {
          try
            {
                if (context.Object == null)
                {
                    byte[] nullMP = new byte[] { MessagePackCode.Nil };
                    await context.HttpContext.Response.Body.WriteAsync(nullMP);
                }
                else
                {
                    byte[] bytes = MessagePackSerializer.Serialize(context.Object);

                    await context.HttpContext.Response.Body.WriteAsync(bytes);
                }
            }
            catch (Exception ex)
            {
                context.HttpContext.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
                await context.HttpContext.Response.Body.WriteAsync(Encoding.UTF8.GetBytes(ex.Message));
            }
    }
}

MessagePackMediaTypeFormatter

This formatter is used in the client solutions and along with the ReadAsAsync extension method it handles both serializing and deserializing of models.

public sealed class MessagePackMediaTypeFormatter : MediaTypeFormatter
{
    public MessagePackMediaTypeFormatter(string mediaType)
    {
        this.SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse(mediaType));
    }
    public override bool CanReadType(Type type)
    {
        return true;
    }

    public override bool CanWriteType(Type type)
    {
         return true;
    }

    public override async Task<object> ReadFromStreamAsync(Type type, Stream readStream, HttpContent content, IFormatterLogger formatterLogger)
    {
        if (readStream.Length == 0)
            {
                if (type.IsValueType && Nullable.GetUnderlyingType(type) == null)
                    return Activator.CreateInstance(type);
                else
                    return null;
            }
            else
            {
                object model = await MessagePackSerializer
                   .DeserializeAsync(type, readStream)
                   .ConfigureAwait(false);

                return model;
            }
    }

    public override async Task WriteToStreamAsync(Type type, object value, Stream writeStream, HttpContent content, TransportContext transportContext)
    {
      byte[] bytes = MessagePackSerializer.Serialize(value);
      
            if (value == null)
            {
                byte[] nullMP = new byte[] { MessagePackCode.Nil };
                using MemoryStream memoryStream = new MemoryStream(nullMP);
                memoryStream.Seek(0, SeekOrigin.Begin);
                memoryStream.Position = 0;
                await memoryStream.CopyToAsync(writeStream).ConfigureAwait(false);
            }
            else
            {
                using MemoryStream memoryStream = new MemoryStream(bytes);
                memoryStream.Seek(0, SeekOrigin.Begin);
                memoryStream.Position = 0;
                await memoryStream.CopyToAsync(writeStream).ConfigureAwait(false);
            }
    }
}

The solution has the following dependencies:

The MessagePackAnalyzer helps you catch errors before compiling. I am using it here because it notifies when there is a missing KeyAttribute in the model.The files are:

I decorated the model with the attributes required by message pack:

[MessagePackObject]
public class WeatherForecast
{
    [Key(0)]
    public DateTime Date { get; set; }

    [Key(1)]
    public int TemperatureC { get; set; }

    [Key(2)]
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);

    [Key(3)]
    public string Summary { get; set; }
}

And I also prefer to keep a tab over the media type format in the same solution since it’s used to describe a model schema:

public static class Constants
{
    public static class MediaTypes
    {
        public static class Output
        {
            public const string WeatherMPack = "application/vnd.theweather.weatherforecast.v1+mpack";
        }
    }
}

This is the api project in our solution. It provides the endpoints to retrieve the weather forecast and the models are first packed and then sent to the client.The files are:In the Startup.cs file, where the services are configured we also add the controller services. Here is where we set up the Input and Output formatters. In this example I am not removing the existing formatters, am only adding our new ones.The controller receives 2 new attributes :

• ProducesAttribute (this sets up the ContentType of the response)
• AcceptStrict (this validates the AcceptHeader). Note that while the RespectBrowserAcceptHeader might work, it does not take in the account the situation where the client does not pass an Accept key in which case it does not block the execution, our custom attribute fixes that.

We can now run the api and send a get request with postman. We should receive a packed byte array. Don’t forget to add the Accept Header

This solution is simple, and i know some of you might argue that the client should not be directly linked with the api (I am using the same models). But i don’t really see a reason why not to do this.The solution has the following dependencies:The files are:There are 2 important constructs in the client.

• The HttpClient is passed through the constructor, this is important because we will pass a client in the integration test
• The Content is read with the ReadAsAsync extension method and the media type formatter

public class WeatherForecastClient
{
    private readonly HttpClient _httpClient;

    public WeatherForecastClient(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<WeatherForecast[]> GetAsync()
    {
        try
        {
            HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, "WeatherForecast");

            requestMessage.Headers.Add("Accept", Constants.MediaTypes.Output.WeatherMPack);

            HttpResponseMessage response = await _httpClient.SendAsync(requestMessage).ConfigureAwait(false);

            if (response.IsSuccessStatusCode)
            {
                var mediaTypeFormatter = new MediaTypeFormatter[]
                {
                    new MessagePackMediaTypeFormatter(Constants.MediaTypes.Output.WeatherMPack)
                };

                WeatherForecast[] forecasts = await response
                    .Content
                    .ReadAsAsync<WeatherForecast[]>(mediaTypeFormatter)
                    .ConfigureAwait(false);

                return forecasts;
                    
            }
            else if (response.StatusCode == System.Net.HttpStatusCode.InternalServerError)
            {
                string content = await response.Content.ReadAsStringAsync().ConfigureAwait(false);

                if (!string.IsNullOrEmpty(content))
                {
                    throw new Exception(content);
                }
                else
                {
                    throw new Exception(response.ReasonPhrase);
                }
            }
            else
            {
                //TODO :Return a response instead of the object
                throw new Exception(response.ReasonPhrase);
            }
        }
        catch (Exception)
        {
            //LOG
            throw;
        }
    }
}

To test the controller and the client, I am using the WebApplicationFactory which bootstraps the application in memory. The factory also provides the HttpClient needed to deliver messages to the apiThe dependencies are:The MessagePackAnalyzer helps you catch errors before compiling. I am using it here because it notifies when there is a missing KeyAttribute in the model.The files are:We can now use the Client method and test the controller: