StackOverflow when Serializing with BotFramework Activity Attachment

[rido-min]

When using AdaptiveCards in an Activity.Attachment, the serializer throws an StackOverflow exception

Found this bug when updating to latest BotFramework version, 4.23.1, that requires AdaptiveCards 3.1.0

This error does not repro using

<PackageReference Include="AdaptiveCards" Version="2.7.3" />
<PackageReference Include="Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.18.1" />

Since BotFramework is Archived, and it's no longer receiving updates, I was wondering if there is a way to fix the issue by updating AdaptiveCards.

Repro

// <PackageReference Include="Microsoft.Bot.Builder.Integration.AspNet.Core" Version="4.23.1" />

using AdaptiveCards;
using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Integration.AspNet.Core.Handlers;
using Microsoft.Bot.Schema;
using Newtonsoft.Json;


var card = new AdaptiveCard(new AdaptiveSchemaVersion(1, 2));
var cardAttachment = new Attachment() { ContentType = AdaptiveCard.ContentType, Content = card };
var reply = MessageFactory.Attachment([cardAttachment]);


using var sw = new StringWriter();
using var writer = new JsonTextWriter(sw);
BotMessageHandlerBase.BotMessageSerializer.Serialize(writer, reply);
var json = sw.ToString();

Console.WriteLine(json);

Error

   at Microsoft.Bot.Schema.Converters.AttachmentMemoryStreamConverter.HasMemoryStream(System.Object)
   at Microsoft.Bot.Schema.Converters.AttachmentMemoryStreamConverter.WriteJson(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.JsonSerializer)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(Newtonsoft.Json.JsonWriter, Newtonsoft.Json.JsonConverter, System.Object, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonObjectContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(Newtonsoft.Json.JsonWriter, System.Collections.IEnumerable, Newtonsoft.Json.Serialization.JsonArrayContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonObjectContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(Newtonsoft.Json.JsonWriter, System.Object, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(Newtonsoft.Json.JsonWriter, System.Object, System.Type)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(Newtonsoft.Json.JsonWriter, System.Object, System.Type)
   at Newtonsoft.Json.JsonSerializer.Serialize(Newtonsoft.Json.JsonWriter, System.Object)
   at Program.<Main>$(System.String[])

[npiasecki]

Well, thanks for posting this, as I thought I was nuts, and I encountered this while upgrading an old bot to .NET 10. I'm certainly not rewriting the old bot, it will just get deleted at some point. But I'd prefer it not to be today, and for it to like to work for maybe the next 3 years, hoping that the need for the old bot will go away in the meantime.

The issue is the AttachmentMemoryStreamConverter in the BotFramework is doing something bizarre to try find MemoryStream instances.

It gets triggered because the Attachment class in BotFramework is decorated for good old Newtonsoft.Json to use it as a custom JsonConverter:

    /// <summary>
    /// Gets or sets embedded content.
    /// </summary>
    /// <value>The embedded content.</value>
    [JsonProperty(PropertyName = "content")]
    [JsonConverter(typeof(AttachmentMemoryStreamConverter))]
    public object Content { get; set; }

When it encounters this line in AdaptiveCards's AdaptiveHeight, it gets stuck in a loop calling HasMemoryStream() recursively on the Auto property, and down the rabbit hole it goes until you get a stack overflow:

    public static AdaptiveHeight Auto { get; } = new AdaptiveHeight(AdaptiveHeightType.Auto);

I guess it is looking for memory streams. Honestly, I have no idea what it is doing. I don't use any memory streams in my cards, so I just wanted to make the faulty converter go away entirely.

The first part is to create a class, say, called FixedContractResolver like this:

public sealed class FixedContractResolver : DefaultContractResolver
{
    protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
    {
        var prop = base.CreateProperty(member, memberSerialization);

        var attr = member.GetCustomAttribute<JsonConverterAttribute>();

        if (attr != null && attr.ConverterType.ToString().Contains("AttachmentMemoryStreamConverter"))
        {
            // Remove the messed up converter
            prop.Converter = null;

            // Force our converter, you could attach your own fixed version here
            // prop.Converter = new FixedAttachmentMemoryStreamConverter();
        }

        return prop;
    }
}

Here we are looking for anything that converter attribute set to something like AttachmentMemoryStreamConverter, we are just going to set it to null and blow it away. If you needed MemoryStream serialization for whatever reason, you could install your fixed on here.

The next trick is to figure out where to install our FixedContractResolver. I was able to figure out it is using the serialization settings in the ConnectorClient class. Where exactly this is getting instantiated I could not figure out, and by this point I was getting ticked off, so instead I just installed it in my ActivityHandler directly whenever it is called:

protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext,
    CancellationToken cancellationToken)
{
    var connectorClient = turnContext.TurnState.Get<IConnectorClient>();
    connectorClient.SerializationSettings.ContractResolver = new FixedContractResolver();
    // ... do the actual work, now that serializer is fixed

I hope this helps someone. I guess it's not really a bug in AdaptiveCards, and it's unfortunate that BotFramework is yet another example of why humans can't build code that lasts longer than 10 years. The AI will just throw it all away and rewrite it anyway...

[catmanjan]

Thanks for that @npiasecki I am in the exact same position as you now.

Just to add to this, IConnectorClient is from Microsoft.Bot.Connector and you need

using Newtonsoft.Json.Serialization;
using Newtonsoft.Json;

for FixedContractResolver

[rido-min]

one way to fix this issue was to pin BotBuilder to 4.22.3

... or avoid the AdaptiveCards package and create the cards with plain JSON, or another library like FluentCards