New IBufferWriter<byte>.AsStream() extension

[Sergio0694]

PR Type

What kind of change does this PR introduce?

• Feature

What is the current behavior?

There is currently no way to interoperate between the IBufferWriter<T> interface and the Stream class. Many APIs in the BCL and in 3rd party libraries use Stream as the standard way to accept an instance that can be written to or read from, and there is no built-in way to have a memory stream that is also using memory pooling, because none of the types in the BCL and in the HighPerformance package currently support both features at the same time. This PR fixes that 😄🚀

Consider this example that I saw from a user in the C# Discord server:

public byte[] Compress(byte[] source)
{
    MemoryStream output = new MemoryStream();
    using (DeflateStream dstream = new DeflateStream(output, CompressionLevel.Optimal))
    {
        dstream.Write(source, 0, source.Length);
    }
    return output.ToArray();
}

public byte[] Decompress(byte[] source)
{
    MemoryStream input = new MemoryStream(source);
    MemoryStream output = new MemoryStream();
    using (DeflateStream dstream = new DeflateStream(input, CompressionMode.Decompress))
    {
        dstream.CopyTo(output);
    }
    return output.ToArray();
}

You can see how the code is very memory inefficient: the MemoryStream type will just new-up arrays as it goes, and at the end ToArray() is used too, which will duplicate the arrays too. Even by removing that, the main issue within MemoryStream remains. With the new extension introduced in this PR, these two APIs can be rewritten much more efficiently, like this:

public IMemoryOwner<byte> Compress(ReadOnlySpan<byte> span)
{
    ArrayPoolBufferWriter<byte> bufferWriter = new ArrayPoolBufferWriter<byte>();

    using DeflateStream deflateStream = new DeflateStream(bufferWriter.AsStream(), CompressionLevel.Optimal);

    deflateStream.Write(span);

    return bufferWriter;
}

public IMemoryOwner<byte> Decompress(ReadOnlyMemory<byte> memory)
{
    ArrayPoolBufferWriter<byte> bufferWriter = new ArrayPoolBufferWriter<byte>(memory.Length);

    using DeflateStream deflateStream = new DeflateStream(memory.AsStream(), CompressionMode.Decompress);

    deflateStream.CopyTo(bufferWriter.AsStream());

    return bufferWriter;
}

Which heavily leverages all the various APIs and helpers in the HighPerformance package, and gives us the following results:

Method	Categories	Mean	Error	StdDev	Ratio	Gen 0	Gen 1	Gen 2	Allocated
new[]	COMPRESS	29,923.5 us	174.19 us	162.94 us	1.00	312.5000	312.5000	312.5000	3089853 B
pool	COMPRESS	29,116.0 us	120.55 us	106.87 us	0.97	-	-	-	297 B
new[]	DECOMPRESS	832.9 us	9.96 us	8.83 us	1.00	337.8906	336.9141	336.9141	2966680 B
pool	DECOMPRESS	119.6 us	0.70 us	0.62 us	0.14	-	-	-	392 B

This benchmark compresses and decompresses a 1MB buffer, using the two methods detailed above.
You can see the vastly reduced memory allocations using the pooled writer backed stream 🚀

What is the new behavior?

This PR introduces this new extension:

namespace Microsoft.Toolkit.HighPerformance.Extensions
{
    public static class ArrayPoolBufferWriterExtensions
    {
        public static Stream AsStream(this ArrayPoolBufferWriter<byte> writer);
    }

    public static class IBufferWriterExtensions
    {
        public static Stream AsStream(this IBufferWriter<byte> writer);
    }
}

Which helps to interoperate between the IBufferWriter<T> interface and the Stream class. In particular, since the HighPerformance package includes the ArrayPoolBufferWriter<T> type, this extension allows users to use that as a Stream, and then keep working with the resulting ReadOnlyMemory<T> produced by that type, as shown above.

PR Checklist

Please check if your PR fulfills the following requirements:

• Tested code with current supported SDKs
• Pull Request has been submitted to the documentation repository instructions. Link:
• Sample in sample app has been added / updated (for bug fixes / features)
  • Icon has been created (if new sample) following the Thumbnail Style Guide and templates
• Tests for the changes have been added (for bug fixes / features) (if applicable)
• Header has been added to all new source files (run build/UpdateHeaders.bat)
• Contains NO breaking changes

[ghost]

Thanks Sergio0694 for opening a Pull Request! The reviewers will test the PR and highlight if there is any conflict or changes required. If the PR is approved we will proceed to merge the pull request 🙌

[Sergio0694]

Removed "do not merge" tag as #3477 is now merged 🚀

[ghost]

Hello @michael-hawker!

Because this pull request has the auto merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

[Kyaa-dost]

@Sergio0694 the build seems to be failing could you try updating the branch one more time?

[Sergio0694]

@Kyaa-dost Ah, thanks for the ping! That error was caused by #3510 being merged, which bumped the System.Runtime.CompilerServices.Unsafe package to version 5.0.0. There's a small bug in that release where Unsafe.As<T>(object) is missing the [NotNullWhenNotNull] annotation for the return value on .NET Standard 2.0 (I fixed that in dotnet/runtime#42827, but that didn't make it in time for the final 5.0.0 release, sadly). Because of that, the compiler was not tracking nullability correctly in an extension method and showing a warning that shouldn't have been there (as it was right after a null check on that same variable). Suppressed using ! for now, should work just fine now 😄