- Option 1 (preferred): Go to Project Settings>Package Manager. Add a scoped registry with https://registry.npmjs.org as the url. Add com.cysharp.utf8json in the scope list. Then go to Package Manager, next to the + button click change from Unity Registry to In Project package view and install
- Option 2: In the Unity Package Manager, click the + button, select Add package from git URL and enter https://www.github.com/adrenak/utf8json.git@upm
Definitely Fastest and Zero Allocation JSON Serializer for C#(.NET, .NET Core, Unity and Xamarin), this serializer write/read directly to UTF8 binary so boostup performance. And I adopt the same architecture as the fastest binary serializer, MessagePack for C# that I've developed.
> This benchmark is convert object to UTF8 and UTF8 to object benchmark. It is not to string(.NET UTF16), so Jil, NetJSON and Json.NET contains additional UTF8.GetBytes/UTF8.GetString call. Definitely means does not exists encoding/decoding cost. Benchmark code is in sandbox/PerfBenchmark by BenchmarkDotNet.
> I've tested more benchmark - Benchmark of Jil vs Utf8Json for test many dataset patterns(borrwed from Jil Benchmark) and three input/output compare(Object <-> byte[](Utf8), Object <-> Stream(Utf8) and Object <-> String(UTF16)). If target is UTF8(both byte[] and Stream), Utf8Json wins and memory allocated is extremely small.
Utf8Json does not beat MessagePack for C#(binary), but shows a similar memory consumption(there is no additional memory allocation) and achieves higher performance than other JSON serializers.
The crucial difference is that read and write directly to UTF8 binaries means that there is no overhead. Normaly serialization requires serialize to Stream or byte[], it requires additional UTF8.GetBytes cost or StreamReader/Writer overhead(it is very slow!).
``csharp
TargetClass obj1;
// Object to UTF8 byte[]
[Benchmark]
public byte[] Utf8JsonSerializer()
{
return Utf8Json.JsonSerializer.Serialize(obj1, jsonresolver);
}
// Object to String to UTF8 byte[]
[Benchmark]
public byte[] Jil()
{
return utf8.GetBytes(global::Jil.JSON.Serialize(obj1));
}
// Object to Stream with StreamWriter
[Benchmark]
public void JilTextWriter()
{
using (var ms = new MemoryStream())
using (var sw = new StreamWriter(ms, utf8))
{
global::Jil.JSON.Serialize(obj1, sw);
}
}
`
For example, the OutputFormatter of ASP.NET Core needs to write to Body(Stream), but using Jil's TextWriter overload is slow. (This not means Jil is slow, for example StreamWriter allocate many memory(char[1024] and byte[3075]) on constructor (streamwriter.cs#L203-L204) and other slow features unfortunately).
`csharp
// ASP.NET Core, OutputFormatter
public class JsonOutputFormatter : IOutputFormatter //, IApiResponseTypeMetadataProvider
{
const string ContentType = "application/json";
static readonly string[] SupportedContentTypes = new[] { ContentType };
public Task WriteAsync(OutputFormatterWriteContext context)
{
context.HttpContext.Response.ContentType = ContentType;
// Jil, normaly JSON Serializer requires serialize to Stream or byte[].
using (var writer = new StreamWriter(context.HttpContext.Response.Body))
{
Jil.JSON.Serialize(context.Object, writer, _options);
writer.Flush();
return Task.CompletedTask;
}
QuickStart, you can call Utf8Json.JsonSerializer.Serialize/Deserialize.
`csharp
var p = new Person { Age = 99, Name = "foobar" };
// Object -> byte[] (UTF8)
byte[] result = JsonSerializer.Serialize(p);
// byte[] -> Object
var p2 = JsonSerializer.Deserialize(result);
// Object -> String
var json = JsonSerializer.ToJsonString(p2);
// Write to Stream
JsonSerializer.Serialize(stream, p2);
`
In default, you can serialize all public members. You can customize serialize to private, exclude null, change DateTime format(default is ISO8601), enum handling, etc. see the Resolver section.
Performance of Serialize
---
This image is what code is generated when object serializing.
`csharp
// Disassemble generated serializer code.
public sealed class PersonFormatter : IJsonFormatter {
private readonly byte[][] stringByteKeys;
public PersonFormatter()
{
// pre-encoded escaped string byte with "{", ":" and ",".
this.stringByteKeys = new byte[][]
{
JsonWriter.GetEncodedPropertyNameWithBeginObject("Age"), // {\"Age\":
JsonWriter.GetEncodedPropertyNameWithPrefixValueSeparator("Name") // ,\"Name\":
};
}
public sealed Serialize(ref JsonWriter writer, Person person, IJsonFormatterResolver jsonFormatterResolver)
{
if (person == null) { writer.WriteNull(); return; }
// WriteRawX is optimize byte->byte copy when we know src size.
UnsafeMemory64.WriteRaw7(ref writer, this.stringByteKeys[0]);
writer.WriteInt32(person.Age); // itoa write directly to avoid ToString + UTF8 encode
UnsafeMemory64.WriteRaw8(ref writer, this.stringByteKeys[1]);
writer.WriteString(person.Name);
writer.WriteEndObject();
}
// public unsafe Person Deserialize(ref JsonReader reader, IJsonFormatterResolver jsonFormatterResolver)
}
`
Object to JSON's main serialization cost is write property name. Utf8Json create cache at first and after that only do memory copy. Optimize part1, concatenate "{", ":" and "." to cached propertyname. Optimize part2, use optimized custom memory copy method(see: UnsafeMemory.cs). Normally memory copy is used Buffer.BlockCopy but it has some overhead when target binary is small enough, releated to dotnet/coreclr - issue #9786 Optimize Buffer.MemoryCopy and [dotnet/coreclr - Add a fast path for byte[] to Buffer.BlockCopy #3118](https://github.com/dotnet/coreclr/pull/3118). Utf8Json don't use Buffer.BlockCopy and generates length specialized copy code that can reduce branch cost.
Number conversion is often high cost. If target encoding is UTF8 only, we can use itoa algorithm so avoid int.ToString and UTF8 encode cost. Especialy double-conversion, Utf8Json ported google/double-conversion algorithm, it is fast dtoa and atod works.
Other optimize techniques.
* High-level API uses internal memory pool, don't allocate working memory under 64K
* Struct JsonWriter does not allocate any more and write underlying byte[] directly, don't use TextWriter
* Avoid boxing all codes, all platforms(include Unity/IL2CPP)
* Heavyly tuned dynamic IL code generation, it generates per option so reduce option check: see:DynamicObjectResolver.cs * Call Primitive API directly when IL code generation knows target is primitive
* Getting cached generated formatter on static generic field(don't use dictionary-cache because lookup is overhead)
* Don't use IEnumerable abstraction on iterate collection, specialized each collection types, see:CollectionFormatter.cs * Uses optimized type key dictionary for non-generic methods, see: ThreadsafeTypeKeyHashTable.cs
Performance of Deserialize
---
When deserializing, requires property name to target member name matching. Utf8Json avoid string key decode for matching, generate automata based IL inlining code.
use raw byte[] slice and try to match each ulong type (per 8 character, if it is not enough, pad with 0).
`csharp
// Disassemble generated serializer code.
public sealed class PersonFormatter : IJsonFormatter {
// public sealed Serialize(ref JsonWriter writer, Person person, IJsonFormatterResolver jsonFormatterResolver)
public unsafe Person Deserialize(ref JsonReader reader, IJsonFormatterResolver jsonFormatterResolver)
{
if (reader.ReadIsNull()) return null;
reader.ReadIsBeginObjectWithVerify(); // "{"
byte[] bufferUnsafe = reader.GetBufferUnsafe();
int age;
string name;
fixed (byte* ptr2 = &bufferUnsafe[0])
{
int num;
while (!reader.ReadIsEndObjectWithSkipValueSeparator(ref num)) // "}" or ","
{
// don't decode string, get raw slice
ArraySegment arraySegment = reader.ReadPropertyNameSegmentRaw();
byte* ptr3 = ptr2 + arraySegment.Offset;
int count = arraySegment.Count;
if (count != 0)
{
// match automata per long
ulong key = AutomataKeyGen.GetKey(ref ptr3, ref count);
if (count == 0)
{
if (key == 6645569uL)
{
age = reader.ReadInt32(); // atoi read directly to avoid GetString + int.Parse
continue;
}
if (key == 1701667150uL)
{
name = reader.ReadString();
continue;
}
}
}
reader.ReadNextBlock();
}
}
return new PersonSample
{
Age = age,
Name = name
};
}
}
`
Of course number conversion(decode to string -> try parse) is high cost. Utf8Json directly convert byte[] to number by atoi/atod algorithm.
Built-in support types
---
These types can serialize by default.
Primitives(int, string, etc...), Enum, Nullable<>, TimeSpan, DateTime, DateTimeOffset, Guid, Uri, Version, StringBuilder, BitArray, Type, ArraySegment<>, BigInteger, Complext, ExpandoObject , Task, Array[], Array[,], Array[,,], Array[,,,], KeyValuePair<,>, Tuple<,...>, ValueTuple<,...>, List<>, LinkedList<>, Queue<>, Stack<>, HashSet<>, ReadOnlyCollection<>, IList<>, ICollection<>, IEnumerable<>, Dictionary<,>, IDictionary<,>, SortedDictionary<,>, SortedList<,>, ILookup<,>, IGrouping<,>, ObservableCollection<>, ReadOnlyOnservableCollection<>, IReadOnlyList<>, IReadOnlyCollection<>, ISet<>, ConcurrentBag<>, ConcurrentQueue<>, ConcurrentStack<>, ReadOnlyDictionary<,>, IReadOnlyDictionary<,>, ConcurrentDictionary<,>, Lazy<>, Task<>, custom inherited ICollection<> or IDictionary<,> with paramterless constructor, IEnumerable, ICollection, IList, IDictionary and custom inherited ICollection or IDictionary with paramterless constructor(includes ArrayList and Hashtable), Exception and inherited exception types(serialize only) and your own class or struct(includes anonymous type).
Utf8Json has sufficient extensiblity. You can add custom type support and has some official/third-party extension package. for example ImmutableCollections(ImmutableList<>, etc), Utf8Json.FSharpExtensions(FSharpOption, FSharpList, etc...). Please see extensions section.
Object Serialization
---
Utf8Json can serialze your own public Class or Struct. In default, serializer search all public instance member(field or property) and uses there member name as json property name. If you want to avoid serialization target, you can use [IgnoreDataMember] attribute of System.Runtime.Serialization to target member. If you want to change property name, you can use [DataMember(Name = string)] attribute of System.Runtime.Serialization.
`csharp
// JsonSerializer.Serialize(new FooBar { FooProperty = 99, BarProperty = "BAR" });
// Result : {"foo":99}
public class FooBar
{
[DataMember(Name = "foo")]
public int FooProperty { get; set; }
[IgnoreDataMember]
public string BarProperty { get; set; }
}
`
Utf8Json has other option, allows private/internal member serialization, convert property name to camelCalse/snake_case, if value is null does not create property. Or you can use a different DateTime format(default is ISO8601). The details, please read Resolver section. Here is sample.
Serialize ImmutableObject(SerializationConstructor)
---
Utf8Json can deserialize immutable object like this.
` public struct CustomPoint
{
public readonly int X;
public readonly int Y;
public CustomPoint(int x, int y)
{
this.X = x;
this.Y = y;
}
}
`
Utf8Json choose constructor with the most matched argument by name(ignore case).
> MessagePack for C# choose least matched argument, please be aware of the opposite. This is design miss of MessagePack for C#.
If can not match automatically, you can specify to use constructor manually by [SerializationConstructorAttribute].
`csharp
public class CustomPoint
{
public readonly int X;
public readonly int Y;
public CustomPoint(int x, int y)
{
this.X = x;
this.Y = y;
}
// used this constructor.
[SerializationConstructor]
public CustomPoint(int x)
{
this.X = x;
}
}
`
ShouldSerializeXXX pattern
---
UtfJson supports ShouldSerialize feature of Json.NET. If defined public bool ShouldMemberName() method, call method before serialize member value and if false does not output member.
`csharp
public class MyPerson
{
public string Name { get; set; }
public string[] Addresses { get; set; }
// ShouldSerializemembername*
// method must be public and return bool and parameter less.
public bool ShouldSerializeAddresses()
{
if (Addresses != null && Addresses.Length != 0)
{
return true;
}
else
{
return false;
}
}
}
--
// {"Name":"foo"}
JsonSerializer.ToJsonString(new MyPerson { Name = "foo", Addresses = new string[0] });