Stack usage

[laurgarn]

With a big network of data, json-io may take a lot of stack space both at writing and reading.
For example, I have to allocate 2 Mio of stack in order to read and/or write a json.gz of 50 Mio on disk.
This minimum amount varies impredictably.
I have crafted a compatible writer on a branch that is happy with just 256 kio of stack space, by trying to limit depth first strategy.
It is noticeably slower but still usable (17s vs 10s, and 18s to parse, no change) .
Topping on the cake, it can output a far more human friendly json when formatted.
This is not an advocacy to use my solution, but allowing a "least deep strategy" would be a nice plus.

[jdereg]

I'm interested in seeing what you have created.

[laurgarn]

https://github.com/laurgarn/json-io/tree/write_less_deep_over_master
Kinda POC, especially because it adds a ObjectWriter totally aside JsonWriter.
I hope it will help you design something more, hum, polished than what I ended up with.

[jdereg]

The “stack” issue is relevant to both reading and writing. The parsing attempts to minimize stack depth, however, it does use recursion. I plan to release a version that eliminates the recursion using a graph traversal with a ‘visited’ set for cycle detection. I don’t have an ETA, but that is on my radar.

The same is true for writing JSON, and the same approach would eliminate the stack usage.

[laurgarn]

You're right. I forgot to insist on the gain at reading side too.
Nice to know it is on your radar :-)