Support for large connection data sets.

[Marsjohn-11]

When dealing with very large datasets (e.g., 20,000+ items), the current implementation loads the entire merged list into memory, or into storage, which could cause performance issues, out-of-memory errors, or SQLite record exceptions.

While relay's cursor based pagination may make this experience unlikely, offset-based pagination (Assuming we have some offset connection merge policy), local population, or alternative initial sync type back-fills might support these use cases.

Potential Solutions:
• Implement windowing in the cache to only keep a subset of items in memory
• Add support for pagination boundaries to limit the maximum number of items stored
• Implement a sliding window approach that discards items that are far from the current view

Key to this (from what I see) is the idea that records are merged into the initial response's first record edges|nodes fields. It seems like we may benefit from some windowed record chained above the current caches to help manage this. Curious what others thoughts are on this.

I think you've been already looking at some of this concept in #121 (comment). Sorry if this is a dupe

[BoD]

Thanks for raising!

There is definitely a potential issue with large datasets, as you point out with the current pagination system:

• on disk: pages are merged together in one record, containing a list of all items. This can potentially exceed the maximum size* of a single record if the list is huge
• in memory: lists are fully loaded into memory when they're exposed to the client. This can certainly lead to OOM for huge lists.

*: there is not really a "maximum record size", but at the moment trying to load records bigger than Android's SQLite CursorWindow limit (2MB by default) will fail.

#121 is definitely related, but is about big records in general, not specifically in regards to pagination, where there is definitely some approaches we could take to improve the story.