More markers than posts visible on clustered maps

[bb]

Describe the bug

I think there's an issue with fetching markers for the map. It's easily verified with deployments which have clustering enabled, I didn't bother counting on those which don't use clustering.

There are more markers on the map than there are posts in the database. When clustering is enabled, you just need to zoom out, so there's only one cluster left. This one cluster should have the total number of posts (minus those which don't have location data).

Where was the bug observed * https://covid-brasil.ushahidi.io/views/map -- Source Web: 5307 -- single cluster when zoomed out: 5400 (i.e. 93 more than actually exist).

• https://redaktor.ushahidi.io/views/map -- Source Web: 6037, Twitter: 211, minues not shown: 615 -> 5633 should be on the map. But when I zoom out, I see 3 clusters: 5234+554+12=5800.

and others as well.

To Reproduce

Steps to reproduce the behavior: 1. Go to ' https://covid-brasil.ushahidi.io/views/map' 2. (no login needed) 3. Click on '-' in the lower left corner of the map multiple times until all markers on the map are aggregated to one cluster (or until the minimum zoom level is reached). 4. Compare the number in the cluster with the sum posts of all the sources on the left.

Expected behavior Numbers should match.

Is there a workaround? What is it. I'm not aware of any.

Screenshots https://user-images.githubusercontent.com/4932/91698041-61ede080-eb72-11ea-9d3e-774054e7ae3e.png

URL / Environment where this happened https://covid-brasil.ushahidi.io/views/map

If the bug is datasource related (e.g. SMS or Targeted SMS), please specify the configuration details (e.g. Frontline + Clickatell, etc):

Desktop Hardware Details (please complete the following information): * MacOS Chrome (latest version) * MacOS Safari (latest version) * Windows 10 Chrome (latest version) * Windows 10 Edge (latest version)

Smartphone Hardware Details (please complete the following information): * Iphones (iOS 13.6.1 on all), Safari

Aha! Link: https://ushahiditeam.aha.io/features/PROD-88

[bb]

I made some interesting observations:

1. The total on the map count is always a multiple of 200 (i.e. 5000, 5200, 5400, 5600, 5800) etc... and interestingly, the limit/offset on the geojson queries is exactly this (200).
2. Many geojson calls are made twice, others not at all: The first 5 requests (offsets 0, 200, 400, 600, 800) are made exactly once. The next 5 are made exactly twice (offsets 1000, 1000, 1200, 1200, 1400, 1400, 1600, 1600, 1800, 1800) and the next ones are done four times (or less, depending on the total of posts (offsets 2000, 2000, 2000, 2000, 2200, 2200, 2400, 2400)...

Now let's have a look at the loadPosts function: https://github.com/ushahidi/platform-client/blob/d2a6e414edf8298433057c8272ba12ed3377ed1e/app/main/posts/views/post-view-map.directive.js#L154. It depends (among other things) on the limit (200) defined in https://github.com/ushahidi/platform-client/blob/d2a6e414edf8298433057c8272ba12ed3377ed1e/app/main/posts/views/post-view-map.directive.js#L20 and on the requestBlockSize (5) which is defined one line after the limit.

So I recommend having a closer look at the loadPosts function and the way it calls itself recursively (line 194) and the calculations/conditions leading to the recursive call (lines 181 to 192).