MaxConnectionPoolSize is not respected under heavy load

[aaronjwood]

The default internal connection pool size for this driver seems to be set at 100. I'm seeing that this can be broken. Basic test to reproduce this behavior:

    test('Connection pool', async () => {
        const driver = Neo4J.driver('bolt://localhost:7687', Neo4J.auth.basic('neo4j', 'neo4j'), {
            disableLosslessIntegers: true,
        });

        const p: Promise<any>[] = [];
        for (let i = 0; i < 1500; i++) {
            p.push(new Promise(async (res, rej) => {
                const session = driver.session();
                await session.run('MATCH (n) RETURN n');
                session.close();
                res();
            }));
        }

        await Promise.all(p);
    }, 30000);

Watch netstat while you run this test and see that the number of connections goes into the thousands:

watch -n .5 'netstat -an | grep "7687" | grep ESTABLISHED | wc -l'

I've tested this against Neo4j 3.5.3 and 3.5.12 using driver versions 1.7.5 and 4.0.0-beta01.

[zhenlineo]

#500

[zhenlineo]

Hi @aaronjwood ,
Thanks very much for reporting this to us. We are releasing a new 4.0.0-rc1 driver with the fix inside. The fix ensures the maximum active connection count is always less or equal to MaxConnectionPoolSize.

[aaronjwood]

Thanks! Any chance this can be backported to 1.7.x? I don't want to upgrade to the beta driver release.

[zhenlineo]

Hi @aaronjwood,
The change was also merged in 1.7. But I currently do not have a patch release date for next 1.7. I will let you know once I got more information.

[zhenlineo]

Hi @aaronjwood ,

The 4.0.0 js driver is fully released. It supports all 3.3+ servers. The API is slightly changed, you can read more here.

Could you verify if the problem is solved and close this issue if it is no longer the problem? Or let us know if you think there is more need to be done.
Cheers,
Zhen

[aaronjwood]

We hadn't planned on upgrading to 4.0.0 soon as the API has changed a bit and requires some work on our end to swap it in. I'll let you know if 4.0.0 fixes the issue when we upgrade to it but for now we've layered our own session pool on top of the lib to work around the issue.