High Number of frameworks having problems starting

[billywhizz]

in the last run and the current one just-js and a number of other frameworks seem to have failed to start. it has never failed before and nothing has changed since round 21.

it seems the number of frameworks failing in same way has increased significantly in last round and the one currently running. is there some environmental issue causing problems maybe?

https://tfb-status.techempower.com/results/15bfed85-1093-485d-b161-5f61b7abd0be

[NateBrady23]

A few spot checks have me believing it's the postgres upgrade. @joanhey does that seem right to you?

[joanhey]

After some research, it seems correct that it affects these fails.
I have had to look at other fws, because Just doesn't give any errors.
The others return ... authentication type: 10

It's again the problem with the auth encryption. That come from postgresql v10.

New in Postgresql 14:

Change the default of the password_encryption server parameter to scram-sha-256 (Peter Eisentraut)

Previously it was md5. All new passwords will be stored as SHA256 unless this server setting is changed or the password is specified in MD5 format. Also, the legacy (and undocumented) Boolean-like values which were previously synonyms for md5 are no longer accepted.

Now I'll send a PR.

Curiously it only affects some fws.

By setting password_encryption to scram-sha-256 you also get scram-sha-256 authentication, even if you have md5 in pg_hba.conf.

https://www.postgresql.org/docs/current/auth-password.html

[billywhizz]

@joanhey is it possible to change the default back to MD5 in your database scripts? seems like it should be: https://www.postgresql.org/docs/14/runtime-config-connection.html#GUC-PASSWORD-ENCRYPTION

[joanhey]

Scram-sha-256 was introduced in Postgesql v10. (October 5, 2017)
And it's the recommended method for a most secure connection. And any new PG driver need to support it.

I changed the default to md5 again in postgresql.conf in the PR.
And Just-js is working again, and the rest.

Historically this bench use md5, because at the time of this change, most pg drivers didn't support scram-sha-256. And still now after the last run.
Something similar happens with Mysql 8.

[billywhizz]

thanks @joanhey - i will look at adding scram-sha-256 support to the postgres driver for just-js.

[joanhey]

A new think in Postgresql 14 is pipeline mode in libpg, and also work with older servers versions.

Some fws in this bench use hand made versions (just-js, drogon, ...).

What distinguishes Pipeline Mode is that it provides an out-of-the-box solution greatly reducing the application code’s complexity handling the client-server session.

https://www.postgresql.org/docs/current/libpq-pipeline-mode.html
https://www.percona.com/blog/how-postgresql-pipeline-mode-works/
https://www.cybertec-postgresql.com/en/pipeline-mode-better-performance-on-slow-network/

And some discussion about these hand made versions, exist in this repo.

PD I hope that more pg drivers start to add the pipeline mode.

[billywhizz]

PD I hope that more pg drivers start to add the pipeline mode.

@joanhey i don't understand this comment when a number of frameworks were already doing pipelining on previous postgres protocol but were told it wasn't allowed? somebody needs to clarify this.

[joanhey]

@billywhizz
I was talking about pg drivers, not about the benchmark.

And remember that pipeline is not for use in all apps.

Since each SQL statement is essentially idempotent it is up to the client logic to make sense of the results. Sending SQL statements and pulling out results that have no relation with each other is one thing but life gets more complicated when working with logical outcomes that have some level of interdependence.

It is possible to bundle asynchronous SQL statements as a single transaction. But as with all transactions, failure of any one of these asynchronously sent SQL statements will result in a rollback for all the SQL statements.

Of course, the API does provide error handling in the case of pipeline failures. In the case of a FATAL condition, when the pipeline itself fails, the client connection is informed of the error thus flagging the remaining queued operations as lost. Thereafter normal processing is resumed as if the pipeline was explicitly closed by the client, and the client connection remains active.

[billywhizz]

@joanhey i still don't understand the implication of what you are saying. i raised the point that this is officially supported behaviour in postgres wire protocol already here
and we were told it is not allowed.

so, is pipelining allowed or not going forward?