Upgrade to Yarn v4

[caponetto]

We are currently using Yarn v3.5 in this repository. However, the latest stable version is Yarn v4.9.1, which includes performance improvements, bug fixes, and new features that could benefit our development workflow.

[lresende]

We must be compatible with the yarn version supported by JupyterLab, and ideally, use the jlpm bundled with JupyterLab to avoid incompatibilities...

[caponetto]

I see, thanks for the heads up @lresende. There's an interesting discussion here btw. Also, JL is still using Yarn v3.5 (ref). I think we can close this issue to avoid confusion and create a new one as soon as JL starts to use a new Yarn version.