Fix 2129 - CUDA required dependencies

[bhashemian]

related to #2129

Description

This PR adds cupy dependency (which requires CUDA), and modifies CI/CD workflow to exclude the dependencies that requires CUDA for non-CUDA instances with full dependency installation (quick-py3).

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[bhashemian]

I am adding the CUDA version specific binaries.

[wyli]

the installation is not straightforward here... How would a pip install user get this cupy feature?

[bhashemian]

the installation is not straightforward here... How would a pip install user get this cupy feature?

@wyli
A user can simply install it from source by pip install cupy==9.0.0, which is part of requirements-dev.txt. If it's preferred, I can leave it to be the same for our CI/CD workflow. The reason that I wanted to use CUDA-version-specific binaries is to speed up the workflow. Does it make sense?

[wyli]

the installation is not straightforward here... How would a pip install user get this cupy feature?

@wyli
A user can simply install it from source by pip install cupy==9.0.0, which is part of requirements-dev.txt. If it's preferred, I can leave it to be the same for our CI/CD workflow. The reason that I wanted to use CUDA-version-specific binaries is to speed up the workflow. Does it make sense?

thanks, it makes sense. is it possible to skip the requirement based on system info, such as:

MONAI/requirements-dev.txt

Line 33 in 45c6cf6

cucim~=0.19.0; platform_system == "Linux"

otherwise this requirements file always requires Cuda... it's too restrictive. another option is to have requirements-dev-cpu.txt and requirements-dev.txt? cc @rijobro @ericspod

[bhashemian]

the installation is not straightforward here... How would a pip install user get this cupy feature?

@wyli
A user can simply install it from source by pip install cupy==9.0.0, which is part of requirements-dev.txt. If it's preferred, I can leave it to be the same for our CI/CD workflow. The reason that I wanted to use CUDA-version-specific binaries is to speed up the workflow. Does it make sense?

thanks, it makes sense. is it possible to skip the requirement based on system info, such as:

MONAI/requirements-dev.txt

Line 33 in 45c6cf6

cucim~=0.19.0; platform_system == "Linux"

otherwise this requirements file always requires Cuda... it's too restrictive. another option is to have requirements-dev-cpu.txt and requirements-dev.txt? cc @rijobro @ericspod

Here the condition is based on GPU availability and CUDA version and I haven't seen any environment marker that identifies that. Do you know any?

I like the idea of having cpu and gpu-specific requirements.

[bhashemian]

@rijobro @ericspod @wyli
There are two other PR's that depends on the decision here to move forward. Can we agree on an interim solution to proceed? Thanks.

[bhashemian]

@wyli would it be fine to move forward with this?

[wyli]

no sure what the best way forward is, how about we keep the original requirements-dev.txt, and only change the CI/CD yml, so that we get the basic CI/CD for the cupy related?

[rijobro]

We have requirements.txt, requirements-min.txt, requirements-dev.txt. Should we introduce a requirements-cuda.txt? It would be cleaner to include a file when needed rather than remove individual packages when not needed.

[wyli]

I feel this needs more discussions -- we also have docs/requirements.txt, and there are some requirements only for type hints/code formatting. perhaps we should have a requirements folder and with all the different files

[ericspod]

We can hotfix the CI/CD yaml files for now and come back to what to do about the requirements files. It's getting out of hand to have multiple files for different configurations, users won't find it obvious what they need to install. Perhaps there's a more involved requirements management process we can use, maybe something like poetry?

[rijobro]

We could also have a script that will call the relevant requirement files? You know, like ./install_dependencies.py --cuda or ./install_dependencies.py --min.

[wyli]

Hi @behxyz We discussed this issue on Friday, the conclusion is to keep the requirements-*.txt unchanged for now, we'll go through the other OSS to find a good solution for the deps.

[wyli]

closing this in favour of #2858 #2849