[2678] Add transform to fill holes and to filter

[Spenhouet]

Fixes #2678 .

Description

A few sentences describing the changes proposed in this pull request.

Status

Ready/Work in progress/Hold

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[Spenhouet]

@Nic-Ma @wyli I finalized my pull request but the merge checks throw a "import scipy.ndimage (No module named 'scipy')." and I don't get why? I used the same optional import of scipy.ndimage as it is done in monai.apps.pathology.utils.
Any idea how to fix that?

[Nic-Ma]

Hi @Spenhouet ,

Thanks for your quick PR.
For the CI test errors, please:

1. Add the unit test file to https://github.com/Project-MONAI/MONAI/blob/dev/tests/min_tests.py#L138 to skip min tests.
2. Update the type-hints according to error message: monai/transforms/post/array.py:336:13: error: Returning Any from function declared to return "ndarray[Any, Any]" [no-any-return]

Thanks.

[Spenhouet]

@Nic-Ma I don't know how to address the type hint regarding numpy. This does not seem to have been done anywhere in the repository so far. I did use the NdarrayTensor type which is already in use. This one is also not further typeable. So I really don't know how to resolve this?

[Nic-Ma]

Hi @Spenhouet ,

Maybe you can change that line to:

return np.asarray(np.where(np.isin(img, self.applied_labels), img, 0))

Thanks.

[Spenhouet]

@Nic-Ma This resolves it for mypy but then Pylance complains:

Expression of type "ndarray[Unknown, Unknown]" cannot be assigned to return type "NdarrayTensor@call"
Type "ndarray[Unknown, Unknown]" cannot be assigned to type "NdarrayTensor@call"

But since you are not linting with Pylance... this would resolve the issue I guess. Feels just like this actually a mypy issue and should be reported as bug on their side.
I will commit it like this for now.

[Spenhouet]

@Nic-Ma @ericspod I feel like the discussion on #2678 is not finished yet. As I see it, the pull request should rather implement the fill_holes7 logic since it is faster. But @ericspod mentioned that maybe someone might implement a faster solution (I guess directly in C?). Not sure if this should stay open until then or if we should try to merge an implementation of fill_holes7 first?

[wyli]

I think we can merge this initial version first and improve the performance later if needed. What do you think @Nic-Ma @ericspod? Also, 'filter' is a very generic name (and name of a python build-in function), perhaps we should get a more specific name for the transform.

[Spenhouet]

@wyli How about LabelFilter or ValueFilter?

[wyli]

Labelfilter sounds good to me

[Spenhouet]

I now changed the implementation to the "growing" logic. It is way faster than the connected component one and it becomes much faster if the filling is only done for a subset of labels.

I implemented it so that it directly works on batches and does not need to iterate over batches.

The pull request should be reviewable now.

[Spenhouet]

@Nic-Ma Something failed here but I can't see the logs of the pull request checks.

[ericspod]

We should merge this version first once we've figured out the issue with the tests. I can't see the logs either so @Spenhouet if you update the branch with changes from dev this will trigger the tests again and hopefully you'll get an error log this time.

The solution later down the line is to define a convolution operation using max rather than sum as its function or otherwise define binary morphology operators in a Pytorch-compatible way on GPU.

[wyli]

there was an issue with the github action today https://www.githubstatus.com/incidents/7p1nnvkgh96y let me rerun this pipeline

[Spenhouet]

@wyli Thanks for rerunning the pipeline. The checks passed now.

[Spenhouet]

Interestingly the performance for one-hot encoded images is way better when iterating over the encoding instead of letting the binary_dilation do it via the right window / structure.

Implementation with windows / structure:

structure = np.zeros((3, *[3] * spatial_dims))
structure[1, ...] = ndimage.generate_binary_structure(spatial_dims, connectivity or spatial_dims)

img_arr_filtered = img_arr[tuple(applied_labels), ...]
tmp = np.zeros(img_arr_filtered.shape, dtype=bool)
ndimage.binary_dilation(
    tmp,
    structure=structure,
    iterations=-1,
    mask=np.logical_not(img_arr_filtered),
    origin=0,
    border_value=1,
    output=tmp,
)
img_arr[tuple(applied_labels), ...] = np.logical_not(tmp)

Implementation with iteration:

structure = ndimage.generate_binary_structure(spatial_dims, connectivity or spatial_dims)

for label in applied_labels:
    tmp = np.zeros(img_arr.shape[1:], dtype=bool)
    ndimage.binary_dilation(
        tmp,
        structure=None,
        iterations=-1,
        mask=np.logical_not(img_arr[label]),
        origin=0,
        border_value=1,
        output=tmp,
    )
    img_arr[label] = np.logical_not(tmp)

	structure	iteration
2 label	73s	1.4s
all labels	134s	31s

I will therefore implement the iteration.

[Spenhouet]

I pushed the change and now let's see if the checks pass.

[Nic-Ma]

/black

[Nic-Ma]

Looks good to me now.
Thanks so much for the great feature and implementation!