Skip to content

Fix labels from Slicer plugin offset by extra background class #716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Fix labels from Slicer plugin offset by extra background class #716

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
50f962f
Fix labels from Slicer plugin offset by extra background class
adamaji Mar 31, 2022
4cceaea
Merge branch 'main' into segment-labelmap
diazandr3s Apr 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions plugins/slicer/MONAILabel/MONAILabel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1279,8 +1279,9 @@ def onSaveLabel(self):
qt.QApplication.setOverrideCursor(qt.Qt.WaitCursor)
segmentationNode = self._segmentNode
labelmapVolumeNode = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLLabelMapVolumeNode")
slicer.modules.segmentations.logic().ExportVisibleSegmentsToLabelmapNode(
segmentationNode, labelmapVolumeNode, self._volumeNode
save_segment_ids = [segment_id for segment_id in segmentationNode.GetSegmentation().GetSegmentIDs() if segment_id != "background"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we can discard the background label (which is normally used by deepedit multilabel model) along with scribbles background/foreground when you save the label..
but the fix possibly by adding "background" to the list mentioned at: https://github.com/Project-MONAI/MONAILabel/blob/main/plugins/slicer/MONAILabel/MONAILabel.py#L1293

slicer.modules.segmentations.logic().ExportSegmentsToLabelmapNode(
segmentationNode, save_segment_ids, labelmapVolumeNode, self._volumeNode
)

segmentation = segmentationNode.GetSegmentation()
Expand Down Expand Up @@ -1722,8 +1723,11 @@ def onUpdateScribbles(self):
# get scribbles + label
segmentationNode = self._segmentNode
labelmapVolumeNode = slicer.mrmlScene.AddNewNodeByClass("vtkMRMLLabelMapVolumeNode")
slicer.modules.segmentations.logic().ExportVisibleSegmentsToLabelmapNode(
segmentationNode, labelmapVolumeNode, self._volumeNode
_, segment = self.currentSegment()
selected_label_name = segment.GetName()
save_segment_ids = [selected_label_name, "background_scribbles", "foreground_scribbles"]
slicer.modules.segmentations.logic().ExportSegmentsToLabelmapNode(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a known issue for scribbles.. it currently can't support/persist scribbles per label.. specially in case for multi-label.. what you need is save/persist the bg/fg scribbles (similar to fg/bg fudical/click points) per label.. so the behavior/user experience in case of multilabel might not be good.

However, i will let @masadcv to decide what he needs from the label node.. i guess he might want to get all labels and pick fg/bg label masks as part of pre-transforms + current label name (i guess he needs current label name in case of he wants all labels; which might be the reason, it's not working) and send back the results with corrections..

again only corrected label should get updated.. i guess it needs to be fixed/verified as well..

In future these methods might want to use existing label masks as input along with fg/bg label masks..
Or until that future comes, it's ok for me to include only fg/bg scribbles sent to server to run the scribbles action..

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamaji @SachidanandAlle understood.
I would like to keep labels and do selection on the server side. This is to make it future-proof where we can use more advanced multi-label post-processing methods such as CRF that take into account other labels as well.

The scribbles labels are currently hard-coded in server app, these need to be passed from the client - in case the order changes or similar shift in label are present (due to any changes to client). Hence, a need for dynamic updates for this.

As Sachi said, background may need to be removed before saving final segmentation output, but this can be done at https://github.com/Project-MONAI/MONAILabel/blob/main/plugins/slicer/MONAILabel/MONAILabel.py#L1293

I will continue this in PR #717 where I will merge the changes from here for updating the current label and provide a generalisable solution to the problem that we can extend in future to multi-label problems. As this needs proper testing with multilabel problem, I will take sometime and get back with an update when ready.

segmentationNode, save_segment_ids, labelmapVolumeNode, self._volumeNode
)
scribbles_in = tempfile.NamedTemporaryFile(suffix=self.file_ext, dir=self.tmpdir).name
self.reportProgress(5)
Expand Down Expand Up @@ -1753,9 +1757,7 @@ def onUpdateScribbles(self):

# display result from server
self.reportProgress(90)
_, segment = self.currentSegment()
label = segment.GetName()
self.updateSegmentationMask(result_file, [label])
self.updateSegmentationMask(result_file, [selected_label_name])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes,, this will make sure we update the selected mask only 👍

except:
slicer.util.errorDisplay(
"Failed to post process label on MONAI Label Server using {}".format(scribblesMethod),
Expand Down