Brain extraction workflow does not reuse cache

[effigies]

While testing FreeSurfer, two immediately succeeding runs with the same working directory rerun large chunks of the brain extraction workflow, due to an "outdated cache" detected for init_aff. This task had previously run successfully.

191011-18:12:31,369 nipype.workflow INFO:
	 [Node] Outdated cache found for "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff".
191011-18:12:31,901 nipype.utils INFO:
	 Running nipype version 1.2.3 (latest: 1.2.3)
191011-18:12:33,297 nipype.workflow INFO:
	 [MultiProc] Running 1 tasks, and 1 jobs ready. Free memory (GB): 13.77/13.97, Free processors: 1/8.
                     Currently running:
                       * smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff
191011-18:12:34,337 nipype.workflow INFO:
	 [Node] Setting-up "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff" in "/scratch/smriprep_wf/single_subject_01_wf/anat_preproc_wf/brain_extraction_wf/init_aff".
191011-18:12:34,338 nipype.workflow INFO:
	 [Node] Outdated cache found for "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff".
191011-18:12:34,341 nipype.workflow INFO:
	 [Node] Running "init_aff" ("niworkflows.interfaces.ants.AI"), a CommandLine Interface with command:
antsAI -c [10,0.000001,10] -d 3 -x /home/smriprep/.cache/templateflow/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_desc-BrainCerebellumExtraction_mask.nii.gz -m Mattes[/scratch/smriprep_wf/single_subject_01_wf/anat_preproc_wf/brain_extraction_wf/res_tmpl/tpl-OASIS30ANTs_res-01_T1w_resampled.nii.gz,/scratch/smriprep_wf/single_subject_01_wf/anat_preproc_wf/brain_extraction_wf/res_target/sub-01_T1w_maths_corrected_resampled.nii.gz,32,Regular,0.250000] -o initialization.mat -p 0 -s [15.000000,0.100000] -t Affine[0.100000] -v 1
191011-18:12:37,280 nipype.workflow INFO:
	 [Node] Finished "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff".
191011-18:12:37,301 nipype.workflow INFO:
	 [Job 21] Completed (smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.init_aff).
191011-18:12:37,303 nipype.workflow INFO:
	 [MultiProc] Running 0 tasks, and 2 jobs ready. Free memory (GB): 13.97/13.97, Free processors: 8/8.
191011-18:12:37,365 nipype.workflow INFO:
	 [Node] Outdated cache found for "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.norm".

[oesteban]

I believe this is a result of not using the content hash, since nipype was patched too recently (nipy/nipype#3072)

[effigies]

But I'm running with the same version of nipype...

[oesteban]

What do you find below the last line ( [Node] Outdated cache found for "smriprep_wf.single_subject_01_wf.anat_preproc_wf.brain_extraction_wf.norm")? The diff between inputs should be shown.

[effigies]

I don't think it was. But I showed the first outdated cache on the first line. Anyway, not at that computer now. Can try rerunning on Monday.

[oesteban]

Oh I see, yes - you'll need to increase the verbosity to see the inputs that changed.

[effigies]

Ah, the issue is that the brain extraction mask isn't pre-downloaded into the Docker image, so the timestamp hash is new every time.

[oesteban]

Sorry, I wasn't clear before - we set the hash to content and write it to the config file, but then the config file is ignored because of that bug in nipype, I guess

[effigies]

Wait, you want to do content hashes in general? Isn't that going to take a very long time?

[oesteban]

We've been doing it for a while in fMRIPrep and it doesn't seem to be a lot slower. Regardless, I agree that mask should be cached.

[effigies]

I'm adding it to #117.