Improved HPC Support through cluster_config() #297
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #296.
Implements
cluster_config()
for full granular control over HPC job schedulers such as Slurmsbatch
, SGE / Torque / PBSqsub
and LSFbsub
.These systems expect a shell script to be executed with the full configuration options contained within, including the loading of environment modules etc., which would otherwise not be possible to pass in a single command line.
Rather than dynamically create an actual script on the filesystem, this implementation employs a Heredoc for more optimal programmatic control.
EDIT: rebasing after merging #298, now inherently supports array jobs just by adding a line
#SBATCH --array=
or equivalent in theoptions
.FYI @michaelmayer2