Nextflow, disabling trap command

milechin · May 10, 2024, 2:13pm

For each process, Nextflow creates a working directory and creates a collection of hidden files to run the job (e.g. .command.begin, .command.log, etc). The .command.run is a bash script that contains nthe commands to execute the process as a job. In this file, we noticed under nxf_main() function there are three trap commands:

trap on_exit EXIT
trap on_term TERM INT USR2
trap ‘’ USR1

We have found that the last trap command, “trap ‘’ USR1”, is causing an issue for our pipeline. We found a hack to overcome this issue, but I am wondering if anyone knows of a way to control which trap commands are included in the .command.run script? Either via optional flags or maybe a configuration file? Or are these just hard coded into a template file?

milechin · May 20, 2024, 2:46pm

I took a deeper dive into the Nextflow Github repository. The .command.run file is generated from a template file command-run.txt. Source:

github.com

nextflow-io/nextflow/blob/1923fef6231e9846e604ee3eb50ecb61c5fbd0bf/docs/developer/nextflow.executor.md?plain=1#L21


      
          ```{note}
          Some classes may be excluded from the above diagram for brevity.
          ```
          
          ## Notes
          
          The `Executor` class is the base class for all Nextflow executors. The main purpose of an `Executor` is to submit tasks to an underlying compute environment, such as an HPC scheduler or cloud batch executor. It uses a `TaskMonitor` to manage the lifecycle of all tasks and a `TaskHandler` to manage each individual task. Most executors use the same polling monitor, but each executor implements its own task handler to customize it for a particular compute environment. See [nextflow.processor](nextflow.processor.md) for more details about these classes.
          
          The built-in executors include the local executor (`LocalExecutor`) and the various grid executors (SLURM, PBS, LSF, etc), all of which extend `AbstractGridExecutor`. The `LocalExecutor` implements both "script" tasks (processes with a `script` or `shell` block) and "native" tasks (processes with an `exec` block).
          
          The `BashWrapperBuilder` is used by executors to generate the wrapper script (`.command.run`) for a task, from a template script called `command-run.txt`, as well as the task configuration and the execution environment.

The file itself is found in the link below. The trap commands are found under function nxf_main (Line 131), which are part of the template.

github.com

nextflow-io/nextflow/blob/master/modules/nextflow/src/main/resources/nextflow/executor/command-run.txt?plain=1#L131


      
              {{stage_inputs}}
          }
          
          nxf_unstage() {
              true
              {{unstage_controls}}
              [[ ${nxf_main_ret:=0} != 0 ]] && return
              {{unstage_outputs}}
          }
          
          nxf_main() {
              trap on_exit EXIT
              trap on_term TERM INT USR2
              trap '' USR1
          
              [[ "${NXF_CHDIR:-}" ]] && cd "$NXF_CHDIR"
              {{container_boxid}}
              {{scratch_cmd}}
              [[ $NXF_DEBUG > 0 ]] && nxf_env
              {{touch_file}}
              set +u