# How to speed-up FreeSurfer's recon-all?

**URL:** <https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788>\
**Category:** Q&A\
**Tags:** image-processing, researcher\
**Created:** [May 15, 2023, 7:42pm UTC](https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788 "2023-05-15T19:42:13Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mitchellxh](https://ask.cyberinfrastructure.org/user_avatar/ask.cyberinfrastructure.org/mitchellxh/32/1033_2.png) [@mitchellxh](https://ask.cyberinfrastructure.org/u/mitchellxh)\
**Post date:** [May 15, 2023, 7:42pm UTC](https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788/1 "2023-05-15T19:42:13Z")

</div>

I work in a neuroimaging lab and we have many T1w MRI scans we’d like to process with the FreeSurfer [recon-all processing stream](https://surfer.nmr.mgh.harvard.edu/fswiki/recon-all). We also have access to an HPC with both GPU and CPU resources and would like to know how we can maximize processing speed performance. How can I reduce the individual runtime of a recon-all job?

---

<div class="post-metadata">

**Author:** ![mitchellxh](https://ask.cyberinfrastructure.org/user_avatar/ask.cyberinfrastructure.org/mitchellxh/32/1033_2.png) [@mitchellxh](https://ask.cyberinfrastructure.org/u/mitchellxh)\
**Post date:** [May 15, 2023, 7:53pm UTC](https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788/2 "2023-05-15T19:53:11Z")

</div>

FreeSurfer’s recon-all processing stream can be parallelized for the workflows on each hemisphere. This can significantly speed-up the individual subject runtime.

The flags needed to be added to the traditional recon-all command are:

> -parallel  
> -openmp $N\_cores

It’s important to set the $N\_cores to the number of cores allocated to your job. There is typically diminishing returns on requesting more than 8 cores per subject, dependent on the size and availability of your HPC. For speed-testing of recon-all parallelization you can reference this table:  
[https://rcs.bu.edu/examples/imaging/freesurfer/appendix/](https://rcs.bu.edu/examples/imaging/freesurfer/appendix/)

An example of the revised command for parallelization may look like:

> recon-all -all -s $subject -parallel -openmp 8

---

<div class="post-metadata">

**Author:** ![jfossot](https://ask.cyberinfrastructure.org/user_avatar/ask.cyberinfrastructure.org/jfossot/32/932_2.png) [@jfossot](https://ask.cyberinfrastructure.org/u/jfossot)\
**Post date:** [May 26, 2023, 4:15pm UTC](https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788/3 "2023-05-26T16:15:20Z")

</div>

This is really interesting take on workflow parallization, The approach taken by some other tools e.g [Nextflow](https://www.nextflow.io/docs/edge/config.html) is to run in parallel as long as you define processes. How is this example similar or different from running in parallel on the HPC?

@mitchellxh thanks for bringing this up

---

<div class="post-metadata">

**Author:** ![mitchellxh](https://ask.cyberinfrastructure.org/user_avatar/ask.cyberinfrastructure.org/mitchellxh/32/1033_2.png) [@mitchellxh](https://ask.cyberinfrastructure.org/u/mitchellxh)\
**Post date:** [May 27, 2023, 5:16pm UTC](https://ask.cyberinfrastructure.org/t/how-to-speed-up-freesurfers-recon-all/2788/4 "2023-05-27T17:16:30Z")

</div>

Great question!

Within FreeSurfer, ‘recon-all’ is the command and oft-mentioned workflow for structural reconstruction of brain MRI. This processing stream is limited to only processing a single brain at-one-time. Therefore using traditional parallelization on a cluster is a perfect fit, that is, submitting 10’s to 100’s of ‘recon-all’ jobs at once.

A typical ‘recon-all’ job with 1-core will take 5-6 hours. This probably doesn’t matter to the researcher who needs to process 10’s to 100’s of brains. However, in cases where they may need to process a single brain (or have seemingly unlimited resources…) they can speed-up the individual runtimes with these flags.

Briefly, because ‘recon-all’ splits the brain into 2 hemispheres for reconstruction, it is forced to sequentially process the left-hemisphere and then the right-hemisphere. The developers recognized this bottleneck and added support for Open-MP framework. By simply adding the above flags and allocating across 4 cores (2-cores per hemisphere), the ‘recon-all’ job will now take 3-4 hours.

Hope this adds more context!  
-Mitch