# \#programming-for-hpc

**URL:** https://ask.cyberinfrastructure.org/tag/programming-for-hpc/20.md

[Latest](https://ask.cyberinfrastructure.org/latest.md) · [Categories](https://ask.cyberinfrastructure.org/categories.md) · [Tags](https://ask.cyberinfrastructure.org/tags.md)

---

## [How to manage Performance issues when using VSCode to connect to an HPC login node](https://ask.cyberinfrastructure.org/t/how-to-manage-performance-issues-when-using-vscode-to-connect-to-an-hpc-login-node/4408)

<div class="topic-metadata">

**Author:** [@jfossot](https://ask.cyberinfrastructure.org/u/jfossot)\
**Replies:** 0\
**Last updated:** [March 7, 2026, 8:42pm UTC](https://ask.cyberinfrastructure.org/t/how-to-manage-performance-issues-when-using-vscode-to-connect-to-an-hpc-login-node/4408 "2026-03-07T20:42:07Z")

</div>

The core problem: VSCode users connecting to an HPC login node cause performance issues, mainly through heavy file I/O (VSCode’s file watcher crawls the filesystem aggressively). Recommended solutions, in order of popul…

---

## [What solution does your institution provide for running SAS interactively on your cluster?](https://ask.cyberinfrastructure.org/t/what-solution-does-your-institution-provide-for-running-sas-interactively-on-your-cluster/4375)

<div class="topic-metadata">

**Author:** [@ktrn](https://ask.cyberinfrastructure.org/u/ktrn)\
**Replies:** 0\
**Last updated:** [December 12, 2025, 12:54am UTC](https://ask.cyberinfrastructure.org/t/what-solution-does-your-institution-provide-for-running-sas-interactively-on-your-cluster/4375 "2025-12-12T00:54:05Z")

</div>

Our users find the standard interactive environment (the SAS Display Manager) very difficult to use. It opens as several small, isolated windows, which is a significant downgrade from the modern SAS Studio experience the…

---

## [How to run a research deepfake model on Jetstream2](https://ask.cyberinfrastructure.org/t/how-to-run-a-research-deepfake-model-on-jetstream2/4035)

<div class="topic-metadata">

**Author:** [@juanjo.garciamesa](https://ask.cyberinfrastructure.org/u/juanjo.garciamesa)\
**Replies:** 2\
**Last updated:** [March 10, 2025, 9:03pm UTC](https://ask.cyberinfrastructure.org/t/how-to-run-a-research-deepfake-model-on-jetstream2/4035 "2025-03-10T21:03:47Z")

</div>

I recently was able to deploy an instance in Jetstream2 and run a deepfake project that focuses on individual recognition in paper wasps. I composed a document with the steps I followed and I thought it would be of inter…

---

## [Push singularity image from cluster](https://ask.cyberinfrastructure.org/t/push-singularity-image-from-cluster/4149)

<div class="topic-metadata">

**Author:** [@sasmita](https://ask.cyberinfrastructure.org/u/sasmita)\
**Replies:** 5\
**Last updated:** [February 16, 2025, 5:33pm UTC](https://ask.cyberinfrastructure.org/t/push-singularity-image-from-cluster/4149 "2025-02-16T17:33:08Z")

</div>

How to push a singularity image to docker hub from cluster.

---

## [How to access files on cloud (gs, aws, etc) from code, such like Python, R?](https://ask.cyberinfrastructure.org/t/how-to-access-files-on-cloud-gs-aws-etc-from-code-such-like-python-r/4011)

<div class="topic-metadata">

**Author:** [@yshen16](https://ask.cyberinfrastructure.org/u/yshen16)\
**Replies:** 0\
**Last updated:** [August 21, 2024, 1:19pm UTC](https://ask.cyberinfrastructure.org/t/how-to-access-files-on-cloud-gs-aws-etc-from-code-such-like-python-r/4011 "2024-08-21T13:19:59Z")

</div>

Hi Folks, I’d like to ask if anyone would like to share some experience on how to get access to the file system on cloud, such as google cloud storage or aws S3 bucket, or azure, etc. directly from code, such as python,…

---

## [Tell me, why I should use Anaconda instead of Python Virtual Environment?](https://ask.cyberinfrastructure.org/t/tell-me-why-i-should-use-anaconda-instead-of-python-virtual-environment/4054)

<div class="topic-metadata">

**Author:** [@jfossot](https://ask.cyberinfrastructure.org/u/jfossot)\
**Replies:** 1\
**Last updated:** [September 17, 2024, 1:44pm UTC](https://ask.cyberinfrastructure.org/t/tell-me-why-i-should-use-anaconda-instead-of-python-virtual-environment/4054 "2024-09-17T13:44:49Z")

</div>

By my experience, setting up a conda environment is as complicated as it can get and I have to remember certain things to be able to activate and deactivate the environment. Also, the environment has to be turned on for …

---

## [Anaconda license situation at your site?](https://ask.cyberinfrastructure.org/t/anaconda-license-situation-at-your-site/4005)

<div class="topic-metadata">

**Author:** [@RonRahaman](https://ask.cyberinfrastructure.org/u/RonRahaman)\
**Replies:** 4\
**Last updated:** [September 4, 2024, 2:27pm UTC](https://ask.cyberinfrastructure.org/t/anaconda-license-situation-at-your-site/4005 "2024-09-04T14:27:08Z")

</div>

Hi all, Our site is trying to deal with Anaconda’s ongoing changes to its terms of service, which might make it prohibitively expensive to provide Anaconda as a software module on our clusters. Have any of your sites d…

---

## [A Research Computing Applications & Data Specialist (Neuroscience track)](https://ask.cyberinfrastructure.org/t/a-research-computing-applications-data-specialist-neuroscience-track/3971)

<div class="topic-metadata">

**Author:** [@ktrn](https://ask.cyberinfrastructure.org/u/ktrn)\
**Replies:** 0\
**Last updated:** [July 29, 2024, 1:56pm UTC](https://ask.cyberinfrastructure.org/t/a-research-computing-applications-data-specialist-neuroscience-track/3971 "2024-07-29T13:56:16Z")

</div>

The Research Computing Applications Team at Boston University is hiring a Research Computing Applications & Data Specialist (Neuroscience track). This position focuses on supporting the faculty and graduate students wor…

---

## [Resources for GPU and Google JAX programming](https://ask.cyberinfrastructure.org/t/resources-for-gpu-and-google-jax-programming/3133)

<div class="topic-metadata">

**Author:** [@sam\_dt](https://ask.cyberinfrastructure.org/u/sam_dt)\
**Replies:** 0\
**Last updated:** [May 17, 2024, 2:27pm UTC](https://ask.cyberinfrastructure.org/t/resources-for-gpu-and-google-jax-programming/3133 "2024-05-17T14:27:23Z")

</div>

When I first started my project in Google JAX, I knew that that Google JAX was a pretty new coding library made for GPUs and that examples and teaching materials would be scarce for that. And in addition, I was pretty ne…

---

## [How to port optimized applications from one ACCESS system to another?](https://ask.cyberinfrastructure.org/t/how-to-port-optimized-applications-from-one-access-system-to-another/2829)

<div class="topic-metadata">

**Author:** [@pkousha](https://ask.cyberinfrastructure.org/u/pkousha)\
**Replies:** 2\
**Last updated:** [April 18, 2024, 3:45pm UTC](https://ask.cyberinfrastructure.org/t/how-to-port-optimized-applications-from-one-access-system-to-another/2829 "2024-04-18T15:45:55Z")

</div>

Hi, I wonder if there are best practices for porting optimized applications from one system to another in ACCESS? how can I ensure efficient performance from one system to another?

---

## [Standard way(s) of launching MPI executable program?](https://ask.cyberinfrastructure.org/t/standard-way-s-of-launching-mpi-executable-program/3044)

<div class="topic-metadata">

**Author:** [@wirawan0](https://ask.cyberinfrastructure.org/u/wirawan0)\
**Replies:** 2\
**Last updated:** [March 11, 2024, 1:50pm UTC](https://ask.cyberinfrastructure.org/t/standard-way-s-of-launching-mpi-executable-program/3044 "2024-03-11T13:50:29Z")

</div>

I want to poll the community to gather knowledge on one very basic thing: What is the standard way of launching an MPI executable program? Is this pretty universally acceptable across different clusters, different job sc…

---

## [How to set up perl debugging environment in popular IDEs in HPC OnDemand](https://ask.cyberinfrastructure.org/t/how-to-set-up-perl-debugging-environment-in-popular-ides-in-hpc-ondemand/3075)

<div class="topic-metadata">

**Author:** [@yshen16](https://ask.cyberinfrastructure.org/u/yshen16)\
**Replies:** 0\
**Last updated:** [February 22, 2024, 9:34pm UTC](https://ask.cyberinfrastructure.org/t/how-to-set-up-perl-debugging-environment-in-popular-ides-in-hpc-ondemand/3075 "2024-02-22T21:34:25Z")

</div>

Hi all, I’d like to post an interesting question I encountered from a ticket I reeived recently. A user asked if we can help set up a perl debugging environment in some programming IDE instead of using the command line …

---

## [Parallel Debuggers: which do you choose?](https://ask.cyberinfrastructure.org/t/parallel-debuggers-which-do-you-choose/2949)

<div class="topic-metadata">

**Author:** [@nucci](https://ask.cyberinfrastructure.org/u/nucci)\
**Replies:** 1\
**Last updated:** [October 11, 2023, 7:13pm UTC](https://ask.cyberinfrastructure.org/t/parallel-debuggers-which-do-you-choose/2949 "2023-10-11T19:13:51Z")

</div>

Greetings. For many years we have offered our users a choice of parallel debuggers on our clusters. We’re looking to reduce the number of offerings to reduce system complexity, training requirements, financial costs, et…

---

## [Examine specific lines of code for memory profiling](https://ask.cyberinfrastructure.org/t/examine-specific-lines-of-code-for-memory-profiling/2963)

<div class="topic-metadata">

**Author:** [@shrek](https://ask.cyberinfrastructure.org/u/shrek)\
**Replies:** 1\
**Last updated:** [September 30, 2023, 7:50pm UTC](https://ask.cyberinfrastructure.org/t/examine-specific-lines-of-code-for-memory-profiling/2963 "2023-09-30T19:50:44Z")

</div>

How can I obtain detailed insights for analyzing hardware memory usage in fine detail within a simple Python function as shown below, especially when examining specific lines of code contributing to memory growth, simila…

---

## [Page-level memory management for NVIDIA GPUs](https://ask.cyberinfrastructure.org/t/page-level-memory-management-for-nvidia-gpus/2947)

<div class="topic-metadata">

**Author:** [@ShrutiDongare](https://ask.cyberinfrastructure.org/u/ShrutiDongare)\
**Replies:** 0\
**Last updated:** [September 21, 2023, 9:39pm UTC](https://ask.cyberinfrastructure.org/t/page-level-memory-management-for-nvidia-gpus/2947 "2023-09-21T21:39:43Z")

</div>

My objective is to monitor memory access patterns for generative AI workloads and subsequently develop optimization strategies for managing memory costs across various memory tiers. One example of memory tiers would be G…

---

## [Profile Python processes across multiple cores](https://ask.cyberinfrastructure.org/t/profile-python-processes-across-multiple-cores/2867)

<div class="topic-metadata">

**Author:** [@cec5550](https://ask.cyberinfrastructure.org/u/cec5550)\
**Replies:** 2\
**Last updated:** [August 17, 2023, 3:30pm UTC](https://ask.cyberinfrastructure.org/t/profile-python-processes-across-multiple-cores/2867 "2023-08-17T15:30:38Z")

</div>

I am running many different processes with Python’s multiprocessing module. What would be the best way to profile each process so that I could determine the limiting process for my Python program? I would like to see the…

---

## [Visualization with Python from HPC cluster](https://ask.cyberinfrastructure.org/t/visualization-with-python-from-hpc-cluster/2835)

<div class="topic-metadata">

**Author:** [@cec5550](https://ask.cyberinfrastructure.org/u/cec5550)\
**Replies:** 3\
**Last updated:** [August 17, 2023, 2:51pm UTC](https://ask.cyberinfrastructure.org/t/visualization-with-python-from-hpc-cluster/2835 "2023-08-17T14:51:47Z")

</div>

Is there any way to visualize results of Python script using a plotting library such as matplotlib or seaborn from the cluster itself? If not, what would be a good work around to this workflow?

---

## [Create a SQLite database on TACC](https://ask.cyberinfrastructure.org/t/create-a-sqlite-database-on-tacc/2905)

<div class="topic-metadata">

**Author:** [@sohil.shrestha](https://ask.cyberinfrastructure.org/u/sohil.shrestha)\
**Replies:** 1\
**Last updated:** [August 16, 2023, 4:28pm UTC](https://ask.cyberinfrastructure.org/t/create-a-sqlite-database-on-tacc/2905 "2023-08-16T16:28:56Z")

</div>

I am trying to train a deep learning model in TACC. My project involves creating a sqlite3 database . I get the following error when i try to create a database: (sqlite3.OperationalError) disk I/O error \[SQL: PRAGMA ta…

---

## [Remote VSCode: Recommended Method for Connecting?](https://ask.cyberinfrastructure.org/t/remote-vscode-recommended-method-for-connecting/2864)

<div class="topic-metadata">

**Author:** [@mitchellxh](https://ask.cyberinfrastructure.org/u/mitchellxh)\
**Replies:** 3\
**Last updated:** [June 28, 2023, 3:26pm UTC](https://ask.cyberinfrastructure.org/t/remote-vscode-recommended-method-for-connecting/2864 "2023-06-28T15:26:02Z")

</div>

Recently, my group has experienced an influx in requests to remotely connect to the cluster with their local VSCode environment. When the login nodes do not provide enough resources, what methods are people advising thei…

---

## [Best practice for compiling MATLAB code](https://ask.cyberinfrastructure.org/t/best-practice-for-compiling-matlab-code/2852)

<div class="topic-metadata">

**Author:** [@mitchellxh](https://ask.cyberinfrastructure.org/u/mitchellxh)\
**Replies:** 2\
**Last updated:** [June 17, 2023, 1:01pm UTC](https://ask.cyberinfrastructure.org/t/best-practice-for-compiling-matlab-code/2852 "2023-06-17T13:01:49Z")

</div>

It’s been some time since I’ve needed to use MATLAB non-interactively and am wondering what’s the best practice for compiling my code and submitting as batch jobs. Additionally, my center’s HPC has a limited number of sh…

---

## [Interested in using TensorFlow with MPI on ACCESS HPC systems](https://ask.cyberinfrastructure.org/t/interested-in-using-tensorflow-with-mpi-on-access-hpc-systems/2810)

<div class="topic-metadata">

**Author:** [@Jobair.16](https://ask.cyberinfrastructure.org/u/Jobair.16)\
**Replies:** 1\
**Last updated:** [June 3, 2023, 3:38pm UTC](https://ask.cyberinfrastructure.org/t/interested-in-using-tensorflow-with-mpi-on-access-hpc-systems/2810 "2023-06-03T15:38:04Z")

</div>

Could you provide any tips or resources to get started?

---

## [How to use multiple GPU node to train/fine-tune large language models?](https://ask.cyberinfrastructure.org/t/how-to-use-multiple-gpu-node-to-train-fine-tune-large-language-models/2816)

<div class="topic-metadata">

**Author:** [@sohil.shrestha](https://ask.cyberinfrastructure.org/u/sohil.shrestha)\
**Replies:** 3\
**Last updated:** [May 31, 2023, 7:00pm UTC](https://ask.cyberinfrastructure.org/t/how-to-use-multiple-gpu-node-to-train-fine-tune-large-language-models/2816 "2023-05-31T19:00:57Z")

</div>

I am using UT’s Texas Advanced Computing Center (TACC)'s resources to fine tune large language models. However, I am able to fine-tune it with 1-batch size making the fine-tuning extremely slow and unstable. My understa…

---

## [Code profiling w/ nvhpc](https://ask.cyberinfrastructure.org/t/code-profiling-w-nvhpc/2712)

<div class="topic-metadata">

**Author:** [@nucci](https://ask.cyberinfrastructure.org/u/nucci)\
**Replies:** 1\
**Last updated:** [March 20, 2023, 11:00pm UTC](https://ask.cyberinfrastructure.org/t/code-profiling-w-nvhpc/2712 "2023-03-20T23:00:37Z")

</div>

I am looking to port a Fortran (yes, Fortran) application to use GPU/CUDA, using the nvhpc compilers. Prior to any conversion work, I am looking to identify the code hotspots via a profiler. In the past I would use the …

---

## [Systems Administrators in New York City: Servers, Clusters and Supercomputers for Computational Biochemistry](https://ask.cyberinfrastructure.org/t/systems-administrators-in-new-york-city-servers-clusters-and-supercomputers-for-computational-biochemistry/2350)

<div class="topic-metadata">

**Author:** [@DESRESCareers](https://ask.cyberinfrastructure.org/u/DESRESCareers)\
**Replies:** 0\
**Last updated:** [May 18, 2022, 11:42am UTC](https://ask.cyberinfrastructure.org/t/systems-administrators-in-new-york-city-servers-clusters-and-supercomputers-for-computational-biochemistry/2350 "2022-05-18T11:42:31Z")

</div>

Exceptional sysadmins sought to manage systems infrastructure, networks, and information security for a New York–based interdisciplinary research group. Successful hires will be responsible for our servers and cluster e…

---

## [Research Software & Programming Analyst @ Princeton University](https://ask.cyberinfrastructure.org/t/research-software-programming-analyst-princeton-university/2576)

<div class="topic-metadata">

**Author:** [@JessicaJenks](https://ask.cyberinfrastructure.org/u/JessicaJenks)\
**Replies:** 0\
**Last updated:** [October 21, 2022, 4:22pm UTC](https://ask.cyberinfrastructure.org/t/research-software-programming-analyst-princeton-university/2576 "2022-10-21T16:22:14Z")

</div>

Princeton Research Computing, a consortium of campus groups led by the Princeton Institute for Computational Science & Engineering (PICSciE) and the Office of Information Technology’s Research Computing group, is recruit…

---

## [Research Engineer Position at Oregon Health Sciences University](https://ask.cyberinfrastructure.org/t/research-engineer-position-at-oregon-health-sciences-university/2375)

<div class="topic-metadata">

**Author:** [@Adib\_Zaman](https://ask.cyberinfrastructure.org/u/Adib_Zaman)\
**Replies:** 0\
**Last updated:** [June 3, 2022, 6:08pm UTC](https://ask.cyberinfrastructure.org/t/research-engineer-position-at-oregon-health-sciences-university/2375 "2022-06-03T18:08:13Z")

</div>

Good evening, I am reaching out to this group to share an exciting opportunity in research informatics at the Oregon Health Sciences University (OHSU). Oregon Clinical and Translational Research Institute (OCTRI) is loo…

---

## [What is the difference between the subprocess module and the os module in python? (For running shell commands)](https://ask.cyberinfrastructure.org/t/what-is-the-difference-between-the-subprocess-module-and-the-os-module-in-python-for-running-shell-commands/2261)

<div class="topic-metadata">

**Author:** [@tolu\_058](https://ask.cyberinfrastructure.org/u/tolu_058)\
**Replies:** 1\
**Last updated:** [March 22, 2022, 8:20pm UTC](https://ask.cyberinfrastructure.org/t/what-is-the-difference-between-the-subprocess-module-and-the-os-module-in-python-for-running-shell-commands/2261 "2022-03-22T20:20:22Z")

</div>

I want to incorporate shell commands in my python HPC script and I was wondering which python module is better to use - os or subprocess? os seems fairly easier to use but subprocess is the recommended module.

---

## [How do I use Linux commands in a Python script?](https://ask.cyberinfrastructure.org/t/how-do-i-use-linux-commands-in-a-python-script/2260)

<div class="topic-metadata">

**Author:** [@tolu\_058](https://ask.cyberinfrastructure.org/u/tolu_058)\
**Replies:** 1\
**Last updated:** [March 22, 2022, 8:08pm UTC](https://ask.cyberinfrastructure.org/t/how-do-i-use-linux-commands-in-a-python-script/2260 "2022-03-22T20:08:55Z")

</div>

I am running a Python script in a cluster, and I want to execute Linux commands through my python script when I run the script, opposed to manually typing each line of Linux code in the terminal, How would I do this?

---

## [What is the preferred way to install R packages/libraries on Monsoon?](https://ask.cyberinfrastructure.org/t/what-is-the-preferred-way-to-install-r-packages-libraries-on-monsoon/1694)

<div class="topic-metadata">

**Author:** [@jasonbuechler](https://ask.cyberinfrastructure.org/u/jasonbuechler)\
**Replies:** 0\
**Last updated:** [December 30, 2020, 6:19pm UTC](https://ask.cyberinfrastructure.org/t/what-is-the-preferred-way-to-install-r-packages-libraries-on-monsoon/1694 "2020-12-30T18:19:36Z")

</div>

Prerequisites Understanding external dependencies Most R package/library installation issues are due to the library requiring software that is unavailable via CRAN (R’s internal package repository). Every R package shoul…

---

## [CyberTeams CAREERS project in Rhode Island](https://ask.cyberinfrastructure.org/t/cyberteams-careers-project-in-rhode-island/2002)

<div class="topic-metadata">

**Author:** [@gkhanna](https://ask.cyberinfrastructure.org/u/gkhanna)\
**Replies:** 0\
**Last updated:** [August 7, 2021, 2:24pm UTC](https://ask.cyberinfrastructure.org/t/cyberteams-careers-project-in-rhode-island/2002 "2021-08-07T14:24:03Z")

</div>

The CAREERS (Cyberteam to Advance Research and Education in Eastern Regional Schools) CyberTeam Program is a 3-year initiative funded by the NSF to build a regional pool of Research Computing Facilitators (RCF) to suppor…

[Next page](https://ask.cyberinfrastructure.org/tag/programming-for-hpc/20.md?match_all_tags=true&page=1&tags%5B%5D=programming-for-hpc)
