Ask.Cyberinfrastructure

Do you have bioinformatic specific script examples to share?


#1

A researcher in bioinformatics on our campus is seeking advice for use of Comet resources and specifically is seeking an example batch job script for any of the bioinformatics tools. She had seen the examples scripts at this URL (https://portal.xsede.org/sdsc-comet#running:jobscripts) but was hoping to be pointed to something more bioinformatic specific.

Also, this individual was able to upload her data to Oasis, but was not able to determine how to ‘connect’ Comet to Oasis (is it as simple as a CD command?


#2

I’m not staff at SDSC nor do I know a lot about Comet, but I know that they have Singularity installed, so I would open up her world to running scripts via containers. And Comet has quite a nice set. You could start like this:

  1. Tell her about reproducible science, how containers can package it up, be run as executables, and help with that.
  2. Find some software she is interested in, find a matching container.
  3. Take her straight to the section here (I can’t link to it, you need to scroll) that is titled “Run the Container on Comet.” Skip over the entire first sections - the content is dated and the sheer length would be overwhelming for a new user to think “I have to do all this?!”
  4. Then jump to “Using Tensorflow with Singularity” that shows the same thing, but with the corresponding sbatch!

At this point she should be able to:

  1. Find the software she needs, via a container
  2. Submit a job to run the container directly or an sbatch

The next steps are where she might be interested, and also say “hey, I need/want this other container.” or “hey, how do I change this?” That’s when you rewind and tell her “Guess what, you can build these on your own, or modify one that is built!” And at that point I’d send her to the Singularity docs to go through one of the Getting Started guides, or some center “Getting Started” guide that you like. The important thing is to not overwhelm her with too much to learn, to show her how easy it can be to run an entire (complicated to generate) software stack, and then when she is comfortable reveal that she can roll her own. And most importantly, she is starting from step 1 with reproducible practices.


Ah! And I just remembered from long ago, a user posted a great tutorial (with the batch submission script!) for Comet! zazing --> https://github.com/zonca/singularity-comet And an asciinema.

No comment (or should I say “Comet” on Oasis) because I don’t know the answer, harhar. :slight_smile:If I were you, I’d check the path at /oasis to see if there are files there? It might be organized by user, project, group, you would find out! If that doesn’t work, just explore the root (/) and you might find something.


How do I install Python 2.7 on my Comet allocation and where can I store data long term?
#3

This seems extremely helpful and containers would ideal to reduce redundant work and offer reproducibility. I appreciate your insight, and your humor! :blush:
~ Cyd