bioinfo@ird.fr

Trainings- HPC – Practice SLURM

HPC Practice SLURM

Description Hands On Lab Exercises for HPC
Related-course materials HPC
Authors Ndomassi TANDO (ndomassi.tando@ird.fr)
Creation Date 25/11/2019
Last Modified Date 03/06/2026

Summary

Preambule

Getting connected to a Linux servers from Windows with SSH (Secure Shell) protocol
Platform Software Description url
mobaXterm an enhanced terminal for Windows with an X11 server and a tabbed SSH client more
putty Putty allows to connect to a Linux server from a Windows workstation. Download
Transferring and copying files from your computer to a Linux servers with SFTP (SSH File Transfer Protocol) protocol
Platform Software Description url
filezilla FTP and SFTP client Download
Viewing and editing files on your computer before transferring on the linux server or directly on the distant server
Type Software url
Distant, consol mode nano Tutorial
Distant, consol mode vi Tutorial
Distant, graphic mode komodo edit Download
Linux & windows based editor Notepad++ Download

Install your ssh keys on your computer

Follow the instructions here according to your OS:

https://bioinfo.ird.fr/index.php/en/tutorials-howtos-add-ssh-keys/

Practice 1: Connect to a linux server with ssh

In mobaXterm:

  1. Click the session button, then click SSH.
    • In the remote host text box, type: bioinfo-master1.ird.fr
    • Check the specify username box and enter your user name
  2. In the console, enter the password when prompted.
    Once you are successfully logged in, you will be use this console for the rest of the lecture.
  3. Type the command sinfo and comment the result
  4. type the command sinfo -N nodes --long and noticed what have been added
  5. Type the command scontrol show nodes

Practice 2: Reserve one core of a node using srun and create your working folder

  1. Type the command squeue and noticed the result
  2. Type the command squeue -u your_login with your_login to change with your account and notice the difference
  3. More details with the command: squeue -O "username,name:40,partition,nodelist,NumCPUs,state,timeused,timelimit"
  4. Type the command hostname then srun hostname . What is the difference?
  5. Type the command srun -p short --pty bash -i then squeue again
  6. Create your own working folder in the /scratch of your node:
cd /scratch
 mkdir login
 with login : the name of your choice

Practice 3 : Transferring files to or from bioinfo-san.ird.fr `rsync

  • Using rsync, transfer a file from your computer to your /users/loginto bioinfo-san.ird.fr. Use lsto verify on bioinfo-san.ird.fr
  • Using rsync, transfer the file /share/formation/Slurm/TPassembly/TP.txt from bioinfo-san.ird.fr to your computer

Practice 4: Transfer your data from the san server to the node

  1. Using rsync, transfer the folder TP_reads_quality located in /share/formation/ into your working directory
  2. Check your result with ls

Practice 5: Use module environment to load your tools

  1. Load FastQC/0.12.1 and multiqc/1.9 modules in bioinfo-itrop environment
  2. Check if the tools are loaded

Practice 6 : Launch analyses

Use FastQC

Launch the command

cd /scratch/$USER/TP_reads_quality/
mkdir ./results/
fastqc --threads 1 --outdir ./results/ ./reads/*

Use multiqc

Launch the command to obtain a report of your analysis

multiqc --filename MultiQC-from-FastQC_AAAA-MM-DD --title i-Trop-Cluster_ --outdir ./ ./results/

Practice 7: Transfering data to the san server

  1. Using rsync, transfer your results from your /scratch/login to your /home/login
  2. Check if the transfer is OK with ls

Practice 8: Deleting your temporary folder

cd /scratch
rm -r login
exit

Practice 9: Launch a job with sbatch

Following the several steps performed during the practice, create a script to launch the analyses made in practice6:

1er step: create the Slurm section in your script

1) Set a name for your job

2) Precise your email

3) Choose the short partition

2nd step: type the commands you want the script to launch:

1) create a personal folder in /scratch with mkdir

2) Using rsync, transfer the folder TP_reads_quality located in /share/formation/ into your working directory

3)Load FastQC/0.12.1 and multiqc/1.9 modules in bioinfo-itrop environment

4) Into the the folder TP_reads_quality, launch the following command:

mkdir ./results/
fastqc --threads 1 --outdir ./results/ ./reads/*
multiqc --filename MultiQC-from-FastQC_AAAA-MM-DD --title i-Trop-Cluster_ --outdir ./ ./results/

5) Using rsync, transfer your results from your /scratch/login to your /home/login

6) Delete the personal folder in the /scratch

Launch the following commands to obtain info on the finished job:

seff <JOB_ID>
sacct --format=JobID,elapsed,ncpus,ntasks,state,node -j <JOB_ID>

Bonus:

We are going to launch a 4 steps analysis:

1) Perform a multiple alignment with the nucmer tool

2) Filter these alignments with the delta-filter tool

3) Generate a tab file easy to parse the with show-coords tools

4) Generate a png image with mummerplot

  • Retrieve the script /projects/medium/formation/Slurm/scripts/alignment_slurm.sh into your /home/login

  • modify the Slurm section and the variables

  • launch the script with sbatch:

sbatch alignment.sh

  • Do a ls -altr in your /home/login. What do you notice?

  • Launch the following commands to obtain info on the finished job:

seff <JOB_ID>
sacct --format=JobID,elapsed,ncpus,ntasks,state,node -j <JOB_ID>

  • Open filezilla and retrieve the png image to your computer

  • Launch the following commande to clear the /scratch of the node

sh /opt/scripts/scratch-scripts/scratch_use.sh

and choose the number of the node used

Links

License

The resource material is licensed under the Creative Commons Attribution 4.0 International License (here).

© 2026 itrop. Built using WordPress and the EmpowerWP theme.