How to use the mutualised scratch partition: /scratch-ib
| Description | Use /scratch-ib |
|---|---|
| Author | ndomassi TANDO (ndomassi.tando@ird.fr) |
| Creation Date | 21/01/2025 |
| Modification Date | 07/03/2025 |
/scratch-ib partition role
/scratch-ib is a mutualized scratch partition hosted on bioinfo-san.ird.fr that store temporary data to analyse
It is shared between all the nodes with the infiniband feature (high rate network)
Users can store directly their data into /scratch-ib on bioinfo-san.ird.fr then use one or several infiniband nodes to process then directly in /scratch-ib
Calculation can be launched on several nodes at the same time.
/scratch-ib partition features
- /scratch-ib is accessible through the Internet via the bioinfo-san.ird.fr server
- It is a 128TB partition
- Data on this partition is only accessible by nodes with the infiniband feature.
Use /scratch-ib
1) Upload your data in /scratch-ib using bioinfo-san.ird.fr
To see the transfer process: https://bioinfo.ird.fr/index.php/en/tutorials-howtos-i-trop-cluster/#howto-1
2) Reserve infiniband ressources:
Only certain nodes are equiped with infiniband for now , check the list below:
| node | partitions | CPU | RAM total (Gb) | RAM by CPU (Gb/cpu) | scratch size (To) |
|---|---|---|---|---|---|
| node0 | normal | 24 | 144 | 6 | 13 |
| node1 | normal | 24 | 144 | 6 | 2 |
| node4 | normal | 72 | 257 | 4 | 13 |
| node5 | normal | 88 | 516 | 6 | 13 |
| node7 | normal | 24 | 145 | 6 | 3 |
| node8 | long | 24 | 48 | 2 | 13 |
| node9 | normal | 64 | 144 | 2 | 13 |
| node11 | runner | 24 | 48 | 2 | 3 |
| node17 | normal | 48 | 144 | 3 | 13 |
| node20 | normal | 20 | 64 | 3 | 13 |
| node21 | normal | 24 | 144 | 6 | 13 |
| node24 | normal | 20 | 64 | 3 | 13 |
| node25 | supermem | 40 | 1 032 | 26 | 14 |
| node26 | gpu | 24 | 192 | 8 | 14 |
| node27 | normal | 24 | 144 | 6 | 14 |
To select an infiniband node, you need to add the following slurm option
--constraint=infiniband
For example, to connect i a normal partition node with infiniband in interactive mod:
srun -p short --constraint=infiniband --pty bash -i
In a script launched with sbatch, add the following options at the beginning:
#SBATCH -p short
#SBATCH --constraint=infiniband
3) Launch your analyses directly in the /scratch-ib partition
cd /scratch-ib
Do not forget to modify your script with /scratch-ib instead of /scratch
If your data is in the /projects folder, you can integrate the transfer in your script replacing san by san-ib (stands for infiniband) and /scratch by /scratch-ib
scp -r san-ib:path_to_files/file /scratch-ib/folder
More informations on infiniband here:
