Cyfronet

1. Sign up at https://portal.plgrid.pl/
2. Apply for grant and activate it at https://portal.plgrid.pl/services
   • Note: Access to the Ares cluster at the Cyfronet centre [grant required]

$ ssh <login>@ares.cyfronet.pl

<login> = username starting with “plg”

Add infor about ssh key

• module avail : lists available libraries/packages
• module list : lists loaded libraries/packages
• module load <package> : loads specified package
• module spider <keyword> : searches available packages with the keyword
• module purge : purges the environment

• hpc-grants : shows available grants, resource allocations, consumed resourced
• hpc-fs : shows available storage
• hpc-jobs : shows currently pending/running jobs
• hpc-jobs-history : shows information about past jobs

• $HOME
• $SCRATCH
• $PLG_GROUPS_STORAGE (starts with “plgg”)

Add how to share files

1. Load necessary modules:

       module load python/2.7.18-gcccore-11.2.0
       module load libtool
       module load emacs
       module load gcc/11.2.0
       module load gcccore/11.2.0
       module load cmake/3.22.1-gcccore-11.2.0
       module load gsl/2.7-gcc-11.2.0
        

2. Move to $SCRATCH or $PLG_GROUPS_STORAGE (preferred):

    $ cd $PLG_GROUPS_STORAGE/<groupname> 

3. Download bootstrap script and make it executable:

    $ wget https://herwig.hepforge.org/downloads/herwig-bootstrap
    $ chmod +x herwig-bootstrap 

4. Start an interactive node session (Easier to debug if installation fails):

    $ srun -p plgrid-now -t 5:00:00 --mem=5GB -A <grantname-cpu> --pty /bin/bash 

   Note: Only one interactive plgrid-now session can be run at a time. Adjust the session duration and memory allocation as required. The above specified parameters should suffice in general.

5. Run bootstrap and exit session when completed:

    $ ./herwig-bootstrap -j4 $PWD/Herwig
    $ exit 

6. Activate Herwig environment :

    $ source Herwig/bin/activate 

Run script (run.sh) :

     #!/bin/bash
     #SBATCH --job-name=Sample_Run
     #SBATCH --nodes=1
     #SBATCH --ntasks-per-node=1
     #SBATCH --cpus-per-task=1
     #SBATCH --mem-per-cpu=1GB
     # set wall time to 1 day
     #SBATCH --time=1-00:00:00
     ## Name of partition
     #SBATCH -p plgrid
     cd <Run directory>
     source <path/to/>Herwig/bin/activate
     export HW=Herwig
     ${HW} run *.run --seed=$seed -N 1000000 > /dev/null &
     HWPID="$!"
     wait "${HWPID}"

1. Submit parallel jobs:

   $ sbatch --array=0-5 -A <grantname-suffix> run.sh

   • Suffix = cpu, cpu-bigmem, gpu
2. sprio: show status of pending jobs
3. sshare: show usage of available grants
4. Run interactive bash session:

   $ srun -p plgrid-now -t 1:00:00 --mem=1GB -A <grantname-cpu> --pty /bin/bash

1. Detailed information about Ares - https://docs.cyfronet.pl/display/~plgpawlik/Ares#Ares-AccesstoAres
2. Slurm documentation - https://slurm.schedmd.com/quickstart.html