BigDFT
Overview
BigDFT is a DFT massively parallel electronic structure code (GPL license) using a wavelet basis set. Wavelets form a real space basis set distributed on an adaptive mesh (two levels of resolution in our implementation). GTH or HGH pseudopotentials are used to remove the core electrons.
Using BigDFT with GPU on RAAD GPU Cluster
You need to follow below steps to start working with GPU on RAAD for bigDFT. Apparently no changes are required in source code to port your code to GPU. Please make sure you have access to submit jobs to Graphics queue, or you may request for the access by emailing to Helpdesk.
Steps for using GPU Version of bigDFT on RAAD
1) Create your job file as below; (do not copy paste)
#PBS -S /bin/bash
#PBS -q graphics
#PBS -N BARTtest2
#PBS -l select=1:ncpus=16:mem=48Gb
#PBS -l place=free:excl
#PBS -l walltime=168:00:00
module load bigDFT/162_GPU
cd $PBS_O_WORKDIR
source bart.sh
mpirun -n 16 bart > bart.out
2) In the same directory, issue below command;
echo "ACCEL OCLGPU" >> input.perf
3) Finally submit your job file using qsub.
Your job will be scheduled on gfx1 to gfx6.
To verify if your job is running on GPU or not
- ssh into the respective node where your job is running
- Type 'nvidia-smi' w/o quotes and hit enter
- You should see 'bart' processes listed there indicating these 'bart' processes are running on GPU.
Remarks
- This is the OpenCL version of BigDFT GPU code
- Make sure if you already have access to Graphics queue, so you will be able to submit jobs to this queue.
- You can further tune the Job file and change the parameters for number of CPU's, Memory and introduce Multiple OpenMP threads.