High-Performance Computing at the NIH
GitHub YouTube @nih_hpc RSS Feed
Q-Chem on Biowulf & Helix

Q-Chem is a comprehensive ab initio quantum chemistry package for accurate predictions of molecular structures, reactivities, and vibrational, electronic and NMR spectra. The new release of Q-Chem 4.0 represents the state-of-the-art of methodology from the highest performance DFT/HF calculations to high level post-HF correlation methods:

Q-Chem is a licensed product developed by Q-Chem. All the nodes on the Biowulf cluster are licensed to run Q-Chem.

After you run 'module load qchem', a set of example input files is available in $QC/samples.

On Helix

Q-Chem is a cpu-intensive program that should not be run on Helix.

OpenMP Batch job on Biowulf

Q-Chem can use threads or MPI processes to parallelize. Different methods within Q-Chem can utilize one or both kinds of parallelization. See the section on Running Q-Chem in parallel for more information.

OpenMP (threads) can utilize multiple cores on a node, but cannot run on multiple nodes. Sample batch script: (this file is called qchem.bat)

#!/bin/bash
#SBATCH --job-name="QC"
#SBATCH --mail-type=BEGIN,END

module load qchem/4.3

cd /data/$USER/qchem

# copy the sample data to this directory
cp ${QC}/samples/bsse/frgm_mp2_h2o_h2o_h2o.in .

# run qchem
qchem -nt $SLURM_CPUS_PER_TASK  frgm_mp2_h2o_h2o_h2o.in   frgm_mp2_h2o_h2o_h2o.in.out

Submit this job with

biowulf% sbatch --cpus-per-task=16  --threads-per-core=1  --gres=lscratch:100 qchem.bat

where:

-nt $SLURM_CPUS_PER_TASK tells Q-Chem to run $SLURM_CPUS_PER_TASK threads
--cpus-per-task=16 tells Slurm how many CPUs to allocate
--threads-per-core=1 specifies that one thread should be run on each physical core (i.e. ignore hyperthreading). This is usually recommended for cpu-intensive parallel jobs.
--gres=lscratch:100specifies that 100 GB of local scratch on the node should be allocated to this job. This parameter is required for all Qchem job submissions.

When the qchem module is loaded as part of a batch job, it sets the environment variable $QCLOCALSCR to '/lscratch/$SLURM_JOBID' . This directory is only created if you submit the job with --gres=lscratch:#. (see the User Guide section on using local disk.

MPI batch job on Biowulf

Q-Chem can also run MPI . Sample MPI batch script


#!/bin/bash
#SBATCH --job-name="QC"

module load qchem/4.3

cd /data/$USER/qchem

# copy the sample input file to this directory
cp  ${QC}/samples/sp/dft_b5050lyp_c4h6.in .

# run qchem
make-qchem-nodefile
export QCMACHINEFILE=`pwd`/qchem.$SLURM_JOBID.nodes
qchem -np $SLURM_NTASKS  dft_b5050lyp_c4h6.in   dft_b5050lyp_c4h6.in.out
rm $QCMACHINEFILE

Submit this job with:

biowulf% sbatch --ntasks=4 --ntasks-per-core=1  --gres=lscratch:100 qchem.bat

where:

make-qchem-nodefile a script that writes a file containing a list of nodes to be used
$QCMACHINEFILE environment variable used by Q-Chem to determine the nodes on which to run
-np $SLURM_NTASKS tells Q-Chem to run $SLURM_NTASKS MPI processes
--ntasks=4 tells Slurm how many tasks (MPI processes) to run
--ntasks-per-core=1 tells Slurm to run only one task on each physical core (i.e. ignore hyperthreading). This is usually recommended for cpu-intensive parallel jobs.
--gres=lscratch:100specifies that 100 GB of local scratch on the node should be allocated to this job. This parameter is required for all Qchem job submissions.

Documentation

Q-Chem 4.3 manual