Biowulf High Performance Computing at the NIH
Quantum Espresso: quantum simulation of materials on Biowulf

Quantum Espresso (QE) is an integrated suite of computer codes for electronic-structure calculations and materials modeling, based on density-functional theory, plane waves, and pseudopotentials (norm-conserving, ultrasoft, and projector-augmented wave). The Quantum Espresso distribution contains the core packages PWscf (Plane-Wave Self-Consistent Field) and CP (Car-Parrinello) for the calculation of electronic-structure properties within Density-Functional Theory (DFT), using a Plane-Wave (PW) basis set and pseudopotentials.

References:

Documentation
Important Notes

Interactive job
Interactive jobs should be used for debugging, graphics, or applications that cannot be run as batch jobs.

Allocate an interactive session and run the program. The sample session below illustrates running the QE software on a molecular structure comprising a single water molecule:

[user@biowulf]$ sinteractive --mem=4g
NOTE: the current installation of the software is supposed to be run only on CPU node(s).
[user@@cn3393 ~]$ module load QE
Copy the input data files wat_esp_1.inp, H.pbe-hgh.UPF and O.pbe-hgh.UPF to the current directory:

[user@@cn3393 ~]$  cp $QE_DATA/1_H2O/* . 
Run the pw.x executable:
[user@@cn3393 ~]$ pw.x -i wat_esp_1.inp 
     Program PWSCF v.6.2 starts on 16Apr2018 at  7:52:43 

     This program is part of the open-source Quantum ESPRESSO suite
     for quantum simulation of materials; please cite
         "P. Giannozzi et al., J. Phys.:Condens. Matter 21 395502 (2009);
         "P. Giannozzi et al., J. Phys.:Condens. Matter 29 465901 (2017);
          URL http://www.quantum-espresso.org", 
     in publications or presentations arising from this work. More details at
     http://www.quantum-espresso.org/quote

     Serial version
     Reading input from wat_esp_1.inp

     Current dimensions of program PWSCF are:
     Max number of different atomic species (ntypx) = 10
     Max number of k-points (npk) =  40000
     Max angular momentum in pseudopotentials (lmaxx) =  3

     -------------------------------------
     Parameters for Dispersion Correction:
     -------------------------------------
       atom      VdW radius       C_6     

        H          1.892          4.857
        O          2.536         24.284

     gamma-point specific algorithms are used

     G-vector sticks info
     --------------------
     sticks:   dense  smooth     PW     G-vecs:    dense   smooth      PW
     Sum        8781    8781   2209               620293   620293   77605


     Title: 
     Trial 216 h20 with PBE-Grimme2                                             


     bravais-lattice index     =            1
     lattice parameter (alat)  =      35.4363  a.u.
     unit-cell volume          =   44498.3854 (a.u.)^3
     number of atoms/cell      =            3
     number of atomic types    =            2
     number of electrons       =         8.00
     number of Kohn-Sham states=            4
     kinetic-energy cutoff     =      22.0000  Ry
     charge density cutoff     =      88.0000  Ry
     convergence threshold     =      1.0E-06
     mixing beta               =       0.7000
     number of iterations used =            8  plain     mixing
     Exchange-correlation      = SLA-PW-PBX-PBC ( 1  4  3  4 0 0)
...

     PseudoPot. # 1 for  H read from file:
     ./H.pbe-hgh.UPF
     MD5 check sum: 5f350582108a937bec95f27f5610ab08
     Pseudo is Norm-conserving, Zval =  1.0
     Generated in analytical, separable form
     Using radial grid of  929 points,  0 beta functions with: 

     PseudoPot. # 2 for  O read from file:
     ./O.pbe-hgh.UPF
     MD5 check sum: 631417d4da25374a53828d646ac41374
     Pseudo is Norm-conserving, Zval =  6.0
     Generated in analytical, separable form
     Using radial grid of 1095 points,  1 beta functions with: 
                l(1) =   0

     atomic species   valence    mass     pseudopotential
        H              1.00     1.00000      H( 1.00)
        O              6.00    16.00000      O( 1.00)

     No symmetry found
...

     Initial potential from superposition of free atoms
     Check: negative starting charge=   -1.212224

     starting charge    7.99998, renormalised to    8.00000

     negative rho (up, down):  1.212E+00 0.000E+00
     Starting wfc are    6 randomized atomic wfcs

     total cpu time spent up to now is        2.2 secs

     Self-consistent Calculation

     iteration #  1     ecut=    22.00 Ry     beta= 0.70
     Davidson diagonalization with overlap
     ethr =  1.00E-02,  avg # of iterations =  2.0

     negative rho (up, down):  3.490E-01 0.000E+00

     total cpu time spent up to now is        4.0 secs

     total energy              =     -31.47173255 Ry
     Harris-Foulkes estimate   =     -31.82345694 Ry
     estimated scf accuracy    <       0.63423474 Ry

...

     iteration #  6     ecut=    22.00 Ry     beta= 0.70
     Davidson diagonalization with overlap
     ethr =  7.54E-07,  avg # of iterations =  2.0

     total cpu time spent up to now is       10.8 secs

     End of self-consistent calculation

          k = 0.0000 0.0000 0.0000 ( 38803 PWs)   bands (ev):

   -27.7376 -13.2516  -9.6027  -7.0179

     highest occupied level (ev):    -7.0179

!    total energy              =     -31.62592088 Ry
     Harris-Foulkes estimate   =     -31.62591797 Ry
     estimated scf accuracy    <       0.00000067 Ry

     The total energy is the sum of the following terms:

     one-electron contribution =     -66.95499179 Ry
     hartree contribution      =      34.34170719 Ry
     xc contribution           =      -7.86256326 Ry
     ewald contribution        =       8.84998046 Ry
     Dispersion Correction     =      -0.00005348 Ry

     convergence has been achieved in   6 iterations

     Forces acting on atoms (cartesian axes, Ry/au):

     atom    1 type  2   force =     0.12082671   -0.06367274   -0.29072877
     atom    2 type  1   force =    -0.15108633    0.17278535    0.07505754
     atom    3 type  1   force =     0.03025962   -0.10911261    0.21567123

     Total force =     0.469923     Total SCF correction =     0.000275
...

     PWSCF        :    11.13s CPU        11.29s WALL


   This run was terminated on:   7:52:54  16Apr2018            

=------------------------------------------------------------------------------=
   JOB DONE.
=------------------------------------------------------------------------------=


End the session:
[user@@cn3393 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$

Batch job
Most jobs should be run as batch jobs.

Create a batch input file (e.g. QE.sh). The example below shows how this can be done for a model mocecular system comprising 216 water molecules:

#!/bin/bash
#SBATCH --job-name=testQE
#SBATCH --error=testQE.err
#SBATCH --partition=multinode
#SBATCH --ntasks=16
#SBATCH --cpus-per-task=2
#SBATCH --mem-per-cpu=64g
#SBATCH --time=01-00:00:00
#SBATCH --cpus-per-task=2
cd /data/denisovga/QE
module load QE
module load openmpi/1.10.3/gcc-4.9.1
cp $QE_DATA/216_H2O/* .
export MPI_HOME=/usr/local/OpenMPI/1.10.3/gcc-4.9.1
export MPI_BIN=${MPI_HOME}/bin
${MPI_BIN}/mpirun -np 16 /usr/local/apps/QE/6.2.0-CPU/bin/pw.x -i wat_esp.inp > wat_esp.out

Submit this job using the Slurm sbatch command.

sbatch QE.sh