From the official documentation
This package provides an implementation of the inference pipeline of AlphaFold v2.0. This is a completely new model that was entered in CASP14 and published in Nature. For simplicity, we refer to this model as AlphaFold throughout the rest of this document.
References:
- 2022-04-22: Version 2.2.0 becomes the default
- See official announcement
- Switched to new AlphaFold-Multimer models with reduced numbers of clashes on average and slightly increased accuracy.
- removed
--is_prokaryote_list
option fromrun_singularity
andmsa
as the prokaryotic pairing algorithm did not actually improve accuracy on average - Added
--num_multimer_predictions_per_model=N
option torun_singularity
. Runs N predictions per multimer model - each with different seeds. Defaults to 5 and will increase runtime - Added
--config_model
option torun_singularity
. This allows users to use a customizedalphafold/model/config.py
to alphafold for tuning certain parameters
- 2022-02-22: Version 2.1.2 becomes the default
- No changes to network weights
- Relaxation now defaults to using GPU
- Added options for controlling relaxation:
--[no]run_relax
and--[no]enable_gpu_relax
- New script to generate the multiple sequence alignments (MSAs) only (
msa
). This is highly recommended as MSA generation consumes about 60% of runtime and does not make use of GPU. The workflow is to run msa generation on CPU nodes and then run model prediction on a GPU node withrun_singularity --use_precomputed_msas ...
. This script also supports adding extra options to hhblits or overriding some alphafold defaults with option--hhblits_extra_opts
. Note that this script cannot run on x2650 nodes because it depends on an AVX2 hhblits build.
- 2021-11-15: Version 2.1.1 becomes the default
- the
--preset
option was renamed to --model_preset - the
--db_preset=reduced_dbs
option is now supported for all alphafold versions - The inofficial
--use_ptm
option became obsolete with introduction of the official--model_preset
option and was removed - This version uses the new model parameters (2021-10-27) released with 2.1.1. This includes the parameters for the multimer model. The 2.0.X modules will continue to use the previous parameters.
- the
- 2021-11-14: Database update
- Databases were updated in place: pdb mmcif and pdb70 (211110). New databases only used by multimer model: pdb_seqres, uniprot
- 2021-10-19: Added
--use_ptm
option to run_singularity - Use the pTM models, which were fine-tuned to produce pTM (predicted TM-score) and predicted aligned error values alongside their structure predictions.
- 2021-10-18: Adaptation of the alphafold_advanced notebook from ColabFold available in version 2.0.1.
- Allows prediction of protein complexes with unmodified alphafold network weights. So far only an interactive notebook is available. See below for more details
- 2021-10-01: Version 2.0.0-24-g1d43aaf was tagged as 2.0.1
- The modules for 2.0.0-24-g1d43aaf and 2.0.1 point to the same installation since the release was tagged after this revision was installed.
- 2021-09-21: Version 2.0.0-24-g1d43aaf becomes the default version on biowulf
- Most noticable change should be the inclusion of pLDDT in the PDB B-factor column
- 2021-09-16: Database update (in place)
- The following databases used by alphafold were updated in place: mgnify (2018_12 to 2019_05), pdb70 (200401 to 210901), pdb mmcif (210717 to 210915, 1969 additional structures), uniclust30 (2018_08 to 2021_06 from http://gwdu111.gwdg.de/~compbiol/uniclust/2021_06/). Uniref90 and BFD are unchanged.
- Module Name: alphafold2 (see the modules page for more information)
- Alphafold2 first runs some multithreaded analyses using up to 8 CPUs before running model inference on the GPU. At this point these steps can't be separated and therefore for the first step of the job the GPU will remain idle.
- Example files in
$ALPHAFOLD2_TEST_DATA
- Reference data in
/fdb/alphafold2/
- Alphafold2 expects input to be upper case amino acid sequences
Allocate an interactive session and run the program. In this example the whole pipeline
including multiple sequence alignment and model predictions are run with run_singularity
on a GPU node. This is
not the most efficient way of doing it. Starting with version 2.1.2 we make available a separate script that only does
the alignment and can be run on CPU nodes reducing GPU time by more than 2-fold.
[user@biowulf]$ sinteractive --mem=60g --cpus-per-task=8 --gres=lscratch:100,gpu:v100x:1 salloc.exe: Pending job allocation 46116226 salloc.exe: job 46116226 queued and waiting for resources salloc.exe: job 46116226 has been allocated resources salloc.exe: Granted job allocation 46116226 salloc.exe: Waiting for resource configuration salloc.exe: Nodes cn3144 are ready for job [user@cn3144]$ module load alphafold2/2.2.0
To predict the structure of a protein already in PDB without using its
experimental structure as a template set max_template_date
to
before the release date of the structure. For example, to reproduce the T1049
CASP14 target with 144 aa. On a V100x this prediction runs for about 1h.
[user@cn3144]$ run_singularity --helpfull # use --help for shorter help message Singularity launch script for Alphafold. flags: /usr/local/apps/alphafold2/2.2.0/bin/run_singularity: --[no]benchmark: Run multiple JAX model evaluations to obtain a timing that excludes the compilation time, which should be more indicative of the time required for inferencing many proteins. (default: 'false') --db_preset: <full_dbs|reduced_dbs>: Choose preset MSA database configuration - smaller genetic database config (reduced_dbs) or full genetic database config (full_dbs) (default: 'full_dbs') --[no]dry_run: Print command that would have been executed and exit. (default: 'false') --[no]enable_gpu_relax: Run relax on GPU if GPU is enabled. (default: 'true') --fasta_paths: Paths to FASTA files, each containing a prediction target that will be folded one after another. If a FASTA file contains multiple sequences, then it will be folded as a multimer. Paths should be separated by commas. All FASTA paths must have a unique basename as the basename is used to name the output directories for each prediction. (a comma separated list) --gpu_devices: Comma separated list of devices to pass to NVIDIA_VISIBLE_DEVICES. (default: 'all') --max_template_date: Maximum template release date to consider (ISO-8601 format: YYYY-MM-DD). Important if folding historical test sets. --model_config: Use this file instead of default alphafold/model/config.py --model_preset: <monomer|monomer_casp14|monomer_ptm|multimer>: Choose preset model configuration - the monomer model, the monomer model with extra ensembling, monomer model with pTM head, or multimer model (default: 'monomer') --num_multimer_predictions_per_model: How many predictions (each with a different random seed) will be generated per model. E.g. if this is 2 and there are 5 models then there will be 10 predictions per input. Note: this FLAG only applies if model_preset=multimer (default: '5') (an integer) --output_dir: Path to a directory that will store the results. --[no]run_relax: Whether to run the final relaxation step on the predicted models. Turning relax off might result in predictions with distracting stereochemical violations but might help in case you are having issues with the relaxation stage. (default: 'true') --[no]use_gpu: Enable NVIDIA runtime to run with GPUs. (default: 'true') --[no]use_precomputed_msas: Whether to read MSAs that have been written to disk instead of running the MSA tools. The MSA files are looked up in the output directory, so it must stay the same between multiple runs that are to reuse the MSAs. WARNING: This will not check if the sequence, database or configuration have changed. (default: 'false') ... absl.logging: --[no]alsologtostderr: also log to stderr? (default: 'false') --log_dir: directory to write logfiles into (default: '') --logger_levels: Specify log level of loggers. The format is a CSV list of `name:level`. Where `name` is the logger name used with `logging.getLogger()`, and `level` is a level name (INFO, DEBUG, etc). e.g. `myapp.foo:INFO,other.logger:DEBUG` (default: '') --[no]logtostderr: Should only log to stderr? (default: 'false') --[no]showprefixforinfo: If False, do not prepend prefix to info messages when it's logged to stderr, --verbosity is set to INFO level, and python logging is used. (default: 'true') --stderrthreshold: log messages at this level, or more severe, to stderr in addition to the logfile. Possible values are 'debug', 'info', 'warning', 'error', and 'fatal'. Obsoletes --alsologtostderr. Using --alsologtostderr cancels the effect of this flag. Please also note that this flag is subject to --verbosity and requires logfile not be stderr. (default: 'fatal') -v,--verbosity: Logging verbosity level. Messages logged at this level or lower will be included. Set to 1 for debug logging. If the flag was not set or supplied, the value will be changed from the default of -1 (warning) to 0 (info) after flags are parsed. (default: '-1') (an integer) ... [user@cn3144]$ run_singularity \ --model_preset=monomer_casp14 \ --fasta_paths=$ALPHAFOLD2_TEST_DATA/T1049.fasta \ --max_template_date=2020-05-14 \ --output_dir=$PWD [user@cn3144]$ tree T1049 T1049/ ├── [user 1.1M] features.pkl ├── [user 4.0K] msas │ ├── [user 33K] bfd_uniclust_hits.a3m │ ├── [user 18K] mgnify_hits.sto │ └── [user 121K] uniref90_hits.sto ├── [user 170K] ranked_0.pdb # <-- shown below ├── [user 170K] ranked_1.pdb ├── [user 170K] ranked_2.pdb ├── [user 171K] ranked_3.pdb ├── [user 170K] ranked_4.pdb ├── [user 330] ranking_debug.json ├── [user 170K] relaxed_model_1.pdb ├── [user 170K] relaxed_model_2.pdb ├── [user 170K] relaxed_model_3.pdb ├── [user 170K] relaxed_model_4.pdb ├── [user 171K] relaxed_model_5.pdb ├── [user 11M] result_model_1.pkl ├── [user 11M] result_model_2.pkl ├── [user 11M] result_model_3.pkl ├── [user 11M] result_model_4.pkl ├── [user 11M] result_model_5.pkl ├── [user 771] timings.json ├── [user 87K] unrelaxed_model_1.pdb ├── [user 87K] unrelaxed_model_2.pdb ├── [user 87K] unrelaxed_model_3.pdb ├── [user 87K] unrelaxed_model_4.pdb └── [user 87K] unrelaxed_model_5.pdb [user@cn3144]$ exit
The processes prior to model inference on the GPU consumed up to 40 GB of memory for this protein. Memory requirements will vary with different size proteins.

ranked_0.pdb
, blue) aligned with the actual
structure for this protein
(6Y4F, green)The next example shows how to run a multimer model (available from version 2.1.1). For improved efficiency we pre-generate the multiple sequence alignment on a CPU node using the msa script available since version 2.1.2 on biowulf and then do model prediction only on a GPU node. Note that the separation of MSA generation and model prediction works for monomers and multimers. The example used is a recently published PI3K structure.
[user@biowulf]$ sinteractive --mem=60g --cpus-per-task=8 --gres=lscratch:100 salloc.exe: Pending job allocation 46116226 salloc.exe: job 46116226 queued and waiting for resources salloc.exe: job 46116226 has been allocated resources salloc.exe: Granted job allocation 46116226 salloc.exe: Waiting for resource configuration salloc.exe: Nodes cn3144 are ready for job [user@cn3144]$ module load alphafold2/2.2.0 [user@cn3144]$ cat $ALPHAFOLD2_TEST_DATA/pi3k.fa >sp|P27986|P85A_HUMAN Phosphatidylinositol 3-kinase regulatory subunit alpha OS=Homo sapiens OX=9606 GN=PIK3R1 PE=1 SV=2 MSAEGYQYRALYDYKKEREEDIDLHLGDILTVNKGSLVALGFSDGQEARPEEIGWLNGYN ETTGERGDFPGTYVEYIGRKKISPPTPKPRPPRPLPVAPGSSKTEADVEQQALTLPDLAE QFAPPDIAPPLLIKLVEAIEKKGLECSTLYRTQSSSNLAELRQLLDCDTPSVDLEMIDVH VLADAFKRYLLDLPNPVIPAAVYSEMISLAPEVQSSEEYIQLLKKLIRSPSIPHQYWLTL QYLLKHFFKLSQTSSKNLLNARVLSEIFSPMLFRFSAASSDNTENLIKVIEILISTEWNE RQPAPALPPKPPKPTTVANNGMNNNMSLQDAEWYWGDISREEVNEKLRDTADGTFLVRDA STKMHGDYTLTLRKGGNNKLIKIFHRDGKYGFSDPLTFSSVVELINHYRNESLAQYNPKL DVKLLYPVSKYQQDQVVKEDNIEAVGKKLHEYNTQFQEKSREYDRLYEEYTRTSQEIQMK RTAIEAFNETIKIFEEQCQTQERYSKEYIEKFKREGNEKEIQRIMHNYDKLKSRISEIID SRRRLEEDLKKQAAEYREIDKRMNSIKPDLIQLRKTRDQYLMWLTQKGVRQKKLNEWLGN ENTEDQYSLVEDDEDLPHHDEKTWNVGSSNRNKAENLLRGKRDGTFLVRESSKQGCYACS VVVDGEVKHCVINKTATGYGFAEPYNLYSSLKELVLHYQHTSLVQHNDSLNVTLAYPVYA QQRR >sp|P42336|PK3CA_HUMAN Phosphatidylinositol 4,5-bisphosphate 3-kinase catalytic subunit alpha isoform OS=Homo sapiens OX=9606 GN=PIK3CA PE=1 SV=2 MPPRPSSGELWGIHLMPPRILVECLLPNGMIVTLECLREATLITIKHELFKEARKYPLHQ LLQDESSYIFVSVTQEAEREEFFDETRRLCDLRLFQPFLKVIEPVGNREEKILNREIGFA IGMPVCEFDMVKDPEVQDFRRNILNVCKEAVDLRDLNSPHSRAMYVYPPNVESSPELPKH IYNKLDKGQIIVVIWVIVSPNNDKQKYTLKINHDCVPEQVIAEAIRKKTRSMLLSSEQLK LCVLEYQGKYILKVCGCDEYFLEKYPLSQYKYIRSCIMLGRMPNLMLMAKESLYSQLPMD CFTMPSYSRRISTATPYMNGETSTKSLWVINSALRIKILCATYVNVNIRDIDKIYVRTGI YHGGEPLCDNVNTQRVPCSNPRWNEWLNYDIYIPDLPRAARLCLSICSVKGRKGAKEEHC PLAWGNINLFDYTDTLVSGKMALNLWPVPHGLEDLLNPIGVTGSNPNKETPCLELEFDWF SSVVKFPDMSVIEEHANWSVSREAGFSYSHAGLSNRLARDNELRENDKEQLKAISTRDPL SEITEQEKDFLWSHRHYCVTIPEILPKLLLSVKWNSRDEVAQMYCLVKDWPPIKPEQAME LLDCNYPDPMVRGFAVRCLEKYLTDDKLSQYLIQLVQVLKYEQYLDNLLVRFLLKKALTN QRIGHFFFWHLKSEMHNKTVSQRFGLLLESYCRACGMYLKHLNRQVEAMEKLINLTDILK QEKKDETQKVQMKFLVEQMRRPDFMDALQGFLSPLNPAHQLGNLRLEECRIMSSAKRPLW LNWENPDIMSELLFQNNEIIFKNGDDLRQDMLTLQIIRIMENIWQNQGLDLRMLPYGCLS IGDCVGLIEVVRNSHTIMQIQCKGGLKGALQFNSHTLHQWLKDKNKGEIYDAAIDLFTRS CAGYCVATFILGIGDRHNSNIMVKDDGQLFHIDFGHFLDHKKKKFGYKRERVPFVLTQDF LIVISKGAQECTKTREFERFQEMCYKAYLAIRQHANLFINLFSMMLGSGMPELQSFDDIA YIRKTLALDKTEQEALEYFMKQMNDAHHGGWTTKMDWIFHTIKQHALN [user@cn3144]$ msa \ --fasta_paths=$ALPHAFOLD2_TEST_DATA/pi3k.fa \ --max_template_date=2021-11-01 \ --model_preset multimer \ --output_dir=$PWD ...snip... [user@cn3144]$ tree pi3k pi3k |-- [user 90M] features.pkl `-- [user 4.0K] msas |-- [user 4.0K] A | |-- [user 5.1M] bfd_uniclust_hits.a3m | |-- [user 237K] mgnify_hits.sto | |-- [user 12M] pdb_hits.sto | |-- [user 276M] uniprot_hits.sto | `-- [user 42M] uniref90_hits.sto |-- [user 4.0K] B | |-- [user 4.9M] bfd_uniclust_hits.a3m | |-- [user 2.0M] mgnify_hits.sto | |-- [user 4.0M] pdb_hits.sto | |-- [user 435M] uniprot_hits.sto | `-- [user 79M] uniref90_hits.sto `-- [user 2.1K] chain_id_map.json [user@cn3144]$ exit
Next, model prediction on a GPU using the MSAs computed in the previous step. We will run only 2 copies of each model with different seeds to reduce runtime relative to the (new in 2.2.0) default of 5. Runtime scales approximately linearly with the number of predictions per model.
[user@biowulf]$ sinteractive --mem=40g --cpus-per-task=6 --gres=lscratch:30,gpu:v100x:1 --time=12:00:00 [user@cn3144]$ module load alphafold2/2.2.0 [user@cn3144]$ run_singularity \ --use_precomputed_msas \ --fasta_paths=$ALPHAFOLD2_TEST_DATA/pi3k.fa \ --max_template_date=2021-11-01 \ --model_preset multimer \ --num_multimer_predictions_per_model=2 \ --output_dir=$PWD ...snip... [user@cn3144]$ exit

To get an idea of runtimes of alphafold2 we first ran 4 individual proteins
on all our available GPUs. The proteins ranged in size from 144 aa to 622 aa. Note
that for all but the smallest protein, K80 GPUs were not suitable and should not
be considered for alphafold2. These tests were run with default settings except for
a fixed --max_template_date=2021-07-31

The runtime to run all 4 protein on a V100x GPU with 8 CPUs and 60GB of memory was 3.2h, slightly less than the individual runtimes of the 4 proteins run separately. For this one job we also increased the number of CPUs to 16 or the number of GPUs to 2, neither of which appeared to shorted the runtime
The resource usage profile of the combined alphafold2 pipeline in our testing thus far is suboptimal and variable. Steps probably should be segregated into individual jobs with proper resources. We hope to optimize this in the future
Note: when running multiple alphafold predictions please use the msa script available for alphafold >=2.1.2 to precompute the multiple sequence alignments on CPU nodes and use GPU only for model predictions. This is shown in the example below as 2 batch jobs.
#!/bin/bash module load alphafold2/2.1.2 msa \ --model_preset=monomer \ --fasta_paths=$ALPHAFOLD2_TEST_DATA/T1049.fasta \ --max_template_date=2020-05-14 \ --output_dir=$PWD
Submit this job using the Slurm sbatch command.
jobid=$(sbatch --cpus-per-task=8 --mem=60g --gres=lscratch:30 alphafold2_msa.sh)
#!/bin/bash module load alphafold2/2.1.2 run_singularity \ --use_precomputed_msas \ --model_preset=monomer \ --fasta_paths=$ALPHAFOLD2_TEST_DATA/T1049.fasta \ --max_template_date=2020-05-14 \ --output_dir=$PWD
Submit this job using the Slurm sbatch command.
sbatch --cpus-per-task=6 --partition=gpu --mem=40g --gres=gpu:v100x:1,lscratch:30 --dependency=afterany:$jobid alphafold2_model.sh
This is an adaptation of the Alphafold2_advanced Colabfold notebook to biowulf. Currently only the notebook is available but a batch wrapper is in development.
Notes
- Currently only available for version alphafold 2.0.1
- jackhmmer is not yet implemented. The notebook currently only uses mmseqs2 via the online API to create the input MAS
- Like the colabfold notebook this does not use templates and uses unmodified alphafold weights trained on single proteins.
- Colab specific input forms had to be removed. Instead, input has to be provided in normal cells with options described in the text
The jupyter function will use an existing tunnel if it has been set up
with the --tunnel
option to sinteractive
. If there is no pre-existing tunnel, it will attempt to
set one up itself. That means it is possible to start the jupyter server in a
batch job and obtain the command to set up the tunnel from your computer to
the login node from the batch output. See our
tunneling documentation for more information
Example use
[user@biowulf ~]$ sinteractive --gres=lscratch:20,gpu:v100x:1 -c16 --mem=60g --tunnel salloc.exe: Pending job allocation 25316671 salloc.exe: job 25316671 queued and waiting for resources salloc.exe: job 25316671 has been allocated resources salloc.exe: Granted job allocation 25316671 salloc.exe: Waiting for resource configuration salloc.exe: Nodes cn3299 are ready for job srun: error: x11: no local DISPLAY defined, skipping error: unable to open file /tmp/slurm-spank-x11.25316671.0 slurmstepd: error: x11: unable to read DISPLAY value Created 1 generic SSH tunnel(s) from this compute node to biowulf for your use at port numbers defined in the $PORTn ($PORT1, ...) environment variables. Please create a SSH tunnel from your workstation to these ports on biowulf. On Linux/MacOS, open a terminal and run: ssh -L 46243:localhost:46243 user@biowulf.nih.gov For Windows instructions, see https://hpc.nih.gov/docs/tunneling [user@cn3144]$ module load alphafold2/2.0.1 [user@cn3144]$ af_colabfold_advanced jupyter name_of_notebook_copy.ipynb
The af_colabfold_advanced
script creates a copy of the notebook
in your current directory that you can work with and starts the jupyter server
up. Once the provided ssh command is used to establish a tunnel to the login node
exactly as when using regular jupyter notebooks, the notebook can be visited
at the address provided by jupyter during startup