IMPORTANT: (May 2017) thirdparty toolboxes not automatically loaded by default
Starting with our installation of R2017a, third party MATLAB toolboxes are no longer loaded by default.
You must either load their individual modules before starting MATLAB, or use the nih_matmod command from within the MATLAB environment.
A matlab.all module is provided to load all toolboxes for backward compatibility with existing workflows that depend on all installed toolboxes being available.
IMPORTANT: (May 2016) Only compiled Matlab code or interactive Matlab jobs allowed on cluster
MATLAB licenses are now only allocated to interactive programming sessions. MATLAB code must be compiled before it can be submitted to the batch system. It is no longer necessary to allocate licenses with the licenses or L options. For details, see this announcement.

Licenses are checked out automatically when you use a MATLAB or its toolboxes. Licenses are returned when you exit your session, or when it times out after 4 hours of inactivity.
Before starting MATLAB, you may wish to check if licenses are available. To do so, check the current MATLAB License status or enter licenses
in the shell like so:
[user@cn1234 ~]$ licenses License Total Free    MATLAB 26 20 SIMULINK 1 1 Bioinformatics 3 3 Curve Fitting 4 4 Global Optimization 1 1 Image Processing 6 4 Builder for Java 1 1 Compiler 2 2 Neural Network 2 2 Optimization 4 4 Dist. Computing 4 4 Part. Diff. Eq. 2 2 Signal Processing 6 6 Simbiology 1 1 Statistic 13 13 Symbolic Math 2 2 System Ident. 2 2 Video and Image 1 0 Wavelet 2 2 IDL 20 20 ACEMD 256 256 Note that some licenses are reserved for use on Biowulf. More info at https://hpc.nih.gov/docs/userguide.html#licenses [user@cn1234 ~]$
When idle for 4 hours, you will see the following message displayed in the Command Window:
"Your MATLAB session has timed out. All license keys have been returned."
To resume your MATLAB session, just start using MATLAB again. If a license is available, MATLAB checks it out automatically. Likewise, as you use the toolboxes and other products that you had checked out, MATLAB obtains license keys for these products as well. If a license key is not available, MATLAB periodically attempts to check out a license, issuing warning messages. After 10 warnings, if a license key is still not available, MATLAB will save the workspace and exit.
As with other applications on the HPC systems, MATLAB is managed using environment modules. To see which versions of MATLAB and thirdparty toolboxes are available, type:
[user@cn1234 ~]$ module avail matlab  /usr/local/lmod/modulefiles  matlabFiji/1.50a matlabeeglab/14.1.2 (D) matlabxjview/8.14 matlabFiji/1.51n (D) matlabkilosort/8738ef7 matlab.all/2015a matlabMBEToolbox/3.0 matlabnpy/524bd14 matlab.all/2016b matlabMIToolbox/3.0.2 matlabpronto/2.0.1 matlab.all/2017b matlabPGEToolbox/3.1 matlabsigprofiler/2.3.2.0 matlab.all/2018a (D) matlabSinCHet/1.0 matlabspm/12 matlab.all/2018b matlabalexnet/1.0 matlabspm/12.6906 matlab/2015a matlabcaffe/1.0.0rc3 matlabspm/12.7219 (D) matlab/2016b matlabcaffe/1.0.0 (D) matlabtimestudio/3.18 matlab/2017b matlabconn/17a matlabvgg/17.1.0 matlab/2018a (D) matlabeeglab/13.4.4b matlabwtsimf/1.4 matlab/2018b (L) Where: L: Module is loaded D: Default Module Use "module spider" to find all possible modules. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". [user@cn1234 ~]$
To select a module, type:
[user@cn1234 ~]$ module load matlab/[ver]
where [ver] is an optional version specification. Without [ver] the default MATLAB version is loaded.
Please note that, for 2017a and later, the complete set of thirdparty toolboxes will not be automatically loaded unless you choose the matlab.all module or load them individually.
To load a thirdparty toolbox, like SPM, for use with MATLAB >= 2017a, do:[user@cn1234 ~]$ module load matlabspm
If you have already begun a MATLAB session, there is no need to restart. The loading can be accomplished using the nih_matmod function:
>> nih_matmod avail Fiji/1.50a alexnet/1.0 kilosort/8738ef7 spm/12.7219 (L,D) Fiji/1.51n (D) caffe/1.0.0rc3 npy/524bd14 timestudio/3.18 MBEToolbox/3.0 caffe/1.0.0 (D) pronto/2.0.1 vgg/17.1.0 MIToolbox/3.0.2 conn/17a sigprofiler/2.3.2.0 wtsimf/1.4 PGEToolbox/3.1 eeglab/13.4.4b spm/12 xjview/8.14 SinCHet/1.0 eeglab/14.1.2 (D) spm/12.6906 Where: D: Default Module L: Module is loaded Use "module spider" to find all possible modules. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". >> nih_matmod load spm
Use help nih_matmod from within MATLAB for complete usage information.
back to top
Because MATLAB is not permitted on the Biowulf login node, users must request resources on a compute node.
First, establish an ssh or XWindows connection to the Biowulf login node. When running an XWindows session on Biowulf, remember to use the X or Y option with your ssh command.
Then use the sinteractive command to request resources on a compute node:
[user@biowulf ~]$ sinteractive salloc.exe: Pending job allocation 15323416 salloc.exe: job 15323416 queued and waiting for resources salloc.exe: job 15323416 has been allocated resources salloc.exe: Granted job allocation 15323416 salloc.exe: Waiting for resource configuration salloc.exe: Nodes cn1640 are ready for job [user@cn1640 ~]$
By default, the sinteractive command only allocates a few cpus and a small amount of memory. You can request more cpus and memory with the cpuspertask=ncpus and mem=GBg options respectively. See sinteractive h for a complete list of options.
To run the MATLAB integrated development environment (IDE also known as the Java virtual machine), an X Windows connection is required. NoMachine's NX is the XWindows client currently recommended by staff for Windows and Mac. When used to start a GNOME desktop session, NX is optimized to provide a fast, responsive MATLAB environment.
HINT: Debugging XWindows clients
If you have trouble starting the MATLAB IDE, ensure that your XWindows client is working properly by typing 'xclock' in the shell. This shouldn't be an issue with a GUI based client like NX.
After connecting to a compute node, load the MATLAB module.
[user@hcn1234 ~]$ module load matlab [user@cn1234 ~]$ matlab&
Including & allows you to continue using the terminal while the MATLAB application is running. You should now see a MATLAB splash screen followed by the IDE:
HINT: Failure to Launch
If you receive an error message that contains "Failure to Launch Desktop Class", cd to $HOME/.matlab, remove the directory R2015b (or appropriate version) and try again.
It is also possible to run MATLAB interactively on the commandline, without the IDE. This would be useful if you do not wish to use XWindows.
[user@cn1234 ~]$ module load matlab [user@cn1234 ~]$ matlab nodisplay < M A T L A B (R) > Copyright 19842018 The MathWorks, Inc. R2018b (9.5.0.944444) 64bit (glnxa64) August 28, 2018 To get started, type doc. For product information, visit www.mathworks.com. >> quit [user@cn1234 ~]$
back to top

It is not possible to submit jobs containing uncompiled MATLAB code to the batch system. However, it is still possible to write and execute shell scripts containing uncompiled MATLAB code from within an interactive session. This method might be useful for running a few instances of MATLAB on several cpus of a single node, and it will only require one MATLAB license. But for longrunning jobs or jobs requiring more than a few instances of MATLAB, you should compile your code instead. Here, a basic calculation (the pythagorean theorem) is run using multiple approaches. 
The simplest (but perhaps least useful) way to run a MATLAB job in the background would be to hardcode variables and MATLAB commands directly into a script. Even though this method is not very practical it serves as a useful example to MATLAB users unfamiliar with bash scripting:
#!/bin/bash # this file is called hyp1.sh module load matlab matlab nojvm<<EOF a = 3; b = 4; H = sqrt(a^2 + b^2) exit EOF
You could run this job in the background of an interactive session like so:
[user@cn0001 ~]$ ./hyp1.sh > hyp1_output 2>&1 &
HINT: Permission denied error
If you receive an error such as this:
bash: ./hyp1.sh: Permission denied
Use the chmod command to make the file executable, like so:
[user@cn0001]$ chmod 755 hyp1.sh
In this case, the output will be redirected to hyp1_output. The computation will proceed in the background allowing the you to run another session of MATLAB interactively or run other MATLAB programs in the background using the same license. If you use this strategy to run a few instances of MATLAB in parallel you must be careful not to overload your allocated CPUs.
HINT:`EOF' warning
If a message similar to the following is found in your output file, make sure to remove any trailing whitespace from both of the EOF lines.
/var/spool/slurm/slurmd/job48226/slurm_script: line 10: warning: heredocument at line 5 delimited by endoffile (wanted `EOF')
You can set up a batch script that contains MATLAB commands and pass variables to the job at the time of submission. This might be useful for very simple MATLAB analyses. For example:
#!/bin/bash # this file is called hyp2.sh a=$1 b=$2 echo "a is $a, b is $b"; module load matlab matlab nojvm<<EOF H = sqrt($a^2 + $b^2) exit EOF
In this script the values of a and b are expected to be passed from the command line. This is accomplished like so:
[user@cn0001 ~]$ ./hyp2.sh 22 5 > hyp2_output 2>&1 &
Similar to the previous example, the output is directed to hyp2_output.
Of course it is possible to write your own functions and call them in a shell script using the same syntax as above. Let's assume you write a function and save it in hyp.m
function H = hyp(a,b) % return the hypotenuse (H) % from the legs (a and b) % convert chars to doubles % (this is necessary when % code is compiled) if ischar(a), a = str2double(a); end if ischar(b), b = str2double(b); end H = sqrt(a^2 + b^2)
To run the code in the background, you could write a script like so:
#!/bin/bash # this file is called hyp3.sh a=$1 b=$2 module load matlab matlab nojvm<<EOF cd /full/path/to/hyp.m hyp($a,$b); exit EOF
Similar to the last example you would run this code in the backround with:
[user@cn0001 ~]$ ./hyp3.sh 8 4 > hyp3_output 2>&1 &
This method allows you to develop analyses of any complexity in MATLAB (rather than hardcoding commands into shell scripts) and run them in the background of an interactive session.
One common approach is to write a MATLAB function that accepts a file name as input, loads the file (containing all variables and data), performs some analysis, and then saves the analyzed data to a new .mat file. This minimizes the complexity of the input that must be supplied through a shell script. In this case, the _output file will only be used for diagnosing a debugging problems, because the output will be saved to a new .mat file specified by the user in the function.
back to top
Long running jobs, or jobs that are "embarrassingly parallel" should be compiled and submitted to the batch system. Compiling your code removes the need for MATLAB and toolbox licenses.
The MATLAB compiler generates a shell script called run_X.sh (where the file containing the MATLAB source code is named X.m). This script is used to run the compiled code. It requires at least one argument specifying the location of the MATLAB Component Runtime (MCR) and subsequent arguments are treated as input to the compiled MATLAB function. See the MATLAB compiler for details.
Compile hyp.m from the example above on a Biowulf compute node like so:
[user@cn1234 ~]$ module load matlab [user@cn1234 ~]$ mcc2 m hyp.m
You can now execute hyp.m by calling run_hyp.sh from the shell without using any MATLAB licenses like so:
[user@cn1234 ~]$ ./run_hyp.sh /usr/local/matlabcompiler/v95 3 4  Setting up environment variables  LD_LIBRARY_PATH is .:/usr/local/matlabcompiler/v95/runtime/glnxa64:/usr/local/matlabcompiler/v95/bin/glnxa64:/usr/local/matlabcompiler/v95/sys/os/glnxa64:/usr/local/matlabcompiler/v95/sys/opengl/lib/glnxa64 H = 5
The first argument gives the location of the MCR appropriate for code compiled in MATLAB version 2018b. The next two arguments are the inputs to hyp.m.
To submit the compiled code as a batch job, you could write another script:
#!/bin/bash # this file is called hyp4.sh # #SBATCH jobname=hypotenuse4 #SBATCH mailtype=BEGIN,END /full/pathname/run_hyp.sh /usr/local/matlabcompiler/v95 $a $b
Note the optional inclusion of #SBATCH options that set the job name and email alerts. You would then submit the job like so:
[user@biowulf ~]$ sbatch export=a=3,b=4 /full/pathname/hyp4.sh
Note that the values of a and b are passed at the time of job submission using the export option with sbatch. In this example, the standard output and standard error from the job are automatically directed to a file called slurm#.out where # is the assigned jobid.
It might be cumbersome to write a shell script to call another shell script that will invoke the compiled executable. In this instance, the sbatch wrap option comes in handy. It allows you to pass sbatch a command instead of a filename. In this way, you could submit run_hyp.sh as a batch job directly without the need for an intermediary script.
[user@biowulf ~]$ sbatch wrap=\"/full/pathname/run_hyp.sh /usr/local/matlabcompiler/v95 3 4\"
Note the double quotes escaped with backslashes that enclose the command. This is necessary because the command contains white space.
For more information on using the sbatch command for SLURM, visit the Job Submission section of the Biowulf User Guide.
back to top
The swarm program is a convenient way to submit large numbers of jobs. With swarm you can run over 1,000 instances of your compiled MATLAB code distributed across the cluster. You create a swarm command file containing a single line for each independent job. The swarm program will then package up the commands into batch jobs and submit them to Slurm for you.
Only compiled MATLAB jobs can be run using swarm. To run a swarm based on the example above, create a swarm command file named hyp.swarm with each line containing a single command:
Special considerations for running a swarm include changing the location of the MCR cache and using the Matlab compiler runtime from local scratch. This is exemplified below.
export MCR_CACHE_ROOT=/lscratch/$SLURM_JOB_ID; \ test d /lscratch/$SLURM_JOB_ID/v95  tar C /lscratch/$SLURM_JOB_ID xf /usr/local/matlabcompiler/v95.tar.gz \ && /full/pathname/run_hyp.sh /lscratch/$SLURM_JOB_ID/v95 3 4 export MCR_CACHE_ROOT=/lscratch/$SLURM_JOB_ID; \ test d /lscratch/$SLURM_JOB_ID/v95  tar C /lscratch/$SLURM_JOB_ID xf /usr/local/matlabcompiler/v95.tar.gz \ && /full/pathname/run_hyp.sh /lscratch/$SLURM_JOB_ID/v95 5 6 export MCR_CACHE_ROOT=/lscratch/$SLURM_JOB_ID; \ test d /lscratch/$SLURM_JOB_ID/v95  tar C /lscratch/$SLURM_JOB_ID xf /usr/local/matlabcompiler/v95.tar.gz \ && /full/pathname/run_hyp.sh /lscratch/$SLURM_JOB_ID/v95 7 8 export MCR_CACHE_ROOT=/lscratch/$SLURM_JOB_ID; \ test d /lscratch/$SLURM_JOB_ID/v95  tar C /lscratch/$SLURM_JOB_ID xf /usr/local/matlabcompiler/v95.tar.gz \ && /full/pathname/run_hyp.sh /lscratch/$SLURM_JOB_ID/v95 9 10 export MCR_CACHE_ROOT=/lscratch/$SLURM_JOB_ID; \ test d /lscratch/$SLURM_JOB_ID/v95  tar C /lscratch/$SLURM_JOB_ID xf /usr/local/matlabcompiler/v95.tar.gz \ && /full/pathname/run_hyp.sh /lscratch/$SLURM_JOB_ID/v95 11 12
Submit this file to the batch system with the command:
[user@biowulf ~]$ swarm gres lscratch:10 f hyp.swarm
Please note that, with this type of swarm, subjob packing (p option) should not be used. Bundling (b), however, would work as expected.
For more detailed info about running swarms of MATLAB jobs on Biowulf, including common pitfalls to avoid and strategies for automation, see this recorded training session.
The MATLAB Parallel Computing Toolbox enables you to develop distributed and parallel MATLAB applications and execute them using multiple cores in a single node or to utilize the graphical processing units (GPUs) of a properly equipped machine.
Please NOTE:
1. In it's current configuration, the Parallel Computing Toolbox does not scale beyond a single node. This will allow your job to run up to 16 times faster on the norm partition. This may be sufficient for many jobs. But this toolbox will not allow you to run jobs on multiple nodes. To run jobs of larger scale, use sbatch or swarm.
2. Only four Parallel Computing Toolbox licenses are available for users to share. As a workaround you could compile your parallel code as described above and submit it as a batch job or even in a swarm.
3. Within MATLAB and in online documentation this toolbox is referred to as the Parallel Computing Toolbox. Within the licensing software it is referred to as the Distributed Processing Toolbox. There is a related Mathworks product that is not currently installed on our systems that extends the functionality of the Parallel Computing Toolbox called the Distributed Computing Server. This has been the source of confusion.
See MATLAB Parallel Computing Toolbox for examples on using the toolbox on biowulf.
back to top>> ver  MATLAB Version: 9.5.0.944444 (R2018b) MATLAB License Number: 35 Operating System: Linux 3.10.0693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 Java Version: Java 1.8.0_152b16 with Oracle Corporation Java HotSpot(TM) 64Bit Server VM mixed mode  MATLAB Version 9.5 (R2018b) Simulink Version 9.2 (R2018b) Bioinformatics Toolbox Version 4.11 (R2018b) Computer Vision System Toolbox Version 8.2 (R2018b) Curve Fitting Toolbox Version 3.5.8 (R2018b) Deep Learning Toolbox Version 12.0 (R2018b) Image Processing Toolbox Version 10.3 (R2018b) MATLAB Compiler Version 7.0 (R2018b) MATLAB Compiler SDK Version 6.6 (R2018b) Optimization Toolbox Version 8.2 (R2018b) Parallel Computing Toolbox Version 6.13 (R2018b) Signal Processing Toolbox Version 8.1 (R2018b) Statistics and Machine Learning Toolbox Version 11.4 (R2018b) Symbolic Math Toolbox Version 8.2 (R2018b) System Identification Toolbox Version 9.9 (R2018b) Wavelet Toolbox Version 5.1 (R2018b) >>
In addition, these 3rd party toolboxes are installed in some versions of MATLAB:
The SPM12 Toolbox also has the following toolboxes installed: