The latest upgrade to this partition consists of 6 nodes. Hardware details:

• 2 x Intel Xeon Gold 6140 CPU @ 2.30GHz (18 cores each)
• 1.5 TB memory
• 1.8 TB SSD local disk
• 10 Gb/s ethernet

Interactive sessions on the Biowulf cluster are allocated using the sinteractive command if they need a walltime shorter than 36h.

NIMH users who need interactive sessions with walltimes longer than 36h can use spersist instead:

[user@local_computer]$ ssh biowulf.nih.gov
...
[user@biowulf]$ spersist --cpus-per-task=4 --mem=16g
salloc.exe: Pending job allocation 24833446
salloc.exe: job 24833446 queued and waiting for resources
salloc.exe: job 24833446 has been allocated resources
salloc.exe: Granted job allocation 24833446
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn1501 are ready for job
[user@cn1501 ~]$ 

spersist sessions

• have a default allocation of 2 CPUs and 4GB of memory
• do not have a walltime limit but do end when the login node is rebooted
• may share some resources with other spersist sessions on the same node
• are allocated analogously to sinteractive sessions
• replace long-lived interactive sessions on Felix
• run on a limited number of nodes in the persist partition funded by NIMH

We recommend allocating no less than 4 CPUs and 16 GB of memory for spersist sessions which will be used for graphical applications.

Like sinteractive sessions, spersist sessions are terminated if their controlling terminal exits. This happens, for example, if you disconnect from the NIH network to go home with your laptop. To maintain the spersist sessions even when you disconnect, they have to be attached to a persistent session on the biowulf login node. There are two ways of creating such persistent sessions:

1. A persistent graphical session can be created with the NoMachine NX client. From within an NX session on the login node, start a terminal and allocate an spersist session. Since NX sessions can be kept active until the login node is rebooted, your spersist session started from within NX will also remain active. In addition, this allows the execution of graphical applications from your spersist session.
2. A text based persistent session can be created with tmux (tmux crash course, quick guide) or screen. The choice between tmux and screen is a matter of personal taste. As with NX, you start your spersist session from a tmux/screen window and then disconnect from your tmux/screen session before logging out. Then you can reconnect to the login node and re-attach the tmux/screen session where your spersist session will be waiting for you.

Like sinteractive, spersist makes it easy to set up tunnels between compute nodes and the login node (also see our tunneling documentation). Such tunnels are necessary for applications that start a server you would want to connect to from your local workstation. Examples include Jupyter, Tensorboard, and Paraview amongst others. Example session:

user@biowulf:~$ spersist --tunnel
salloc.exe: Pending job allocation 26710013
salloc.exe: job 26710013 queued and waiting for resources
salloc.exe: job 26710013 has been allocated resources
salloc.exe: Granted job allocation 26710013
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn1501 are ready for job

Created 1 generic SSH tunnel(s) from this compute node to 
biowulf for your use at port numbers defined 
in the $PORTn ($PORT1, ...) environment variables.


Please create a SSH tunnel from your workstation to these ports on biowulf.
On Linux/MacOS, open a terminal and run:

    ssh  -L 45000:localhost:45000 biowulf.nih.gov

For Windows instructions, see https://hpc.nih.gov/docs/tunneling

user@cn1501:~$ echo $PORT1
45000
user@cn1501:~$

While you have active spersist or sinteractive sessions with tunnels, the reconnect_tunnels command will reconstruct an ssh command you can use to connect all required tunnels from your local workstation to biowulf. See our tunneling documentation for details.

NoMachine NX

To run lightweight graphical applications such as RStudio, please use NoMachine to connect to the login node and start your spersist session from within the NX session. This is analogous to using graphical applications with sinteractive

VNC

For more demanding graphics applications, VNC servers are available on the 'spersist nodes'. Using VNC requires an extra step. In your Biowulf shell, run spersist as before with the --vnc flag, then create a tunnel to the biowulf login node from your workstation. You are now ready to start up a VNC client from your workstation and connect to the VNC session. Note that you will have to supply your NIH username and password when prompted by the VNC client.

Sample session (user input in bold):

[user@biowulf ~]$ spersist --cpus-per-task=4 --mem=16g --vnc
salloc.exe: Pending job allocation 24833781
salloc.exe: job 24833781 queued and waiting for resources
salloc.exe: job 24833781 has been allocated resources
salloc.exe: Granted job allocation 24833781
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn1501 are ready for job
[+] Loading TurboVNC 
Starting VNC server ... please be patient...
VNC server started on display 1 port 5901
VNC configured with SSH forwarding.

Please create a tunnel from your workstation to biowulf.nih.gov port 45001:
On Linux/MacOS, open a terminal and run:

        ssh -L 45001:localhost:45001 biowulf.nih.gov

For Windows instructions see https://hpc.nih.gov/docs/tunneling

Then connect your VNC client to localhost port 45001

The VNC connection will terminate when this shell exits.
[user@cn1501 ~]$

VNC based spersist sessions cannot be started from within an NX session.

Firefox

The firefox browser is available on the spersist nodes, but not from within the desktop environment menu. Open a terminal within the desktop environment and run module load firefox to make it available, then run firefox to start the browser.

If Firefox opens with the error dialog saying "Firefox is already running, but is not responding...", this is likely due to your having Firefox open elsewhere on the HPC systems (like the Biowulf login node or another spersist session). Closing the other session or using a different Firefox profile will get around this issue.

Please send questions and comments to staff@hpc.nih.gov