Friendly Interactive Jobs

Locales: Data Centers and Campus RC University of Vermont
,
1

I’m analyzing some data but it is too large to fit on my personal computer. How can I use an interactive session on the cluster that is not just a command line interface?

OR:

I’m debugging some code but I can’t run the code on my personal compute for some reason. How can I debug code on the cluster without using a command line / vim / emac / nano /

Will be self answered.

2

An alternative to running interactive jobs via the command line interface is to setup remote debugging through an IDE.

I will walk you through the core details of how to do this with Microsoft Visual Studio Code.

Start by installing https://code.visualstudio.com
Then install the Remote Development Pack:
Visual Studio Code Remote Development
Remote Development - Visual Studio Marketplace

Almost ready to go!

Now if you open visual studio code, you should be able to ssh into the login node and edit some source code.
HOWEVER:
The login node should not be used for any computational intensive tasks!

To actually do remote data analytics / remote debugging / remote code development we need to add support for ssh-ing into the compute nodes.

The compute nodes are not (at least at the VACC) are not accessible from the open internet directly. We need to ssh tunnel through the login node.

This is relatively easy to set up.

On your computer create a config file in your ~/.ssh/ directory.

Then add entries like the following:

Host vacc-user1
  HostName vacc-user1.uvm.edu
  User <username>

Host <nodename>
  User <username>
  ProxyCommand ssh vacc-user1 nc <nodename> 22

Now, at your command line, you can type ssh vacc-user1 to ssh into vacc-user1.uvm.edu with your username.
Similarly, if you have a job running on nodename, then you can also run ssh nodename to ssh into it by tunneling through vacc-user.

Now, from Visual Studio Code, if you refresh you should see nodename show up as an ssh target.
Make sure you have started a job on nodename prior to trying to connect through Visual Studio Code or your ssh connection attempt will be denied (SLURM prevents ssh into compute nodes unless you have a job running on them).

Now you are ready to go!
If you want to do interactive development (e.g. via a Jupyter notebook), VSCode has an extension for that. After you install it, simply select some text and then press SHIFT+ENTER to run it in an interactive session.