PyCharm is a popular IDE for the Python programming language, and is developed by Jetbrains.
Remote development means running the back-end of a development environment on a remote device other than the user's personal device.
Remote development using PyCharm is officially supported by Jetbrains, although still in Beta at the time of writing this document.
It is assumed that:
- The user is already somewhat familiar with the PyCharm IDE
- The user has access to either PyCharm Professional or PyCharm Edu
- The user is comfortable manually editing configuration files
Table of Contents
Configuring your ssh connection to the batch nodes
In order to be able to run the PyCharm backend on a batch node, PyCharm needs to know how to connect to it. Since connecting directly to batch nodes is not possible, we need to configure the login node as a proxy in your SSH config.
Essentially, this means that your computer will SSH to the login node, which will then forward the connection to the batch node.
On Unix-based systems, your SSH config is typically located at
Similarly, on Windows, your SSH config will probably be located at
Add the following snippet there:
Host snellius Port 22 HostName snellius.surf.nl Host tcn* HostName %h ProxyJump snellius
Changing node type
tcn to the name of the node 'flavour' you actually want to allocate. You can find a list of node flavours here
Host lisa Port 22 HostName lisa.surfsara.nl Host r*n* HostName %h ProxyJump lisa
All Lisa nodes have the format r*n*, for example: r38n4
Allocate an interactive node
This process is more broadly described here, but it boils down to allocating an interactive node using the
salloc -n 32 -t 4:00:00
SLURM will inform you whether the node allocation has been succesfull:
robertsc@login4:~$ salloc -n 1 -c 4 -p shared -t 06:00:00 salloc: Pending job allocation 8485336 salloc: job 8485336 queued and waiting for resources salloc: job 8485336 has been allocated resources salloc: Granted job allocation 8485336 salloc: Waiting for resource configuration salloc: Nodes r38n4 are ready for job
Pay attention to the name of the node (r38n4 in this example) that gets allocated to you, you'll need it later.
You will be fully charged for the allocated resources, even if you are not actively running any Python scripts through PyCharm. Consider how many cores you really need for testing your code, since with interactive work you are leaving resources idle most of the time.
Connect PyCharm to the allocated node
Click "Connect via SSH"
Click theicon next to the <New Connection> bar,
and then thebutton on the next page.
In the SSH configurations screen:
- Enter your username
- Fill in the node name as Host
- set "Authentication type" to "OpenSSH config and authentication agent"
- click "OK".
Select the added configuration from the drop-down menu, and then click "Check connection and Continue", after which a connection to the node will be made
The first time you connect to a specific node, you will be asked if you trust the RSA key. Press "Yes".
You will be asked to fill in your password twice: once to login to Snellius/Lisa, and once to login to the allocated node.
In the drop-down menu, select PyCharm as IDE.
Next, fill in the directory path of your project as "Project directory".
Click "Download and Start IDE".
PyCharm will download and launch itself on the interactive node.
If everything went right, we find ourselves in the familiar PyCharm environment:
It is worth pointing out that looking in the bottom right of our Development Environment, PyCharm displays the bits/second it is sending and receiving from our node.
As a double check, we can see that PyCharm launched itself on the node if we run the
htop command on the interactive node:
Adding a Python interpreter to PyCharm
In general, adding a Python interpreter to your remote PyCharm project works no different than adding one to a local project.
To add a Python interpreter to your PyCharm project, first go to the general PyCharm settings, and then under "Project" go to "Python interpreter".
There, you can select the flavour of environment you're used to.
We will list some ways to add a Python interpreter to your project which may not immediately be intuitive.
System Python (i.e. from a module)
We need to find the path to the Python binary, which loaded when we execute
module load Python/<xxx>.We can find out its exact location with the
which command like so:
$ module load 2021 $ module load Python/3.9.5-GCCcore-10.3.0 $ which python /sw/arch/Debian10/EB_production/2021/software/Python/3.9.5-GCCcore-10.3.0/bin/python
Then, paste the above path at System Interpreter → triple dots '...' menu → Select Python Interpreter
Existing Conda environment on Lisa/Snellius
Equally, we can find out the path to our Python binary with the with the
which command, after activating our conda environment:
$ module load 2021 $ module load Miniconda3/4.9.2 $ conda activate 2D-VQ-AE-2 $ which python /home/robertsc/.conda/envs/2D-VQ-AE-2/bin/python
The correct conda executable path should select itself automatically.
You should now be setup to work with your project!
Any time you execute any code, your code will be running on your allocated node.