From a07e1e81ae67ec56d0d6cb549ab0ef8cab439452 Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 14:46:54 +0800 Subject: [PATCH 1/7] Add VSCode Remote SSH connection guide Added a guide for connecting VSCode to Apptainer Jupyter Notebook environments via SSH, including steps for SSH key generation, configuration, and connection. --- .../09_vscode_remote_ssh_torch.mdx | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx diff --git a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx new file mode 100644 index 0000000000..316031a227 --- /dev/null +++ b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx @@ -0,0 +1,81 @@ +# Connect VSCode to Apptainer Jupyter Notebook Environments +Some users want to remotely work on Jupyter Notebooks using their Singularity kernels. This is a tedious, error prone process. + +## Step 1: Configure SSH for key-based ssh login +Generate an ssh key with ssh-keygen +``` +$ ssh-keygen -t rsa -b 4096 +``` + +Generating public/private rsa key pair. +``` +Enter file in which to save the key (~/.ssh/id_rsa): ~/.ssh/torch_NetID +``` + +This will create two files on your local computer: +``` +~/.ssh/torch_NetID +~/.ssh/torch_NetID.pub +``` + +Append the public key `(~/.ssh/torch_NetID.pub)` to the Torch `~/.ssh/authorized_keys`. +On your local computer: +``` +$ cat ~/.ssh/torch_NetID.pub +``` + +Copy the full output. Then log in to Torch: +On Torch: +``` +$ mkdir -p ~/.ssh** +$ chmod 700 ~/.ssh** +$ nano ~/.ssh/authorized_keys** +``` + +If authorized_keys already contains an HPC-managed key, do not remove or edit it. Add your custom SSH public key below the existing key, preferably under the line: +``` +# Add custom SSH keys below +``` + +Paste the full public key from ~/.ssh/torch_NetID.pub into authorized_keys, save the file, then run: +``` +$ chmod 600 ~/.ssh/authorized_keys +``` + +## Step 2: Configure local ssh config +Update your `~/.ssh/config` accordingly: + +``` +Host torch + HostName login.torch.hpc.nyu.edu + User NetID + ForwardAgent yes + StrictHostKeyChecking no + UserKnownHostsFile /dev/null + LogLevel ERROR + IdentityFile ~/.ssh/torch_NetID + IdentitiesOnly yes + ServerAliveInterval 60 +``` + +Please note that sometimes the IdentityFile won't work unless you use the standard id_rsa and id_rsa.pub. If the specified file doesn't work try it with id_rsa. + +## Step 3: Connect +Install the VSCode Remote - SSH extension. + +Configure Remote - SSH settings: +Check off Remote.SSH: Lockfiles In Tmp +Uncheck Remote.SSH: Use Flock + +Because Torch requires Microsoft device authentication during SSH login, VSCode may time out before you finish entering the device code. If this happens, increase the Remote - SSH connection timeout to 90 seconds. In VSCode Settings, search for Remote.SSH: Connect Timeout and set it to 120. + +Before connecting, open https://microsoft.com/devicelogin and keep it ready. In VSCode Remote Explorer, connect to the Torch login node: torch. When VSCode prompts for Microsoft device authentication, enter the code on the device login page. This may take a few attempts. VSCode may also prompt more than once during setup because it can open multiple SSH connections. + +After authentication, VSCode may take a while to install or start the VSCode server on Torch. If the connection fails during this step, try connecting again. Once the server is installed successfully, future connections are usually more consistent. + +Once connected, open a terminal in the VSCode remote session. The prompt should show that you are on a Torch login node, for example: +``` +[NetID@torch-login-... ~]$ +``` + + From 2767dc16d87bd022e056db6dbc3f5dcd3528b9f9 Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 14:59:49 +0800 Subject: [PATCH 2/7] Create 10_use_jupyter_notebooks_in_vscode_on_torch.mdx --- ...e_jupyter_notebooks_in_vscode_on_torch.mdx | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx diff --git a/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx new file mode 100644 index 0000000000..89c167571b --- /dev/null +++ b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx @@ -0,0 +1,27 @@ +# Use Apptainer Jupyter Notebooks in VSCode on Torch + +This page builds on the general VSCode access instructions in [Connect VSCode to Torch](./09_vscode_remote_ssh_torch). Before following this workflow, complete that setup and connect VSCode to the Torch login node. + +## Step 1: Install VSCode Jupyter support + +In the VSCode remote SSH window connected to Torch, install or enable the VSCode Jupyter extension. + +## Step 2: Start an interactive job + +Open a terminal in the VSCode remote session. The terminal should start on a Torch login node. + +Submit a dummy or interactive job from the login node. When the job starts, note the assigned compute node. + +## Step 3: Connect to the compute node + +From the VSCode terminal, ssh to the assigned compute node. + +## Step 4: Open the notebook + +Open the target .ipynb notebook in VSCode. + +## Step 5: Select the Apptainer kernel + +Use the notebook kernel picker to select the Apptainer kernel for the notebook. + +If the Apptainer kernel does not appear, confirm that the kernel is installed and visible on Torch before retrying in VSCode. From 9614a9fdd0612fb7f15487140f312688c380659c Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 15:50:15 +0800 Subject: [PATCH 3/7] Update 10_use_jupyter_notebooks_in_vscode_on_torch.mdx --- ...e_jupyter_notebooks_in_vscode_on_torch.mdx | 44 +++++++++++++++---- 1 file changed, 35 insertions(+), 9 deletions(-) diff --git a/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx index 89c167571b..3009468f54 100644 --- a/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx +++ b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx @@ -1,26 +1,52 @@ # Use Apptainer Jupyter Notebooks in VSCode on Torch -This page builds on the general VSCode access instructions in [Connect VSCode to Torch](./09_vscode_remote_ssh_torch). Before following this workflow, complete that setup and connect VSCode to the Torch login node. +This page builds on the general VSCode access instructions in 09_vscode_remote_ssh_torch. Before following this workflow, complete that setup and connect VSCode to the Torch login node. ## Step 1: Install VSCode Jupyter support -In the VSCode remote SSH window connected to Torch, install or enable the VSCode Jupyter extension. +After connecting VSCode to the Torch login node, open the Extensions view in the VSCode remote window and search for Jupyter. -## Step 2: Start an interactive job +Install or enable the Microsoft Jupyter extension in the remote SSH environment. If the extension is not already installed remotely, VSCode should show an option such as: -Open a terminal in the VSCode remote session. The terminal should start on a Torch login node. +Install in SSH: torch -Submit a dummy or interactive job from the login node. When the job starts, note the assigned compute node. +## Step 2: Start a dummy batch job and connect to the compute node -## Step 3: Connect to the compute node +After the Jupyter extension is installed or enabled, open a terminal in the VSCode remote session on a Torch login node. -From the VSCode terminal, ssh to the assigned compute node. +Interactive jobs on Torch may experience intermittent issues. Some compute nodes may work as expected, while others may not. As a workaround, submit a lightweight dummy batch job and then ssh to the allocated compute node after the job is running. -## Step 4: Open the notebook +For example: + +```bash +sbatch -c4 -t2:00:00 --mem=4G --account= --wrap "sleep infinity" +``` + +Replace with the Slurm account for your project. After submitting the job, Slurm will print a job ID, for example: + +```bash +Submitted batch job +``` + +Once the job is running, identify the assigned compute node and connect to it from the VSCode terminal using SSH. +```bash +squeue -u $USER +``` + +```bash +ssh csXXX +``` + +After connecting, the terminal prompt should show the compute node: +```bash +[NetID@csXXX ~]$ +``` + +## Step 3: Open the notebook Open the target .ipynb notebook in VSCode. -## Step 5: Select the Apptainer kernel +## Step 4: Select the Apptainer kernel Use the notebook kernel picker to select the Apptainer kernel for the notebook. From 784df985307d8dd545df4be221f44be1ce95ab78 Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 15:52:34 +0800 Subject: [PATCH 4/7] Update 09_vscode_remote_ssh_torch.mdx --- .../09_vscode_remote_ssh_torch.mdx | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx index 316031a227..35346b9c4e 100644 --- a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx +++ b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx @@ -3,49 +3,49 @@ Some users want to remotely work on Jupyter Notebooks using their Singularity ke ## Step 1: Configure SSH for key-based ssh login Generate an ssh key with ssh-keygen -``` +```bash $ ssh-keygen -t rsa -b 4096 ``` Generating public/private rsa key pair. -``` +```bash Enter file in which to save the key (~/.ssh/id_rsa): ~/.ssh/torch_NetID ``` This will create two files on your local computer: -``` +```bash ~/.ssh/torch_NetID ~/.ssh/torch_NetID.pub ``` Append the public key `(~/.ssh/torch_NetID.pub)` to the Torch `~/.ssh/authorized_keys`. On your local computer: -``` +```bash $ cat ~/.ssh/torch_NetID.pub ``` Copy the full output. Then log in to Torch: On Torch: -``` +```bash $ mkdir -p ~/.ssh** $ chmod 700 ~/.ssh** $ nano ~/.ssh/authorized_keys** ``` If authorized_keys already contains an HPC-managed key, do not remove or edit it. Add your custom SSH public key below the existing key, preferably under the line: -``` +```bash # Add custom SSH keys below ``` Paste the full public key from ~/.ssh/torch_NetID.pub into authorized_keys, save the file, then run: -``` +```bash $ chmod 600 ~/.ssh/authorized_keys ``` ## Step 2: Configure local ssh config Update your `~/.ssh/config` accordingly: -``` +```bash Host torch HostName login.torch.hpc.nyu.edu User NetID @@ -74,7 +74,7 @@ Before connecting, open https://microsoft.com/devicelogin and keep it ready. In After authentication, VSCode may take a while to install or start the VSCode server on Torch. If the connection fails during this step, try connecting again. Once the server is installed successfully, future connections are usually more consistent. Once connected, open a terminal in the VSCode remote session. The prompt should show that you are on a Torch login node, for example: -``` +```bash [NetID@torch-login-... ~]$ ``` From d0a42cc60d5e3ee2bef8460244426bed7a9c146d Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 15:58:48 +0800 Subject: [PATCH 5/7] Update 09_vscode_remote_ssh_torch.mdx --- .../09_vscode_remote_ssh_torch.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx index 35346b9c4e..6827b005e5 100644 --- a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx +++ b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx @@ -1,5 +1,5 @@ # Connect VSCode to Apptainer Jupyter Notebook Environments -Some users want to remotely work on Jupyter Notebooks using their Singularity kernels. This is a tedious, error prone process. +Some users want to remotely work on Jupyter Notebooks using their Apptainer kernels. This is a tedious, error prone process. ## Step 1: Configure SSH for key-based ssh login Generate an ssh key with ssh-keygen @@ -24,12 +24,12 @@ On your local computer: $ cat ~/.ssh/torch_NetID.pub ``` -Copy the full output. Then log in to Torch: +Copy the full output and log on to Torch. On Torch: ```bash -$ mkdir -p ~/.ssh** -$ chmod 700 ~/.ssh** -$ nano ~/.ssh/authorized_keys** +$ mkdir -p ~/.ssh +$ chmod 700 ~/.ssh +$ nano ~/.ssh/authorized_keys ``` If authorized_keys already contains an HPC-managed key, do not remove or edit it. Add your custom SSH public key below the existing key, preferably under the line: From 25e199edfd5d2f7e4a585cc1f073fd02bebcf6e5 Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 16:01:27 +0800 Subject: [PATCH 6/7] Update 10_use_jupyter_notebooks_in_vscode_on_torch.mdx --- .../10_use_jupyter_notebooks_in_vscode_on_torch.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx index 3009468f54..c88c5b79de 100644 --- a/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx +++ b/docs/hpc/06_tools_and_software/10_use_jupyter_notebooks_in_vscode_on_torch.mdx @@ -8,7 +8,7 @@ After connecting VSCode to the Torch login node, open the Extensions view in the Install or enable the Microsoft Jupyter extension in the remote SSH environment. If the extension is not already installed remotely, VSCode should show an option such as: -Install in SSH: torch +`Install in SSH: torch` ## Step 2: Start a dummy batch job and connect to the compute node @@ -22,13 +22,14 @@ For example: sbatch -c4 -t2:00:00 --mem=4G --account= --wrap "sleep infinity" ``` -Replace with the Slurm account for your project. After submitting the job, Slurm will print a job ID, for example: +Replace `` with the account for your project. After submitting the job, Slurm will print a job ID, for example: ```bash Submitted batch job ``` Once the job is running, identify the assigned compute node and connect to it from the VSCode terminal using SSH. +Check the job status and assigned node: ```bash squeue -u $USER ``` From bb9a3393a9ecfb82685b4c75cd5040aa93d4947b Mon Sep 17 00:00:00 2001 From: Amanda-dong <159391549+Amanda-dong@users.noreply.github.com> Date: Fri, 24 Jul 2026 16:11:10 +0800 Subject: [PATCH 7/7] Update 09_vscode_remote_ssh_torch.mdx --- docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx index 6827b005e5..44cae4e5f1 100644 --- a/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx +++ b/docs/hpc/06_tools_and_software/09_vscode_remote_ssh_torch.mdx @@ -1,4 +1,4 @@ -# Connect VSCode to Apptainer Jupyter Notebook Environments +# Connect VSCode to Torch Some users want to remotely work on Jupyter Notebooks using their Apptainer kernels. This is a tedious, error prone process. ## Step 1: Configure SSH for key-based ssh login @@ -67,7 +67,7 @@ Configure Remote - SSH settings: Check off Remote.SSH: Lockfiles In Tmp Uncheck Remote.SSH: Use Flock -Because Torch requires Microsoft device authentication during SSH login, VSCode may time out before you finish entering the device code. If this happens, increase the Remote - SSH connection timeout to 90 seconds. In VSCode Settings, search for Remote.SSH: Connect Timeout and set it to 120. +Because Torch requires Microsoft device authentication during SSH login, VSCode may time out before you finish entering the device code. If this happens, search for Remote.SSH: Connect Timeout and set it to 120. Before connecting, open https://microsoft.com/devicelogin and keep it ready. In VSCode Remote Explorer, connect to the Torch login node: torch. When VSCode prompts for Microsoft device authentication, enter the code on the device login page. This may take a few attempts. VSCode may also prompt more than once during setup because it can open multiple SSH connections.