Skip to content

Commit 65f8ba5

Browse files
acumpelikacumpelik
authored
Merge pull request #74 from LevensteinLab/viggy
fixing tutorial
2 parents fa15d95 + 89acae5 commit 65f8ba5

1 file changed

Lines changed: 34 additions & 44 deletions

File tree

‎docs/Resources/prnn_tutorial.md‎

Lines changed: 34 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -2,49 +2,39 @@
22

33
There are some extra steps in managing the requirements needed for the [Predictive RNN](https://github.com/LevensteinLab/pRNN) project. This guide should help you get everything set up on Misha.
44

5-
1. Ensure you have an account on Misha, and log on. Make sure to allocate an interactive session (e.g. `salloc -t 2:00:00`)!
6-
1. Make sure you've also set up SSH cloning from the cluster, because you'll need to clone repositories into your storage. We're using SSH clone links from here out.
7-
2. See the page on HPCs for more information on this.
8-
2. Clone the `pRNN` repository into your `project` folder (`git clone git@github.com:LevensteinLab/pRNN.git`), and 👁️watch the repository so you can be notified of any updates.
9-
3. In your `HOME` directory (accessible by typing `cd ~`):
10-
1. Make a folder for the virtual enviroment `mkdir venvs`.
11-
2. Clone Dan's fork of gym-minigrid (`git clone git@github.com:dlevenstein/gym-minigrid.git gym-minigrid-dan-fork`).
12-
4. Create a conda environment with Python 3.9.
5+
1. Ensure you have an account on Misha, and log on.
6+
1. Make sure to allocate an interactive session (e.g. `salloc -t 2:00:00`)! If are using the VS Code Proxy (as described on the [vs_code.md](https://levensteinlab.github.io/Lab-Handbook/Resources/vs_code/) page), you're already on a compute node.
7+
2. Make sure you've also set up SSH cloning from the cluster, because you'll need to clone repositories into your storage. We're using SSH clone links from here out.
8+
3. See the page on HPCs for more information on this.
9+
3. Clone the `pRNN` package into your `project` folder. This is accessible by typing `cd ~/../../project/levenstein/YOUR_NETID/`. This brings you to the `/gpfs/radev/project/levenstein/YOUR_NETID`.
10+
1. `git clone git@github.com:LevensteinLab/pRNN.git`
11+
2. 👁️ Watch the repository so you can be notified of any updates!
12+
4. Also clone these external repositories into that project directory:
13+
1. Dan's fork of gym-minigrid (`git clone git@github.com:dlevenstein/gym-minigrid.git gym-minigrid-dan-fork`).
14+
2. Farama Foundation version of minigrid. This is the most up-to-date version. (`git clone git@github.com:Farama-Foundation/Minigrid.git`)
15+
5. Create a conda environment with Python 3.9.
1316
1. Misha doesn't come with Python 3.9 out of the box, and we need this version for later dependencies.
1417
2. Type `module load miniconda` to ensure conda is loaded for use.
15-
3. Run `conda create -n base39 python=3.9` to make an environment called `base39` with python 3.9 installed.
16-
4. Activate it (`conda activate base39`).
17-
5. Make the `PredictiveReplay` environment. This is done by running `make_venv.sh`, which is in the pRNN repo.
18-
1. First comment out/change the following lines. This will ensure that the correct environments remain active.
19-
1. `module --force purge`
20-
2. `module load python/3.9`
21-
3. `virtualenv $VENV_DIR` changed to `virtualenv -p ~/.conda/envs/base39/bin/python $VENV_DIR` (this ensures we're making the virtualenv with python 3.9)
22-
4. Add `pip install virtualenv` before the line in step 3 (line 18).
23-
5. In line 49, change `#pip3 install gym-minigrid` to `pip3 install -e ../../gym-minigrid-dan-fork` (this ensure we're using Dan's forked version)
24-
2. Run `bash make_venv.sh` to create the environment.
25-
6. Load the environment. This is done by loading `load_venv.sh`
26-
1. Comment out the following lines again. This will prevent the underlying conda environment from being purged.
27-
1. `module --force purge`
28-
2. `module load python/3.9`
29-
2. Run `bash load_venv.sh`
30-
7. Activate the `venv` environment. Run `source ~/venvs/PredictiveReplay_39/bin/activate`. by now you should have two parenthesized environment names by each terminal prompt, like this `(PredictiveReplay_39) (base39) [yourNetID@blahblah.misha]$ `.
31-
8. Double check that `gym-minigrid` is installed correctly.
32-
1. Run `pip list | grep minigrid`.
33-
2. If nothing shows up, it didn't install correctly. Run `cd ~/gym-minigrid-dan-fork`, then `pip3 install -e .` to install it into the environment.
34-
9. In your terminal (with both environments activated), run:
35-
1. `pip install "pip<24.1"`
36-
2. `pip install setuptools==59.5.0 wheel==0.37.0`
37-
3. `pip3 install gym==0.21.0 --no-binary :all:`
38-
10. Rerun `bash make_venv.sh` then `bash load_venv.sh`.
39-
1. *Sidebar from Viggy: To be completely honest, I have no idea why steps 9 & 10 work. I spent many hours trying to fix a random numpy error that pops up in the `tutorial.ipynb`, but doing steps 9 and 10 fixes it, despite not reinstalling a different version of numpy. My hypothesis is that the downgraded version of `pip`, `setuptools`, and `wheel` help us install this version of `gym`, and when we rerun `make_venv.sh`, we can reinstall the correct version of numpy (`1.22.4` instead of `1.26.x` that it likes to do). It could be an issue with the order in which we install the packages at first. If you continue to have issues, try steps 9 and 10 again, and "turning \[your kernel\] on and off again". Or ask me, and we can figure it out. One day, I will figure out how to do this with fewer steps.*
40-
11. Finally, make sure that the jupyter notebook kernel can recognize this environment.
41-
1. Run `pip install ipykernel` (this will install it inside *both* environments).
42-
2. Run `ipython kernel install --user --name=prnn-kernel`. To allow jupyter notebooks to pick it up.
43-
3. If you run `jupyter kernelspec list`, you should see `prnn-kernel` as an option.
44-
4. You may need to restart your VSCode or kernel here.
45-
12. You should be able to run `tutorial.ipynb` now. This contains a single run setting up some environments.
46-
1. You may get a `SyntaxError`. This is because there's a small typo to fix. If you go to line 62 in `prnn > utils > Shell.py`, you may see an extra `):`. Delete that.
47-
48-
49-
50-
18+
3. Run `conda create -n prnn_tutorial python=3.9` to make an environment called `prnn_tutorial` with python 3.9 installed.
19+
4. Activate it (`conda activate prnn_tutorial`).
20+
6. In the project folder, create your own separate repo for your work. We recommend using a template such as Patrick Mineault's `true-neutral-cookiecutter` to model the repository after.
21+
1. `pip install cookiecutter`
22+
2. `cookiecutter gh:patrickmineault/true-neutral-cookiecutter`
23+
3. It will prompt you for input regarding the name of the repository.
24+
7. Now, we need to populate our environment with the correct packages. Some of them depend on an older version of pip, so it's important to do the steps in order. Run the following commands:
25+
1. `pip install "pip<24.1" setuptools==59.5.0 wheel==0.37.0`
26+
2. `pip install gym==0.21.0 --no-binary gym`
27+
3. `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --no-cache-dir`
28+
8. We'll also need to install the external repositories from step 3 into environment.
29+
1. `pip3 install -e ~/../../project/levenstein/YOUR_NETID/gym-minigrid-dan-fork`
30+
2. `pip3 install -e ~/../../project/levenstein/YOUR_NETID/Minigrid`
31+
9. Install the `pRNN` package itself into the environment.
32+
1. Navigate to the repository, then `pip install -e .`
33+
10. Make sure that the jupyter notebook kernel can recognize this environment.
34+
1. `pip install ipykernel`
35+
2. Run `ipython kernel install --user --name=prnn_tutorial`. To allow jupyter notebooks to pick it up.
36+
3. If you run `jupyter kernelspec list`, you should see `prnn_tutorial` as an option.
37+
4. You may need to restart your VSCode or kernel here.
38+
1. If you're on VSCode, you can do this fast with `Cmd + Shift + P` to open the Command Palette and then `Developer: Reload Window`.
39+
11. Your environment should be all set to run example code provided with the package, or to start going through the tutorials. The quickstart tutorial can be found [here](https://github.com/LevensteinLab/pRNN/blob/main/docs/quickstart.rst). This contains an example training run.
40+
1. If you would like to run through quickstart interactively, you will need to copy `pRNN/examples/tutorial.ipynb` to your `prnn_tutorial` folder. This can be done with `rsync -r ../pRNN/examples/ .`

0 commit comments

Comments
 (0)