Update TASK_INSTRUCTIONS.md to match the current template

TASK_INSTRUCTIONS.md

@@ -1,111 +1,104 @@
 # Instructions
 
-This is a guide on what to do after you have created a new task repository from the template. More in depth information about how to create a new task can be found in the [OpenProblems Documentation](https://openproblems.bio/documentation/create_task/).
+This is a guide on what to do after you have created a new task repository from the [task_template](https://github.com/openproblems-bio/task_template). More in depth information about how to create a new task can be found in the [OpenProblems documentation](https://openproblems.bio/documentation/create_task/getting-started/).
 
 ## Requirements
 
-A list of required software to start developing a new task can be found in the [OpenProblems Requirements](https://openproblems.bio/documentation/create_task/requirements).
+A list of required software to start developing a new task can be found in the [OpenProblems requirements](https://openproblems.bio/documentation/fundamentals/requirements).
 
 ## First things first
 
 ### `_viash.yaml`
 
-1. Update the `name` field to the name of the task in snake_case the name should start with `task_`.
-2. Update the `description` field to a short description of the task.
-3. Add a keyword to the `keywords` field that describes the task.
-4. Update the `task_template` in the links fields to the name of the task from step 1.
-5. Update the info fields to the text from the task issue.
-6. Replace the task_template to the name of the task in `info.name`.
-7. Update the authors of the task.
-
-After performing the steps above you can remove the comments in the `_viash.yaml` file.
+Work through the `# Step N` comments in the `_viash.yaml` file: the task name, keywords, links, label, summary, description, thumbnail, test resources and authors. After performing the steps you can remove the comments.
 
 ### `common` submodule
 
-Check the instructions [here](README.md) for more information.
+If the `common` folder is empty, initialise the submodule:
 
-## Resources
+```bash
+git submodule update --init --recursive
+```
 
-The OpenProblems team has provided some test resources that can be used to test the task. These resources are stored in the `resources_test` folder. The `scripts/download_resources.sh` script can be used to download these resources.
-If these resources are not sufficient, you can add more resources to the `resources_test` folder. The `scripts/download_resources.sh` script can be updated to download these resources. When using new test resources let the OP team know so they can be added to the s3 bucket.
+Check the [common_resources README](README.md) for more information.
 
-```bash	
-scripts/download_resources.sh
-```
+## Test resources
 
-## Next steps
+The OpenProblems team has provided test resources that can be used to develop the task. Download them into the `resources_test` folder with:
 
-### API files ([docs](https://openproblems.bio/documentation/create_task/design_api))
+```bash
+scripts/sync_resources.sh
+```
 
-Update the API files in the `src/api` folder. These files define the input and output of the methods and metrics. 
+Which resources are synced is defined in the `info.test_resources` section of `_viash.yaml`. When your task requires new test resources, let the OP team know so they can be added to the s3 bucket.
 
-### Components ([docs](https://openproblems.bio/documentation/create_task/create_components))
+## Next steps
 
-To create a component, you can run the respective script in the `script` directory. Before running the script make sure to update the variables `task_name`, `component_name` and `component_lang` and save the file. For additional components you will only need to update the `component_name` and `component_lang` variables.
+### API files ([docs](https://openproblems.bio/documentation/create_task/design-api/))
 
-```bash
-scripts/add_a_control_method.sh
-```
+Update the API files in the `src/api` folder. The `file_*.yaml` files define the file formats passed between components, the `comp_*.yaml` files define the input and output of each type of component.
 
-```bash
-scripts/add_a_method.sh
-```
+### Components ([docs](https://openproblems.bio/documentation/create_task/create-components/))
+
+For each type of component there already is a first component created that you can modify. To create additional components, use the scripts in `scripts/create_component`, or run the underlying script directly:
 
 ```bash
-scripts/add_a_metric.sh
+common/scripts/create_component --name my_method --language python --type method
 ```
 
-For each type of component there already is a first component created that you can modify.
+For each component:
 
-1. Update the `.info` fields in the  `config.vsh.yaml`.
+1. Update the metadata and `.info` fields in the `config.vsh.yaml`.
 2. Add any component specific arguments to the `config.vsh.yaml` file.
 3. Add any additional resources that are required for the component.
-4. Update the docker engine image setup if additional packages are required.
-5. If you know the required memory and or CPU you can adjust the nextflow `.directive.labels` field. In addition if your component requires a GPU you can add the `gpu` label to the field.
+4. Update the docker engine setup if additional packages are required.
+5. If you know the required memory and or CPU you can adjust the nextflow `.directives.label` field. In addition if your component requires a GPU you can add the `gpu` label to the field.
 6. Update the `script.py` or `script.R` file with the code for the component.
 
 > [!NOTE]
 > You can remove the comments in the `config.vsh.yaml` file after you have updated the file.
 
-### Testing Components ([docs](https://openproblems.bio/documentation/create_component/run_tests))
+### Testing components ([docs](https://openproblems.bio/documentation/create_component/run-tests/))
 
-You can test the component by running the following command:
+You can test a single component by running the following command:
 
 ```bash
 viash test /path/to/config.vsh.yaml
 ```
 
-Y0u can also test all components by running the following command:
+You can also test all components by running the following command:
 
 ```bash
-scripts/test_all_components.sh
+scripts/project/test_all_components.sh
 ```
 
-It is possible to customise the command in the above script by adding a `-q` argument to only perform the test on for example methods e.g. ` -q methods`.
-
-
-## Dataset processor ([docs](https://openproblems.bio/documentation/create_task/dataset_processor))
+It is possible to customise the command in the above script by adding a `-q` argument to only perform the test on for example methods, e.g. `-q methods`.
 
-The dataset processor is a script that removes all unnecessary info from the dataset for your task. This info is defined in the `api/file_common_dataset.yaml` file. From this filtered dataset several files are created that are used by the methods and metrics. Safeguarding data leaks and laking sure the structure of the data cannot be altered for a method or a metric.
+### Dataset processor ([docs](https://openproblems.bio/documentation/create_task/dataset-processor/))
 
-To create the data processor there is no template available. You can follow the guideline in the documentation. Store the processor in the `src/process_dataset` folder.
+The dataset processor removes all information from a common dataset that the methods should not be able to see. From this filtered dataset several files are created that are used by the methods and metrics. This safeguards against data leakage, as a method cannot read data it is never handed.
 
-Be sure to update the `file_common_dataset.yaml` file with the correct information required for the methods/metrics.
+The template contains an example processor in `src/data_processors/process_dataset` that you can modify. Be sure to update the `src/api/file_common_dataset.yaml` file with the fields required for the methods and metrics.
 
 > [!IMPORTANT]
 > When using your own datasets please advise the OpenProblems team on how to add these datasets to the s3 bucket.
 > As the dataset processor should make use of the `common` datasets folder in the `resources` or `resources_test` directory.
 
-To create the resources and test_resources for the task we will create a nextflow workflow that will process the datasets. This workflow will be created together with the OpenProblems team.
+### README
 
-## README
-
-To create the task `README` file preform following command:
+To create the task `README` file, run:
 
 ```bash
 scripts/create_readme.sh
 ```
 
-## Benchmarking ([docs](https://openproblems.bio/documentation/create_task/create_workflow))
+### Benchmarking ([docs](https://openproblems.bio/documentation/create_task/create-workflow/))
+
+When you are finished with creating your components and dataset processor, update the `run_benchmark` workflow in `src/workflows` and the scripts in `scripts/run_benchmark` to benchmark the components. This workflow will be created together with the OpenProblems team.
+
+You can test the benchmark locally on the test resources:
 
-When you are finished with creating your components and dataset processor you can create a workflow to benchmark the components. This workflow will be created together with the OpenProblems team.
+```bash
+viash ns build --parallel --setup cachedbuild
+scripts/run_benchmark/run_test_local.sh
+```