Merge scanorama back into one component and fix its scrambled output

CHANGELOG.md

@@ -37,6 +37,9 @@
 * Un-pin the scPRINT version and update parameters (PR #51)
 * Update scPRINT to better handle large datasets, including a new default model (PR #54)
 * Credit contributors missing from the authors list, and fix Martin Kim's orcid (PR #100).
+* Merge `methods/scanorama_correct` and `methods/scanorama_integrate` back into a single `methods/scanorama`
+    component, reverting the split from PR #88. A single `correct_scanpy()` run returns both the corrected counts and
+    the embedding.
 
 ## Bug fixes
 
@@ -53,6 +56,9 @@
 * Fix `methods/pyliger` failing to build: louvain has no python 3.12 wheel and needs cmake to build igraph from source.
 * Bump `methods/cellplm`, `methods/condo`, `methods/drvi` and `metrics/bras` from base image `:1.0.0` to `:1`, so their
     `openproblems` is new enough for the component tests in `common`.
+* Fix `methods/scanorama` scrambling its output: scanorama returns one object per batch with the genes sorted by
+    name, so both axes ended up permuted with respect to the `obs` and `var` they were labelled with. This affected
+    every metric, not just `hvg_overlap`.
 * Fix `methods/geneformer` failing to build: pip's `--filter=blob:none` clone of the huggingface repo no longer works,
     so clone it ourselves. Also needs `transformers<5`, which still has `SpecialTokensMixin`.
 * Fix `methods/cellplm` failing to build: drop the pytorch base image's broken `/usr/local/bin/cmake` shim, which

src/methods/scanorama_integrate/config.vsh.yaml → src/methods/scanorama/config.vsh.yaml

@@ -1,13 +1,16 @@
 __merge__: /src/api/comp_method.yaml
-name: scanorama_integrate
-label: Scanorama-Integrate
+name: scanorama
+label: Scanorama
 summary: Efficient integration of heterogeneous single-cell transcriptomes using Scanorama
 description: |
   Scanorama enables batch-correction and integration of heterogeneous scRNA-seq datasets.
   It is designed to be used in scRNA-seq pipelines downstream of noise-reduction methods,
   including those for imputation and highly-variable gene filtering. The results from
   Scanorama integration and batch correction can then be used as input to other tools
   for scRNA-seq clustering, visualization, and analysis.
+
+  A single run returns both the batch-corrected expression matrix and the integrated
+  embedding it is assembled from.
 references:
   # Hie, B., Bryson, B. & Berger, B. Efficient integration of heterogeneous single-cell
   # transcriptomes using Scanorama. Nat Biotechnol 37, 685–691 (2019).
@@ -17,7 +20,7 @@ links:
   repository: https://github.com/brianhie/scanorama
   documentation: https://github.com/brianhie/scanorama#readme
 info:
-  method_types: [embedding]
+  method_types: [feature, embedding]
   preferred_normalization: log_cp10k
 arguments:
   - name: --dimred

src/methods/scanorama_integrate/script.py → src/methods/scanorama/script.py

@@ -1,7 +1,7 @@
 import sys
+
 import anndata as ad
 import scanorama
-import numpy as np
 
 ## VIASH START
 par = {
@@ -10,14 +10,14 @@
     'dimred': 100
 }
 meta = {
-    'name': 'scanorama-integrate',
+    'name': 'scanorama',
+    'resources_dir': 'src/utils'
 }
 ## VIASH END
 
 sys.path.append(meta["resources_dir"])
 from read_anndata_partial import read_anndata
 
-
 print('Read input', flush=True)
 adata = read_anndata(
     par['input'],
@@ -28,32 +28,30 @@
 )
 
 print('Run scanorama', flush=True)
-split = []
-batch_categories = adata.obs['batch'].cat.categories
-for b in batch_categories:
-    split.append(adata[adata.obs['batch'] == b].copy())
-scanorama.integrate_scanpy(split, dimred=par["dimred"])
-
-#From https://colab.research.google.com/drive/1CebA3Ow4jXITK0dW5el320KVTX_szhxG
-result = np.zeros((adata.shape[0], split[0].obsm["X_scanorama"].shape[1]))
-for i, b in enumerate(batch_categories):
-    result[adata.obs['batch'] == b] = split[i].obsm["X_scanorama"]
+split = [adata[adata.obs['batch'] == batch].copy() for batch in adata.obs['batch'].cat.categories]
+corrected = scanorama.correct_scanpy(split, return_dimred=True, dimred=par['dimred'])
 
+# scanorama returns one object per batch, with the genes sorted by name -- put the
+# cells and genes back in the order of the input before storing the output
+corrected = ad.concat(corrected)
+corrected = corrected[adata.obs_names, adata.var_names]
 
 print("Store output", flush=True)
 output = ad.AnnData(
     obs=adata.obs[[]],
     var=adata.var[[]],
+    layers={
+        'corrected_counts': corrected.X,
+    },
+    obsm={
+        'X_emb': corrected.obsm['X_scanorama'],
+    },
     uns={
         'dataset_id': adata.uns['dataset_id'],
         'normalization_id': adata.uns['normalization_id'],
         'method_id': meta['name'],
-    },
-    obsm={
-        'X_emb': result
-    },
-    shape=adata.shape,
+    }
 )
 
 print("Write output to file", flush=True)
-output.write(par['output'], compression='gzip')
+output.write_h5ad(par['output'], compression='gzip')

src/workflows/run_benchmark/config.vsh.yaml

@@ -104,8 +104,7 @@ dependencies:
   - name: methods/pyliger
   - name: methods/sca
   - name: methods/scalex
-  - name: methods/scanorama_correct
-  - name: methods/scanorama_integrate
+  - name: methods/scanorama
   - name: methods/scanvi
   - name: methods/scmerge2
   - name: methods/scgpt_finetuned

src/workflows/run_benchmark/main.nf

@@ -34,8 +34,7 @@ methods = [
   pyliger,
   sca,
   scalex,
-  scanorama_correct,
-  scanorama_integrate,
+  scanorama,
   scanvi,
   scmerge2,
   scgpt_finetuned.run(