Memory optimization for "DD matrix" calculation on large-scale data

[Zhihong-Lin]

Hi, Thank you for developing this useful tool!

I encountered a memory overflow issue when calculating the "DD matrix" for a large imaging-based ST dataset (5 slices, ~1.8M cells). To reduce memory consumption, I would like to ask if it would be appropriate to perform the following workflow:

1. Run spatial domain detection individually per slice,
2. Compute the centroid of SPACE for each detected spatial domain (as described in the "Spatial domain annotation with spatial reference" section in SpaDo paper ),
3. Calculate the "DD matrix" only between these centroids to identify similar spatial domains across slices?

Would this approach align with SpaDo's methodological rationale, or does the original algorithm inherently require cross-slice information during spatial domain detection/DD matrix computation?

Thank you for your insights!

[BinDuan]

Thank you for your attention. The "DD matrix" is specifically designed for hierarchical clustering using Jensen-Shannon Divergence (JSD), which generally outperforms other distance metrics. However, we have also demonstrated that SpaDo is robust across various distance measures, including Euclidean.

Given the scale of your dataset (1.8M cells), I recommend clustering directly on the cell type distribution without computing the "DD matrix" to save computational resources, with only a slight trade-off in accuracy.

If you have any further questions or need assistance with implementation, please feel free to reach out — I’d be happy to help!