Skip to content

Commit 56cdc59

Browse files
EiffLLTMeyer
EiffL
and
LTMeyer
authored
Expand docs with architecture overview (#22)
* Add architecture overview docs * fixing format * adding content for doc * adding stuff to build the documentation * Adding content * minor edits * Format tests with ruff and add typing * Disable gradients on codec manager encode/decode * Add codec manager specific errors * Recast decoded image into the correct type * Add LegacySurveyImage to codec config * Update documentation test and deploy CI * Use uv run for sphinx invocation * Fix documentation links * Remove python tag of non-python code blocks * Fix mermaid code block * Add sphinx-mermaid to the dependencies * Fix sphinxcontrib-mermaid dependency * Fix missing cross-references * Update deploy workflow trigger to be only on main * Revert LegacySurveyImage from codec config --------- Co-authored-by: Lucas Meyer <lucas.thibaut.meyer@gmail.com>
1 parent 4fd3a8b commit 56cdc59

18 files changed

Lines changed: 3289 additions & 50 deletions

‎.github/workflows/deploy-doc.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Build Documentation
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
# Allows you to run this workflow manually from the Actions tab
8+
workflow_dispatch:
9+
10+
# Allow one concurrent deployment
11+
concurrency:
12+
group: "pages"
13+
cancel-in-progress: true
14+
15+
jobs:
16+
build:
17+
runs-on: ubuntu-latest
18+
19+
steps:
20+
- uses: actions/checkout@v4
21+
- uses: astral-sh/setup-uv@v5
22+
with:
23+
enable-cache: true
24+
- name: Set up Python
25+
uses: actions/setup-python@v5
26+
with:
27+
python-version: '3.11'
28+
- name: Install dependencies
29+
run: |
30+
uv sync --all-extras --dev
31+
- name: Build HTML documentation
32+
run: |
33+
cd docs
34+
uv run sphinx-build -b html . _build/html
35+
36+
- name: Upload artifact
37+
uses: actions/upload-pages-artifact@v3
38+
with:
39+
path: docs/_build/html
40+
41+
# Deploy job - only runs on main branch
42+
deploy:
43+
needs: build
44+
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
45+
permissions:
46+
contents: read
47+
pages: write
48+
id-token: write
49+
environment:
50+
name: github-pages
51+
url: ${{ steps.deployment.outputs.page_url }}
52+
runs-on: ubuntu-latest
53+
54+
steps:
55+
- name: Deploy to GitHub Pages
56+
id: deployment
57+
uses: actions/deploy-pages@v4

‎.github/workflows/test-doc.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Check Documentation Build
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- 'docs/**'
7+
- 'aion/**'
8+
- '.github/workflows/docs-check.yml'
9+
- 'pyproject.toml'
10+
11+
jobs:
12+
docs:
13+
runs-on: ubuntu-latest
14+
15+
steps:
16+
- uses: actions/checkout@v4
17+
- uses: astral-sh/setup-uv@v5
18+
with:
19+
enable-cache: true
20+
- name: Set up Python
21+
uses: actions/setup-python@v5
22+
with:
23+
python-version: '3.11'
24+
- name: Install dependencies
25+
run: |
26+
uv sync --all-extras --dev
27+
- name: Build documentation
28+
run: |
29+
cd docs
30+
uv run sphinx-build -W -b html . _build/html
31+
- name: Check for broken links
32+
run: |
33+
cd docs
34+
uv run sphinx-build -b linkcheck . _build/linkcheck || true

‎.github/workflows/test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ on:
44
pull_request:
55
push:
66
branches:
7-
- master
7+
- main
88

99
jobs:
1010
pre-commit:

‎.readthedocs.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
version: 2
2+
sphinx:
3+
configuration: docs/conf.py
4+
fail_on_warning: false
5+
python:
6+
version: 3.10
7+
install:
8+
- requirements: docs/requirements.txt

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,8 @@ AION-1’s tokenizers cover **39 distinct data types**, grouped by survey and da
8686
Start with our interactive tutorial:
8787
- **[Open in Google Colab](https://colab.research.google.com/github/PolymathicAI/AION/blob/main/notebooks/Tutorial.ipynb)** - Learn AION basics interactively, no local setup required!
8888

89+
For detailed guides, see the [online documentation](https://polymathic-ai.github.io/AION/).
90+
8991
## 📦 Advanced Installation
9092

9193
AION offers flexible installation options to suit your environment and requirements.

‎aion/codecs/config.py‎

Lines changed: 37 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,54 @@
11
# Configuration for codecs
22

3+
from aion.codecs.catalog import CatalogCodec
4+
from aion.codecs.image import ImageCodec
5+
from aion.codecs.scalar import (
6+
GridScalarCodec,
7+
LogScalarCodec,
8+
MultiScalarCodec,
9+
ScalarCodec,
10+
)
11+
from aion.codecs.scalar_field import ScalarFieldCodec
12+
from aion.codecs.spectrum import SpectrumCodec
313
from aion.modalities import (
14+
HSCAG,
15+
HSCAI,
16+
HSCAR,
17+
HSCAY,
18+
HSCAZ,
19+
Dec,
20+
GaiaFluxBp,
21+
GaiaFluxG,
22+
GaiaFluxRp,
23+
GaiaParallax,
24+
GaiaXpBp,
25+
GaiaXpRp,
26+
HSCMagG,
27+
HSCMagI,
28+
HSCMagR,
29+
HSCMagY,
30+
HSCMagZ,
31+
HSCShape11,
32+
HSCShape12,
33+
HSCShape22,
434
Image,
5-
Spectrum,
635
LegacySurveyCatalog,
7-
LegacySurveySegmentationMap,
36+
LegacySurveyEBV,
837
LegacySurveyFluxG,
9-
LegacySurveyFluxR,
1038
LegacySurveyFluxI,
11-
LegacySurveyFluxZ,
39+
LegacySurveyFluxR,
1240
LegacySurveyFluxW1,
1341
LegacySurveyFluxW2,
1442
LegacySurveyFluxW3,
1543
LegacySurveyFluxW4,
16-
LegacySurveyShapeR,
44+
LegacySurveyFluxZ,
45+
LegacySurveySegmentationMap,
1746
LegacySurveyShapeE1,
1847
LegacySurveyShapeE2,
19-
LegacySurveyEBV,
20-
Z,
21-
HSCAG,
22-
HSCAR,
23-
HSCAI,
24-
HSCAZ,
25-
HSCAY,
26-
HSCMagG,
27-
HSCMagR,
28-
HSCMagI,
29-
HSCMagZ,
30-
HSCMagY,
31-
HSCShape11,
32-
HSCShape22,
33-
HSCShape12,
34-
GaiaFluxG,
35-
GaiaFluxBp,
36-
GaiaFluxRp,
37-
GaiaParallax,
48+
LegacySurveyShapeR,
3849
Ra,
39-
Dec,
40-
GaiaXpBp,
41-
GaiaXpRp,
42-
)
43-
from aion.codecs.image import ImageCodec
44-
from aion.codecs.spectrum import SpectrumCodec
45-
from aion.codecs.catalog import CatalogCodec
46-
from aion.codecs.scalar_field import ScalarFieldCodec
47-
from aion.codecs.scalar import (
48-
ScalarCodec,
49-
LogScalarCodec,
50-
MultiScalarCodec,
51-
GridScalarCodec,
50+
Spectrum,
51+
Z,
5252
)
5353

5454
CODEC_CONFIG = {

‎aion/codecs/manager.py‎

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,21 @@
33
Handles dynamic loading and management of codecs for different modalities.
44
"""
55

6-
from typing import Dict, Union, Optional, Type
6+
from typing import Dict, Optional, Type, Union
7+
78
import torch
89

9-
from aion.modalities import Modality
1010
from aion.codecs.base import Codec
1111
from aion.codecs.config import CODEC_CONFIG
12+
from aion.modalities import Modality
13+
14+
15+
class ModalityTypeError(TypeError):
16+
"""Error raised when a modality type is not supported."""
17+
18+
19+
class TokenKeyError(ValueError):
20+
"""Error raised when a token key is not found in the tokens dictionary."""
1221

1322

1423
class CodecManager:
@@ -53,7 +62,7 @@ def _load_codec(self, modality_type: Type[Modality]) -> Codec:
5362
):
5463
config = CODEC_CONFIG[modality_type.__base__]
5564
else:
56-
raise ValueError(
65+
raise ModalityTypeError(
5766
f"No codec configuration found for modality type: {modality_type.__name__}"
5867
)
5968
else:
@@ -76,6 +85,7 @@ def _load_codec(self, modality_type: Type[Modality]) -> Codec:
7685

7786
return codec
7887

88+
@torch.no_grad()
7989
def encode(self, *modalities: Modality) -> Dict[str, torch.Tensor]:
8090
"""Encode multiple modalities.
8191
@@ -98,14 +108,15 @@ def encode(self, *modalities: Modality) -> Dict[str, torch.Tensor]:
98108
if hasattr(modality, "token_key"):
99109
token_key = modality.token_key
100110
else:
101-
raise ValueError(
111+
raise ModalityTypeError(
102112
f"Modality {type(modality).__name__} does not have a token_key attribute"
103113
)
104114

105115
tokens[token_key] = tokenized
106116

107117
return tokens
108118

119+
@torch.no_grad()
109120
def decode(
110121
self,
111122
tokens: Dict[str, torch.Tensor],
@@ -124,14 +135,14 @@ def decode(
124135
Decoded modality instance
125136
"""
126137
if not hasattr(modality_type, "token_key"):
127-
raise ValueError(
128-
f"Modality type {modality_type.__name__} does not have a token_key attribute"
138+
raise ModalityTypeError(
139+
f"Modality type {modality_type} does not have a token_key attribute"
129140
)
130141

131142
token_key = modality_type.token_key
132143
if token_key not in tokens:
133-
raise ValueError(
134-
f"Token key '{token_key}' for modality {modality_type.__name__} not found in tokens dictionary"
144+
raise TokenKeyError(
145+
f"Token key '{token_key}' for modality {modality_type} not found in tokens dictionary"
135146
)
136147

137148
# Get the appropriate codec
@@ -140,7 +151,7 @@ def decode(
140151
# Decode using the codec with any provided metadata
141152
decoded_modality = codec.decode(tokens[token_key], **metadata)
142153

143-
# Casting the decoded modality to be the specific modality type requested
154+
# Cast decoded modality to the correct type
144155
decoded_modality = modality_type(**decoded_modality.model_dump())
145156

146157
return decoded_modality

‎docs/Makefile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
SPHINXBUILD := sphinx-build
2+
SOURCEDIR := .
3+
BUILDDIR := _build
4+
5+
.PHONY: html
6+
html:
7+
$(SPHINXBUILD) -M html $(SOURCEDIR) $(BUILDDIR)

‎docs/_static/polymathic_logo.png‎

22.5 KB
Loading

0 commit comments

Comments
 (0)