diff --git a/CHANGELOG.md b/CHANGELOG.md index ff47ac9d4..8055aab17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,90 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [Unreleased] +## [0.3.0] - Jun 1, 2026 + +### Deprecations + +- Deprecate `ShardStrategy` and default distributed sharding to `CONTIGUOUS` by @kmontemayor2-sc in + https://github.com/Snapchat/GiGL/pull/582 and https://github.com/Snapchat/GiGL/pull/545 +- Migrate `RESOURCE_CONFIG_PATH` to `GIGL_RESOURCE_CONFIG_URI` by @kmontemayor2-sc in + https://github.com/Snapchat/GiGL/pull/648 + +### Changed + +- Replace mypy with [ty](https://github.com/astral-sh/ty) for static type checking and migrate formatting to Ruff by + @svij-sc in https://github.com/Snapchat/GiGL/pull/585 and https://github.com/Snapchat/GiGL/pull/583 +- Consolidate distributed loader and sampler abstractions, including `BaseDistLoader`, `BaseDistNeighborSampler`, + sampler factory helpers, two-phase loader initialization, and shared sampling options by @mkolodner-sc and + @kmontemayor2-sc in https://github.com/Snapchat/GiGL/pull/495, https://github.com/Snapchat/GiGL/pull/532, + https://github.com/Snapchat/GiGL/pull/536, https://github.com/Snapchat/GiGL/pull/570, + https://github.com/Snapchat/GiGL/pull/576, https://github.com/Snapchat/GiGL/pull/579, and + https://github.com/Snapchat/GiGL/pull/561 +- Merge `testing/` into `tests/`, migrate tests to GiGL test case utilities, and improve graph store integration test + coverage by @kmontemayor2-sc and @svij-sc in https://github.com/Snapchat/GiGL/pull/494, + https://github.com/Snapchat/GiGL/pull/479, https://github.com/Snapchat/GiGL/pull/480, + https://github.com/Snapchat/GiGL/pull/515, and https://github.com/Snapchat/GiGL/pull/547 + +### Added + +- Enable GraphStore mode across storage and compute, including GiGL-owned `DistServer`, `DistABLPLoader` GraphStore + mode, multiple GraphStore loaders, and homogeneous and heterogeneous examples by @kmontemayor2-sc in + https://github.com/Snapchat/GiGL/pull/476, https://github.com/Snapchat/GiGL/pull/485, + https://github.com/Snapchat/GiGL/pull/493, https://github.com/Snapchat/GiGL/pull/514, and + https://github.com/Snapchat/GiGL/pull/526 +- Add distributed and C++-based PPR sampling, including PPR sequence generation and new GiGL wheel builds with the + `gigl-core` C++/CUDA extension package, by @mkolodner-sc and @yliu2-sc in https://github.com/Snapchat/GiGL/pull/538, + https://github.com/Snapchat/GiGL/pull/560, https://github.com/Snapchat/GiGL/pull/558, and + https://github.com/Snapchat/GiGL/pull/556 +- Add shared multi-channel graph store sampling backend, remote channels with pinned-memory bulk transfer, and two-phase + sampling APIs by @kmontemayor2-sc in https://github.com/Snapchat/GiGL/pull/577, + https://github.com/Snapchat/GiGL/pull/565, and https://github.com/Snapchat/GiGL/pull/578 +- Add weighted sampling, positional encoding transforms, Graph Transformer encoder, degree tensor computation for + `DistDataset`, and max-label-per-anchor support in the data splitter by @mkolodner-sc and @yliu2-sc in + https://github.com/Snapchat/GiGL/pull/635, https://github.com/Snapchat/GiGL/pull/509, + https://github.com/Snapchat/GiGL/pull/537, https://github.com/Snapchat/GiGL/pull/517, and + https://github.com/Snapchat/GiGL/pull/589 +- Add `CustomResourceConfig` shell-command launchers, custom launcher subprocess dispatch, and GiGL env var propagation + for custom and Vertex AI launchers by @kmontemayor2-sc in https://github.com/Snapchat/GiGL/pull/625, + https://github.com/Snapchat/GiGL/pull/626, https://github.com/Snapchat/GiGL/pull/642, and + https://github.com/Snapchat/GiGL/pull/653 +- Add Vertex AI boot disk, reservation, and Data Preprocessor timeout controls by @zfan3-sc, @kmontemayor2-sc, and + @mkolodner-sc in https://github.com/Snapchat/GiGL/pull/521, https://github.com/Snapchat/GiGL/pull/590, and + https://github.com/Snapchat/GiGL/pull/524 +- Add GBML config wrapper maps for node and edge type metadata, unified metadata extraction, BigQuery latest-table + utility, and SNC example code by @svij-sc, @mkolodner-sc, and @kmontemayor2-sc in + https://github.com/Snapchat/GiGL/pull/643, https://github.com/Snapchat/GiGL/pull/544, + https://github.com/Snapchat/GiGL/pull/516, and https://github.com/Snapchat/GiGL/pull/641 + +### Fixed + +- Fix PPR sampler output edges and memory behavior by @mkolodner-sc in https://github.com/Snapchat/GiGL/pull/562, + https://github.com/Snapchat/GiGL/pull/566, and https://github.com/Snapchat/GiGL/pull/645 +- Fix dataloading of multiple labels, missing anchor-node labels, dataset factory parallel tensor loading, test dataset + edge direction, and `PreprocessedMetadataPbWrapper` `LocalUri` kwargs by @mkolodner-sc, @kmontemayor2-sc, and @svij-sc + in https://github.com/Snapchat/GiGL/pull/612, https://github.com/Snapchat/GiGL/pull/571, + https://github.com/Snapchat/GiGL/pull/606, https://github.com/Snapchat/GiGL/pull/552, and + https://github.com/Snapchat/GiGL/pull/639 +- Fix launcher image selection for graph store storage pools, default `should_use_glt_backend`, C++ installation, SBT + dependency resolution, and types-protobuf v7 `ParseDict` compatibility by @kmontemayor2-sc, @mkolodner-sc, and + @svij-sc in https://github.com/Snapchat/GiGL/pull/615, https://github.com/Snapchat/GiGL/pull/609, + https://github.com/Snapchat/GiGL/pull/619, https://github.com/Snapchat/GiGL/pull/631, and + https://github.com/Snapchat/GiGL/pull/638 +- Make metric exporters fall back to NoOp when initialization fails and fix seeding utility behavior by @mkolodner-sc + and @svij-sc in https://github.com/Snapchat/GiGL/pull/550 and https://github.com/Snapchat/GiGL/pull/644 + +### Misc + +- Add and refine agent/development guidance, including `CLAUDE.md`, `AGENTS.md`, `/all_test`, `/codex-review`, and + `/watch-action` workflow support by @kmontemayor2-sc in https://github.com/Snapchat/GiGL/pull/498, + https://github.com/Snapchat/GiGL/pull/504, https://github.com/Snapchat/GiGL/pull/543, + https://github.com/Snapchat/GiGL/pull/508, https://github.com/Snapchat/GiGL/pull/528, and + https://github.com/Snapchat/GiGL/pull/563 +- Update graph store and in-memory SGS documentation, supervision-edge direction docs, Vertex AI Agent Platform links, + and presubmit whitespace checks by @mkolodner-sc and @kmontemayor2-sc in https://github.com/Snapchat/GiGL/pull/553, + https://github.com/Snapchat/GiGL/pull/593, https://github.com/Snapchat/GiGL/pull/595, + https://github.com/Snapchat/GiGL/pull/608, and https://github.com/Snapchat/GiGL/pull/607 + ## [0.2.0] - Jan 30, 2025 ### Added diff --git a/gigl-core/pyproject.toml b/gigl-core/pyproject.toml index f7b159954..8eddd7635 100644 --- a/gigl-core/pyproject.toml +++ b/gigl-core/pyproject.toml @@ -2,7 +2,7 @@ name = "gigl-core" description = "GiGL C++/CUDA kernels (pybind11 extensions)" readme = "README.md" -version = "0.2.0" +version = "0.3.0" requires-python = "==3.11.*" # Torch is resolved from the ambient environment. gigl-core wheels are ABI-bound # to the torch variant they were built against (cpu or cu128). The parent `gigl` diff --git a/gigl/__init__.py b/gigl/__init__.py index d3ec452c3..493f7415d 100644 --- a/gigl/__init__.py +++ b/gigl/__init__.py @@ -1 +1 @@ -__version__ = "0.2.0" +__version__ = "0.3.0" diff --git a/gigl/dep_vars.env b/gigl/dep_vars.env index 6f6ee5584..8314a0cb5 100644 --- a/gigl/dep_vars.env +++ b/gigl/dep_vars.env @@ -3,11 +3,11 @@ DOCKER_LATEST_BASE_CUDA_IMAGE_NAME_WITH_TAG=us-central1-docker.pkg.dev/external- DOCKER_LATEST_BASE_CPU_IMAGE_NAME_WITH_TAG=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cpu-base:7d3182eeb6446ce3e35910babba990c8e003879d.109.1 DOCKER_LATEST_BASE_DATAFLOW_IMAGE_NAME_WITH_TAG=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dataflow-base:7d3182eeb6446ce3e35910babba990c8e003879d.109.1 -DEFAULT_GIGL_RELEASE_SRC_IMAGE_CUDA=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cuda:0.2.0 -DEFAULT_GIGL_RELEASE_SRC_IMAGE_CPU=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cpu:0.2.0 -DEFAULT_GIGL_RELEASE_SRC_IMAGE_DATAFLOW_CPU=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cpu-dataflow:0.2.0 -DEFAULT_GIGL_RELEASE_DEV_WORKBENCH_IMAGE=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dev-workbench:0.2.0 -DEFAULT_GIGL_RELEASE_KFP_PIPELINE_PATH=gs://public-gigl/releases/pipelines/gigl-pipeline-0.2.0.yaml +DEFAULT_GIGL_RELEASE_SRC_IMAGE_CUDA=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cuda:0.3.0 +DEFAULT_GIGL_RELEASE_SRC_IMAGE_CPU=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cpu:0.3.0 +DEFAULT_GIGL_RELEASE_SRC_IMAGE_DATAFLOW_CPU=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/src-cpu-dataflow:0.3.0 +DEFAULT_GIGL_RELEASE_DEV_WORKBENCH_IMAGE=us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dev-workbench:0.3.0 +DEFAULT_GIGL_RELEASE_KFP_PIPELINE_PATH=gs://public-gigl/releases/pipelines/gigl-pipeline-0.3.0.yaml SPARK_31_TFRECORD_JAR_GCS_PATH=gs://public-gigl/tools/scala/spark_packages/spark-custom-tfrecord_2.12-0.5.0.jar SPARK_35_TFRECORD_JAR_GCS_PATH=gs://public-gigl/tools/scala/spark_packages/spark_3.5.0-custom-tfrecord_2.12-0.6.1.jar diff --git a/pyproject.toml b/pyproject.toml index b22e1f0e6..9d4ba6089 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -2,7 +2,7 @@ name = "gigl" description = "GIgantic Graph Learning Library" readme = "README.md" -version = "0.2.0" +version = "0.3.0" classifiers = [ "Programming Language :: Python", "Programming Language :: Python :: 3", @@ -15,7 +15,7 @@ dependencies = [ "chardet", # gigl-core hosts all C++ / CUDA / pybind11 extensions. Separate wheel per torch # variant (cpu/cu128). Version must match gigl exactly. - "gigl-core==0.2.0", + "gigl-core==0.3.0", "google-cloud-aiplatform", "google-cloud-dataproc", "google-cloud-logging", diff --git a/scripts/bump_version.py b/scripts/bump_version.py index 499f99ed9..ba7c6d3bd 100644 --- a/scripts/bump_version.py +++ b/scripts/bump_version.py @@ -99,7 +99,9 @@ def update_pyproject(version: str) -> None: path = f"{GIGL_ROOT_DIR}/pyproject.toml" with open(path, "r") as f: content = f.read() - content = re.sub(r'(version\s*)=\s*"[\d\.]+"', f'\\1= "{version}"', content) + content = re.sub( + r'(?m)^(version[ \t]*)=[ \t]*"[^"]+"', f'\\1= "{version}"', content + ) # Keep the gigl-core pin in sync with the new version. content = re.sub(r'"gigl-core==[\d\.a-zA-Z]+"', f'"gigl-core=={version}"', content) with open(path, "w") as f: @@ -110,7 +112,9 @@ def update_gigl_core_pyproject(version: str) -> None: path = f"{GIGL_ROOT_DIR}/gigl-core/pyproject.toml" with open(path, "r") as f: content = f.read() - content = re.sub(r'(version\s*)=\s*"[\d\.]+"', f'\\1= "{version}"', content) + content = re.sub( + r'(?m)^(version[ \t]*)=[ \t]*"[^"]+"', f'\\1= "{version}"', content + ) with open(path, "w") as f: f.write(content) diff --git a/uv.lock b/uv.lock index 1909a0793..62d35c155 100644 --- a/uv.lock +++ b/uv.lock @@ -702,7 +702,7 @@ wheels = [ [[package]] name = "gigl" -version = "0.2.0" +version = "0.3.0" source = { editable = "." } dependencies = [ { name = "argo-workflows" }, @@ -965,7 +965,7 @@ typing-stubs = [ [[package]] name = "gigl-core" -version = "0.2.0" +version = "0.3.0" source = { directory = "gigl-core" } [[package]]