[GR-73870] Add AGENTS.md

AGENTS.md

@@ -0,0 +1,80 @@
+# PROJECT KNOWLEDGE BASE
+
+## OVERVIEW
+GraalPy (GraalVM Python) implementation: Java (Truffle) + C (CPython C-API compatibility) + Python stdlib/overrides, built and tested via the `mx` build tool.
+
+## STRUCTURE
+```text
+./
+├── graalpython/                 # Core sources + stdlib + tests (multi-language)
+│   ├── com.oracle.graal.python/ # Main Java implementation (Truffle AST, runtime, builtins)
+│   ├── com.oracle.graal.python.cext/ # C-API (headers + C sources + adapted CPython modules)
+│   ├── com.oracle.graal.python.test/ # Python-level + Java-level tests and runners
+│   ├── lib-graalpython/         # GraalPy-specific Python modules/patches
+│   └── lib-python/              # Vendored/adapted CPython stdlib + CPython tests
+├── mx.graalpython/              # `mx` suite + helper commands (build/test/bench/bisect)
+├── scripts/                     # Dev utilities (launcher, formatting hooks, codegen)
+├── .github/workflows/           # GitHub Actions entrypoints
+├── ci/                          # CI definitions (jsonnet/libsonnet)
+├── docs/                        # User + contributor docs
+├── benchmarks/                  # Benchmark harnesses
+└── mxbuild/                     # GENERATED build artifacts (ignore for code navigation)
+```
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Build / run | `docs/contributor/CONTRIBUTING.md`, `mx.graalpython/` | This repo is `mx`-first (not a typical Maven/Gradle-only build). |
+| Java runtime & Truffle nodes | `graalpython/com.oracle.graal.python/src/com/oracle/graal/python/{runtime,nodes,builtins}` | Main interpreter implementation. |
+| C-API / native extensions | `graalpython/com.oracle.graal.python.cext/{include,src,modules}` | Mirrors CPython naming; many files are adapted from CPython. |
+| Python stdlib overrides | `graalpython/lib-graalpython/` | GraalPy-specific modules executed at startup and/or used by builtins. |
+| Vendored stdlib + CPython tests | `graalpython/lib-python/3/` | Large; treat as upstream-ish unless you are explicitly changing stdlib/tests. |
+| Python-level tests | `graalpython/com.oracle.graal.python.test/src/tests/` | Includes tagged tests + C-API tests. Runner: `.../src/runner.py`. |
+| CI pipelines | `.github/workflows/`, `ci.jsonnet`, `ci/` | Workflows typically drive `mx` gates/tags. |
+| Launchers / helper scripts | `scripts/python.sh`, `scripts/*` | `python.sh` is the local launcher wrapper. |
+
+## CONVENTIONS (DEVIATIONS)
+- `mx` is the primary build/test entrypoint; suite definition lives in `mx.graalpython/suite.py`.
+- `black` is configured but intentionally **disabled** for this repo (root `pyproject.toml`); line length is 120 and version locked to `23` for consistency.
+- Formatting/linting is enforced via **pre-commit** with repo-specific hooks (Eclipse formatter, checkstyle, copyright), and pylint only on `mx.graalpython/*.py`.
+- Large generated/build outputs exist in-tree (`mxbuild/`, `*.dist/`); do not use them as the source of truth when navigating code.
+
+## ANTI-PATTERNS (THIS PROJECT)
+- **Do not** base edits or reviews on `mxbuild/**` or `*.dist/**` outputs; change sources under `graalpython/**`, `mx.graalpython/**`, etc.
+- Security reports: follow `SECURITY.md` (do **NOT** file public issues for vulnerabilities).
+- C-API: heed CPython-style invariants in headers (e.g. `ceval.h`: **NEVER** nest `Py_BEGIN_ALLOW_THREADS` blocks; avoid mixing PyMem/PyObject allocators with `malloc`).
+- Interop: foreign `executable` / `instantiable` objects are **never** called with keyword arguments (see `docs/user/Interoperability.md`).
+
+## UNIQUE STYLES / GOTCHAS
+- Builtins: implemented in Java classes annotated with `@CoreFunctions`; individual operations are Nodes annotated with `@Builtin` (see contributing guide).
+- Many operations have shared equivalents under `com.oracle.graal.python.lib`; prefer using/adding shared lib nodes instead of duplicating patterns.
+- Parser work may require regenerating golden files (see `docs/contributor/CONTRIBUTING.md`).
+
+## COMMANDS
+
+* Import dependent suites / download deps
+  `mx sforceimport`
+* Build and run
+  `mx python-jvm`
+  `mx python -c 'print(42)'`
+* Common local testing
+    * Run cpyext tests
+      `mx graalpytest cpyext`
+    * Rerun a specific failing test
+      `mx graalpytest TEST-SELECTOR`
+    * Run JUnit tests
+      `mx python-gate --tags python-junit`
+* Style / formatting
+  `mx python-style --fix`
+  `mx python-gate --tags style`
+
+## NOTES
+- When searching for implementation, prefer `graalpython/com.oracle.graal.python/src/...` over vendored `lib-python` unless you are intentionally modifying upstream stdlib/tests.
+- If you see very large files under `com.oracle.graal.python.cext/modules/_sqlite/` or `expat/`, treat them as upstream imports/adaptations (patch carefully).
+
+## PULL REQUESTS AND JIRA TICKETS
+
+We use Jira and Bitbucket, and each PR should reference a Jira ticket with the form [GR-XXXX] where XXXX is the ticket number.
+When asked to open pull requests, agents should ask for the Jira ticket number.
+When asked to create a ticket, the `ol-cli jira` tool can be used to create a ticket for the "Python" component.
+When asked to create, run gates on, or check on the builds previously run on a pull request, use the `ol-cli bitbucket` tool.

ci/AGENTS.md

@@ -0,0 +1,18 @@
+# ci/ — CI DEFINITIONS
+
+## OVERVIEW
+Jsonnet/libsonnet CI definitions consumed by GitHub Actions matrix generation.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Main pipelines | `../ci.jsonnet`, `python-gate.libsonnet` | Defines gates/tags executed in CI. |
+| Shared constants | `constants.libsonnet` | Shared settings used across CI definitions. |
+| Helpers | `utils.libsonnet` | Common functions/macros. |
+
+## CONVENTIONS
+- GitHub workflows typically call `ci-matrix-gen.yml`, which evaluates `ci.jsonnet` and these libraries.
+
+## ANTI-PATTERNS
+- Don’t encode secrets or environment-specific paths here.
+- Keep changes compatible with matrix generation (small diffs, deterministic output).

docs/AGENTS.md

@@ -0,0 +1,19 @@
+# docs/ — DOCUMENTATION
+
+## OVERVIEW
+Contributor and user documentation; many repo-wide rules live here.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Build/test instructions | `contributor/CONTRIBUTING.md` | Primary “how to build/run gates” guide. |
+| Runtime internals | `contributor/IMPLEMENTATION_DETAILS.md` | Interpreter design details. |
+| Interop rules | `user/Interoperability.md` | Keyword-arg restrictions for foreign calls, etc. |
+| Native extensions | `user/Native-Extensions.md` | Guidance on native packages and tooling. |
+| Standalone apps | `user/Python-Standalone-Applications.md` | Notes on packaging and limitations. |
+
+## CONVENTIONS
+- Prefer updating docs here rather than duplicating instructions in random READMEs.
+
+## ANTI-PATTERNS
+- Security: follow `SECURITY.md` at repo root; do not instruct users to file public vulnerability issues.

graalpython/AGENTS.md

@@ -0,0 +1,24 @@
+# graalpython/ — SOURCE + STDLIB + TESTS
+
+## OVERVIEW
+Main implementation tree: Java (Truffle interpreter), C (CPython C-API compatibility), and Python (stdlib overlays + tooling/tests).
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Java interpreter | `com.oracle.graal.python/src/com/oracle/graal/python/{runtime,nodes,builtins}` | Most core behavior lives here. |
+| Shared Java lib nodes | `com.oracle.graal.python/src/com/oracle/graal/python/lib` | Prefer adding/reusing lib nodes for common operations. |
+| C-API headers + runtime | `com.oracle.graal.python.cext/include`, `.../src` | CPython-like naming; follow CPython invariants. |
+| Adapted native modules | `com.oracle.graal.python.cext/modules/` | Many files are upstream-derived; patch carefully. |
+| GraalPy stdlib overlays | `lib-graalpython/` | Python files executed at startup / for builtins. |
+| Vendored CPython stdlib/tests | `lib-python/3/` | Treat as upstream-ish unless explicitly changing it. |
+| GraalPy tests | `com.oracle.graal.python.test/src/tests/` | Python-level tests + tag files. |
+| Parser components | `com.oracle.graal.python.pegparser*` | Parser implementation + golden files tests. |
+
+## CONVENTIONS / GOTCHAS
+- This subtree contains both “source of truth” code and vendored/upstream-ish imports; keep patches minimal in `lib-python/` and large C module imports.
+- Some large headers/databases (unicode tables) are generated; avoid editing them by hand unless you also update the generator pipeline.
+
+## ANTI-PATTERNS
+- Don’t use `mxbuild/**` outputs to understand behavior; always navigate `.../src/...` trees.
+- C-API: never mix `PyMem_*` / `PyObject_*` allocators with platform `malloc` family.

graalpython/com.oracle.graal.python.cext/AGENTS.md

@@ -0,0 +1,20 @@
+# com.oracle.graal.python.cext/ — CPYTHON C-API COMPAT
+
+## OVERVIEW
+C headers and native code implementing CPython C-API compatibility plus adapted builtin extension modules.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Public C-API headers | `include/` | Mirrors CPython header structure. |
+| Core C-API impl | `src/` | CPython-like file naming (`unicodeobject.c`, `typeobject.c`, ...). |
+| Adapted extension modules | `modules/` | Large upstream-derived modules (e.g., `_sqlite`). |
+| Embedded/third-party imports | `expat/`, `modules/_sqlite/sqlite/` | Treat as upstream; patch minimally. |
+
+## CONVENTIONS / GOTCHAS
+- Follow CPython invariants in headers (examples: do not nest `Py_BEGIN_ALLOW_THREADS`; “never mix allocators” warnings).
+- Large unicode databases/headers are often generated (e.g., `unicodename_db.h`, `unicodedata_db.h`): avoid hand edits.
+
+## ANTI-PATTERNS
+- **NEVER** nest `Py_BEGIN_ALLOW_THREADS` blocks (see `include/ceval.h`).
+- Never mix `PyMem_*` / `PyObject_*` allocators with platform `malloc` family.

graalpython/com.oracle.graal.python.test/AGENTS.md

@@ -0,0 +1,19 @@
+# com.oracle.graal.python.test/ — TEST SUITES
+
+## OVERVIEW
+Python-level tests, tag files, and harnesses used by `mx python-gate` and `mx graalpytest`.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Python tests | `src/tests/` | Main Python tests; includes C-API related tests. |
+| Tagged/unittest tiers | `src/tests/unittest_tags/` | Tag files select which stdlib tests should pass. |
+| Test runner | `src/runner.py` | Harness for executing Python tests. |
+| Test data | `testData/` | Golden files and fixtures. |
+
+## CONVENTIONS
+- Tests are typically run via `mx python-gate --tags ...` rather than invoking pytest directly.
+- Use `mx [-d] graalpytest TEST-SELECTOR` to rerun a failing test or selection.
+
+## ANTI-PATTERNS
+- Don’t delete failing tests; fix root causes or update tag selections when behavior intentionally changes.

graalpython/com.oracle.graal.python/AGENTS.md

@@ -0,0 +1,32 @@
+# com.oracle.graal.python/ — CORE JAVA IMPLEMENTATION
+
+## OVERVIEW
+Truffle-based Python runtime: AST nodes, builtins, compiler/bytecode support, interop, and shared runtime services.
+
+## STRUCTURE
+```text
+com.oracle.graal.python/
+└── src/com/oracle/graal/python/
+    ├── runtime/     # Context, POSIX emulation, state, interop runtime glue
+    ├── nodes/       # Truffle AST nodes + bytecode nodes
+    ├── builtins/    # Builtin modules/types; @CoreFunctions + @Builtin nodes
+    ├── lib/         # Shared lib nodes/utilities used across builtins/nodes
+    ├── compiler/    # Bytecode compiler + DSL tooling
+    └── util/        # Shared helpers/data structures
+```
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Add/fix builtin | `.../builtins/**` | Modules/classes via `@CoreFunctions`; ops via `@Builtin` Nodes. |
+| Cross-cutting ops | `.../lib/**` | Prefer adding/reusing lib nodes instead of duplicating patterns. |
+| Interop behavior | `.../nodes/interop`, `.../runtime/interop` | Foreign call rules + conversions. |
+| Bytecode execution | `.../nodes/bytecode/**` | Root nodes and bytecode interpreter pieces. |
+
+## CONVENTIONS
+- Code style enforced via pre-commit Eclipse formatter + checkstyle; don’t hand-format Java.
+- Keep naming/layout close to CPython where practical (helps cross-referencing).
+
+## ANTI-PATTERNS
+- Don’t edit generated sources under `mxbuild/**` or distribution outputs; edit `src/**`.
+- Avoid re-implementing common helpers in builtins; check `.../lib/**` first.

graalpython/lib-graalpython/AGENTS.md

@@ -0,0 +1,17 @@
+# lib-graalpython/ — GRAALPY STDLIB OVERLAYS
+
+## OVERVIEW
+Python modules/patches that implement or tweak stdlib behavior for GraalPy; some execute at startup.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Overlay modules | `modules/` | Python modules provided/overridden by GraalPy. |
+| Patches | `patches/` | Patch files applied to external packages on pip installation to fix compatibility issues. |
+
+## CONVENTIONS / GOTCHAS
+- If a file name matches a builtin module name, it can run in that module context during startup (see contributor docs).
+- Keep changes aligned with corresponding Java builtins (`com.oracle.graal.python.builtins`) where relevant.
+
+## ANTI-PATTERNS
+- Don’t fork upstream stdlib behavior unnecessarily; prefer minimal overlays.

graalpython/lib-python/AGENTS.md

@@ -0,0 +1,17 @@
+# lib-python/ — VENDORED CPYTHON STDLIB + TESTS
+
+## OVERVIEW
+Vendored/adapted CPython standard library and CPython test suite used for compatibility validation.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Stdlib sources | `3/` | CPython stdlib tree. |
+| CPython tests | `3/test/` | Large upstream test suite; GraalPy runs selected tests via tag files. |
+| Local lint configs | `3/test/.ruff.toml`, `3/test/**/mypy.ini` | Primarily for the vendored tests area. |
+
+## CONVENTIONS
+- Treat this tree as upstream-ish: changes should be minimal, well-justified, and ideally correspond to upstream fixes.
+
+## ANTI-PATTERNS
+- Don’t apply sweeping refactors or formatting changes here; it makes rebasing/upstream sync harder.

mx.graalpython/AGENTS.md

@@ -0,0 +1,21 @@
+# mx.graalpython/ — MX BUILD TOOLING
+
+## OVERVIEW
+`mx` suite definition and Python utilities used for building, testing, benchmarking, and CI orchestration.
+
+## WHERE TO LOOK
+| Task | Location | Notes |
+|------|----------|-------|
+| Suite definition | `suite.py` | Defines projects, imports, distributions, version metadata. |
+| mx commands | `mx_graalpython.py` | Implements custom `mx` commands (e.g., test runners). |
+| Import helpers | `mx_graalpython_import.py` | `mx sforceimport`-related logic. |
+| Bench tooling | `mx_graalpython_benchmark.py`, `mx_graalpython_python_benchmarks.py` | Performance harness integration. |
+| Bisect tooling | `mx_graalpython_bisect.py` | Benchmark/test bisect support. |
+
+## CONVENTIONS
+- Pylint is enforced (via pre-commit) only for `mx.graalpython/*.py`.
+- Treat `suite.py` as the authoritative dependency/version map; CI workflows typically call into these mx gates.
+
+## ANTI-PATTERNS
+- Don’t re-implement build/test flows in ad-hoc scripts when an `mx` command already exists.
+- Avoid adding heavyweight dependencies here; this code runs in many CI contexts.