Skip to content

README.md

Latest commit

 

History

History
179 lines (122 loc) · 7.57 KB

README.md

File metadata and controls

179 lines (122 loc) · 7.57 KB

MSRA

MSRA is a TOML-like language for describing browser automation and related runtime data.

What this repo contains

  • A VS Code extension for syntax highlighting, diagnostics, completion, hover, definitions, and semantic tokens.
  • A shared Node-based parser, analyzer, and CLI entrypoint used by the editor integration and terminal checks.

Documentation

MkDocs documentation lives under docs/ and splits the MSRA language into separate pages per table family, so [app], [app.warmup], [app.func.*], and related namespaces are documented independently.

CLI

The canonical installable CLI entrypoint is msra-lsp.

In this repository, the easiest way to run it is through the local Node wrapper:

node .\bin\msra.js check .\examples\example.msra
node .\bin\msra.js serve

If you install or link the package, you can also call the bare command:

msra-lsp check .\examples\example.msra
msra-lsp serve

If you prefer npm scripts:

npm run check
npm run serve

If you are already inside vscode-extension, call the root entrypoint with a relative path:

node ..\bin\msra.js check ..\examples\example.msra

CLI modes

  • msra-lsp check <file.msra> parses the file and prints diagnostics.
  • msra-lsp check <file.msra> --json prints diagnostics as JSON.
  • msra-lsp check - reads MSRA content from stdin.
  • msra-lsp serve starts the language server over stdio for editors and external clients.

Python Codegen

This repo also includes a Python generator that turns an .msra document into an async Python client package and a matching Sphinx documentation tree.

python -m msra_codegen generate .\examples\example\example.msra -o .\generated
python -m msra_codegen validate .\generated

Use --no-cleanup if you want to keep the intermediate merged.msra file and preserve any existing files in the target directory.

validate runs python syntax checks, black, isort, flake8, and mypy against the generated project tree.

The generator uses Jinja2 templates, so install its Python dependency first if your environment does not already have it:

python -m pip install -r .\msra_codegen\requirements.txt

The generator writes:

  • pyproject.toml
  • requirements.txt
  • requirements-dev.txt
  • Makefile
  • .gitignore
  • .github/workflows/source-sync.yml
  • .github/workflows/tests.yml
  • .github/workflows/publish.yml
  • .github/ISSUE_TEMPLATE/config.yml
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/ISSUE_TEMPLATE/documentation_issue.yml
  • .github/ISSUE_TEMPLATE/feature_request.yml
  • <package-name>/__init__.py
  • <package-name>/manager.py
  • <package-name>/abstraction/__init__.py
  • <package-name>/abstraction/regexes.py
  • <package-name>/endpoints/*.py
  • referenced extractors/*.js assets
  • tests/conftest.py
  • tests/api_test.py
  • tests/__snapshots__/
  • docs/requirements.txt
  • docs/source/conf.py
  • docs/source/index.rst
  • docs/source/api.rst
  • docs/source/<package-name>.rst
  • docs/source/_api/*.rst
  • merged.msra is written as a temporary intermediate file and is removed by default after generation.

To build the generated docs locally:

python -m pip install -r .\generated\docs\requirements.txt
python -m sphinx -b html .\generated\docs\source .\generated\docs\_build\html

The reusable Jinja2 templates live under msra_codegen/templates/, so the output shape can be adjusted without editing the generator logic itself.

Source/Main Sync

The repository ships the sync workflow template used by generated package repositories. The generator writes .github/workflows/source-sync.yml into the target project, so the manual trigger lives in the target repo's main branch and can be started from GitHub Actions there.

The generated source-sync workflow stays thin: it checks out this repository as the generator logic, reads the consumer repo's .msra source tree, generates the artifact into a staging tree, syncs the generated tree into main, validates the result, and pushes the commit so the target repo's publish.yml can run on push to main.

If a project defines [app.sync] in its MSRA source, the generated sync step can preserve repo-specific runtime artifacts and ignore generated noise. For the FixPrice layout that means tests/__snapshots__ stays in main, while **/__pycache__ and **/*.pyc are removed before commit.

If a project defines [app.issue_templates] in its MSRA source, the generator also writes GitHub issue forms into .github/ISSUE_TEMPLATE/. The source contract only supplies assignee; the three forms are hardcoded in the generator, and contact_links are built from package_owner/package_name plus app.social.discord and app.social.telegram when present.

Before validation, the generated workflow installs the target project's requirements-dev.txt so the validation step runs with the generated dependencies available.

The source and target branches come from the repo-specific sync config, so the generated workflow itself stays thin.

In the recommended repo layout, main is the default branch of the consumer repo, because that is where the manual source-sync workflow lives.

That workflow expects a repo secret named SOURCE_SYNC_TOKEN with read access to the logic repository and write access to the source/target repository.

For a concrete repo layout and the exact manual run flow, see docs/msra-repo-b-sync.md.

For the FixPrice project, the default package name is fixprice_api.

Publishing

This repository also ships a manual publish workflow in .github/workflows/publish.yml.

Run it from the repository default branch and choose one of these targets:

  • extension for the VS Code Marketplace package
  • lsp for the npm CLI and Node language server package
  • codegen for the PyPI generator package
  • all to publish all three packages in one run

The workflow reads the package versions from the package metadata already in the repository, so there is no separate release version input.

It expects these credentials/configs:

  • A npm Trusted Publisher configured for Open-Inflation/engine-reverse-ide, workflow publish.yml, and the npm environment
  • VSCE_PAT for the VS Code Marketplace publish step
  • A PyPI Trusted Publisher configured for Open-Inflation/engine-reverse-ide, workflow publish.yml, and the pypi environment

The npm and PyPI jobs use GitHub OIDC, so they do not need API token secrets.

Language overview

MSRA is TOML-like and supports object references such as <OBJECT> or <OBJECT.value>.

Available OBJECTS:

  • UNSTANDARD_HEADERS (.RESPONSE or .REQUEST -> {key: value})
  • CAPTURED_URLS (.RESPONSE or .REQUEST -> array)
  • COOKIES (for example COOKIES["key"].VALUE["data"], when the string can be represented as a dictionary). Available fields also include DOMAIN, PATH, EXPIRES, HTTP_ONLY, SECURE, SAME_SITE.
  • LOCAL_STORAGE
  • SESSION_STORAGE

Both browser-side methods are supplemented by the sniffer during warmup.

  • INPUT -> any data
  • VARIABLES (.any_key) -> any data
  • DOCUMENT (.PREFIXES or .REGEXES) -> any data, which allows access to other variables in TOML

Filters like CAPTURED_URLS(START="http", TYPE=STR, END="something") are also supported.

All OBJECTS are always available, but their contents depend on runtime context.