docs: complete developer documentation rewrite (#480)#601
Open
planetf1 wants to merge 78 commits intogenerative-computing:mainfrom
Open
docs: complete developer documentation rewrite (#480)#601planetf1 wants to merge 78 commits intogenerative-computing:mainfrom
planetf1 wants to merge 78 commits intogenerative-computing:mainfrom
Conversation
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Contributor
|
The PR description has been updated. Please fill out the template for your PR to be reviewed. |
- CONTRIBUTING.md: writing conventions, PR checklist, code block runability rule, Backend note callout type - .markdownlint.json: fix MD025 front_matter_title so body H1 is allowed alongside YAML frontmatter title - getting-started.md: full tutorial page — install, hello world, user variables, requirements, core concepts, troubleshooting - glossary.md: skeleton in place
Full how-to page covering instruct(), user variables, requirements, custom validation functions (req/check/simple_validate), sampling strategies + IVR loop, grounding context, images, ChatContext, and chat() vs instruct() comparison. Imports verified against source. One inline review note on icl_examples API pending verification.
Covers Ollama (default), OpenAI-compatible, LiteLLM, HuggingFace, and WatsonX backends. ModelOption constants table, system prompt pattern, direct backend construction. Backend note callouts on each provider. Imports verified against source.
Covers @Generative decorator, Literal type constraints, Pydantic structured output, pre/post-conditions (PreconditionException), composing generative pipelines, and chain-of-thought pattern. Imports verified against source.
Covers @tool decorator, MelleaTool.from_callable/from_langchain/ from_smolagents, ModelOption.TOOLS, uses_tool, tool_arg_validator, react() agentic loop with structured output, code_interpreter. Incorporates agent definition and ReACT context from old agents.mdx. Imports verified against source (react is async).
Covers grounding context, RAG with FAISS + generative filtering, @mify / MObject pattern (query/transform, ad-hoc mify, custom stringify, funcs_include), and RichDocument with PDF parsing and table extraction. Incorporates content from mobjects.mdx and generative-slots.mdx. Imports verified against CI examples.
Covers all RAG intrinsic operations: answerability, context relevance, hallucination detection, answer relevance rewriting, query rewriting, citations, and direct Intrinsic/GraniteCommonAdapter usage. Backend note callout on HF requirement. Imports verified against source. Note: adapters.mdx content (tool calling) already covered in tools-and-agents.md.
Covers RejectionSamplingStrategy (with SamplingResult inspection), validation feedback via ValidationResult.reason, SOFAISamplingStrategy dual-model escalation with s2_solver_mode table, BudgetForcingSamplingStrategy, and MajorityVotingStrategyForMath. Review notes on budget forcing and majority voting exports/parameters.
Covers async/sync method table, parallel generation with ModelOutputThunk, wait_for_all_mots, streaming with ModelOption.STREAM + astream(), and context warnings for concurrent ChatContext use. Imports verified.
Covers three abstraction levels (instruct/act/mfuncs), working with Message and Document, validation + sampling strategies via act(), structured output with format=, functional API (mfuncs.act/aact), and aact() async usage. Fixed stale numeric cross-references.
Covers GuardianCheck + GuardianRisk (full enum table), custom criteria, groundedness detection, use as instruct() requirement, and input gate pattern. Backend note on Guardian model independence. Verified against CI example docs/examples/safety/guardian.py.
Covers FastMCP server creation, @mcp.tool decorator, mcp dev UI, ModelOption in tools, multiple tools in one server. Imports verified against mcp_example.ipynb CI notebook.
Covers two independent OTEL trace scopes (application + backend), all configuration env vars, start_session() as context manager for trace lifecycle, console debugging, Jaeger/OTLP export, programmatic status checks, and metrics API (create_counter/create_histogram). Verified against mellea/telemetry/__init__.py and telemetry_example.py. Includes Gen-AI semantic convention attribute tables.
Covers SimpleContext vs ChatContext, ctx introspection helpers (last_output/last_turn), session.clone() for context branching, session.reset(), and extending MelleaSession with a ChatCheckingSession example. Absorbs content from core-concept/context-management.mdx. Verified against session.py and creating_a_new_type_of_session.py.
Conceptual page explaining what generative programs are, the deterministic/stochastic interleaving challenge, requirements as the core reliability mechanism, failure handling, uncertainty compounding, context management, and Mellea's position as execution layer (not orchestrator). Absorbs content from overview/generative-programming.mdx and overview/mellea-welcome.mdx.
Covers the three core data structures (CBlock, Component, ModelOutputThunk), six abstraction layers from MelleaSession down to direct backend.generate_from_context() with lazy thunks, composition via SimpleComponent, and template/prompt engineering (TemplateFormatter, TemplateRepresentation, Jinja2 template resolution, model-specific paths). Verified imports against session_deepdive step files. Absorbs content from prompt-engineering.mdx.
Signed-off-by: Paul S. Schweigert <paul@paulschweigert.com>
Contributor
Author
|
Notes for reviewers:
On Monday I plan to
|
planetf1
changed the title
…ache glossary entries
…tries Add three new content pages: - tutorials/02-mifying-legacy-code: five-step tutorial on @mify — query and transform existing Python objects with m.query() and m.transform(), stringify_func, fields_include, funcs_include, and ad-hoc mify(obj) - tutorials/03-using-generative-slots: five-step tutorial on @Generative — Literal/Pydantic returns, pipeline composition, ChatContext injection, m.reset(), and pre/postcondition validation patterns - evaluation-and-observability/evaluate-with-llm-as-a-judge: how-to covering default LLMaJ behavior, standalone m.validate(), GenerateLog capture, purple elephant effect with check(), simple_validate bypass, combined checks, and SamplingResult metadata Also: - Add all three pages to docs.json nav - Add GenerateLog, LLM-as-a-judge, and Purple elephant effect to glossary - Add first-use glossary cross-links and full example pointers in each page
New pages: - tutorials/04-making-agents-reliable (ReACT, requirements, GuardianCheck) - how-to/refactor-prompts-with-cli (m decompose workflow) - how-to/unit-test-generative-code (pytest markers, TestBasedEval) - integrations/vertex-ai (LiteLLMBackend, vertex_ai/ model strings) - advanced/custom-components (Component protocol, TemplateRepresentation) - evaluation-and-observability/opentelemetry-tracing (spans, OTLP, Jaeger) - examples/index + 4 example pages (data-extraction, legacy-code, rag, telemetry) - community/contributing-guide, building-extensions, code-of-conduct - troubleshooting/faq (10 Q&A) Fixes: - tutorials/01: broken Next steps links; model-config review note added - docs.json: handling-exceptions moved to Eval & Observability (was How-To) - docs.json nav: all new pages registered - glossary: ComponentParseError, GuardianRisk, GuardianCheck expanded - AGENTS.md: Section 10 "Writing Docs" added with key conventions
- Fix MD012 multiple-blank-lines in 20 files (trailing double blank lines) - Fix MD028 blank-line-inside-blockquote in smolagents.md - vertex-ai.md: replace "Hendrik please confirm" review note with verified LiteLLM docs — vertex_project/vertex_location keys are correct and override env vars at call time - inference-time-scaling.md: remove two "review needed" notes on BudgetForcingSamplingStrategy and MajorityVotingStrategyForMath; add source-verified parameter docs for both - inference-time-scaling.md: add sections for RepairTemplateStrategy, MultiTurnStrategy, and BaseSamplingStrategy (all in __all__ but previously undocumented)
…thonExecutionReq - Sampling strategy table: add RepairTemplateStrategy, MultiTurnStrategy, MBRDRougeLStrategy, BaseSamplingStrategy, correct MajorityVoting name to MajorityVotingStrategyForMath - Requirement entry: document PythonExecutionReq (code execution validator) with import path and key parameters
…ING checklist - Key patterns: swap MCP card for Tools and agents (@tool, MelleaTool, react()) - How-to guides: swap Handling exceptions for Use images and vision - Backends: add LiteLLM / Vertex AI card - CONTRIBUTING.md checklist: add item to review landing page cards when adding a major page
unit-test-generative-code.md: - Add single top-of-page callout directing Mellea contributors to contributing-guide#testing; remove all other contributor callouts - Rewrite session fixture using plain OllamaModelBackend (no gh_run) - Rewrite module markers section as generic user guidance with pyproject.toml snippet - Rewrite CI strategy section with a user-owned conftest.py pattern (CI=true) instead of Mellea's internal CICD=1 convention traced-generation-loop.md: - Replace dead internal reference docs/dev/telemetry.md (deleted file) with link to user-facing OpenTelemetry Tracing page mellea-core-internals.md: - mfuncs async row: "Mellea contributors" → "Advanced users building async pipelines" template-formatting.md: - "contributors and advanced users" → "advanced users and library authors"
- Key patterns: remove 'Generative slots' (concept already in How it works section) replace 'Intrinsics and adapters' (too advanced/niche) with: - Async and streaming (use-async-and-streaming) - Safety checks (GuardianCheck via tutorial 04) - Backends: add LangChain as 8th card — makes even 4+4 grid
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #480
Complete rewrite of the Mellea developer documentation, published via Mintlify at https://docs.mellea.ai.
Preview branch: https://ibm-llm-runtime-aaf3a78b.mintlify.app/
Type of PR
What's included
Content — full Diataxis-structured developer docs
All code examples verified against the current codebase. No TODO comments or "review needed" notes remain.
Standards
markdownlintpasses clean (zero errors) — enforced in pre-commit and CI.mdextensions in internal linksdocs/CONTRIBUTING.md)CI
docs-lintjob inci.yml— runsmarkdownlint-clion every PRmarkdownlintpre-commit hook — runs on staged.mdfiles at commit timeCleanup
.mdxfiles removed (overview/,core-concept/,dev/) — all content superseded by the new structureThanks
Thanks to @HendrikStrobelt for the original source content that anchored the concepts and advanced sections.