Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
d892f23
feat: replace fixed MetricDataPoint fields with dynamic tag HashMap
mattmkim Mar 18, 2026
4e928fe
feat: replace ParquetField enum with constants and dynamic validation
mattmkim Mar 18, 2026
d8d71ed
feat: derive sort order and bloom filters from batch schema
mattmkim Mar 18, 2026
ad115bf
feat: union schema accumulation and schema-agnostic ingest validation
mattmkim Mar 18, 2026
d0a995e
feat: dynamic column lookup in split writer
mattmkim Mar 18, 2026
35c3942
feat: remove ParquetSchema dependency from indexing actors
mattmkim Mar 18, 2026
33c4070
refactor: deduplicate test batch helpers
mattmkim Mar 18, 2026
7a5979f
lint
mattmkim Mar 30, 2026
440631b
Merge branch 'main' into matthew.kim/metrics-wide-schema
mattmkim Mar 31, 2026
5eae799
Merge branch 'main' into matthew.kim/metrics-wide-schema
mattmkim Mar 31, 2026
5b2304c
feat(31): sort schema foundation β€” proto, parser, display, validation…
g-talbot Mar 31, 2026
4d42fd9
fix: rustdoc link errors β€” use backticks for private items
g-talbot Apr 1, 2026
b6eb595
feat(31): compaction metadata types β€” extend split metadata, postgres…
g-talbot Mar 31, 2026
76b703a
feat(31): wire TableConfig into sort path, add compaction KV metadata
g-talbot Mar 9, 2026
ff605b9
feat(31): PostgreSQL migration 27 + compaction columns in stage/list/…
g-talbot Mar 9, 2026
723168f
fix(31): close port gaps β€” split_writer metadata, compaction scope, p…
g-talbot Mar 9, 2026
9ca263d
fix(31): final gap fixes β€” file-backed scope filter, META-07 test, de…
g-talbot Mar 10, 2026
73a20ef
fix(31): correct postgres types for window_duration_secs and zonemap_…
g-talbot Mar 10, 2026
75c15a0
style: rustfmt
g-talbot Mar 27, 2026
ef21859
test(31): add metrics split test suite to shared metastore_test_suite…
g-talbot Apr 1, 2026
b4dac46
fix(31): read compaction columns in list_metrics_splits, fix cleanup_…
g-talbot Apr 1, 2026
db51a96
fix(31): correct error types for non-existent metrics splits
g-talbot Apr 1, 2026
605708e
style: rustfmt metastore tests and postgres
g-talbot Apr 1, 2026
f21fff5
fix(31): address PR review β€” align metrics_splits with splits table
g-talbot Apr 6, 2026
f790519
style: fix rustfmt nightly formatting
g-talbot Apr 6, 2026
c55df1a
feat(31): add shared invariants module to quickwit-dst
g-talbot Mar 12, 2026
70793f6
feat(31): check invariants in release builds, add pluggable recorder
g-talbot Mar 12, 2026
1e1e328
feat(31): wire invariant recorder to DogStatsD metrics
g-talbot Mar 12, 2026
f8ee0cd
fix: license headers + cfg(not(test)) for quickwit-dst and quickwit-cli
g-talbot Apr 1, 2026
d40fd11
chore: regenerate third-party license file
g-talbot Apr 6, 2026
0e4be88
style: fix rustfmt nightly formatting for quickwit-dst and quickwit-p…
g-talbot Apr 6, 2026
33dd330
docs: add CLAUDE.md and docs/internals architecture documentation
g-talbot Mar 12, 2026
a7ec5fd
docs: split CLAUDE.md into repo context + opt-in /sesh-mode skill
hushy Mar 18, 2026
4981c26
docs: add machete, cargo doc, and fmt details to CI checklist in CLAU…
g-talbot Mar 31, 2026
c8bf8d7
review: parquet_file singular, proto doc link, fix metastore accessor
g-talbot Apr 1, 2026
cafcac5
style: fix rustfmt nightly comment wrapping in split metadata
g-talbot Apr 6, 2026
a088f53
fix: use plain code span for proto reference to avoid broken rustdoc …
g-talbot Apr 6, 2026
759c2ca
Update quickwit/quickwit-parquet-engine/src/table_config.rs
g-talbot Apr 7, 2026
6454f1d
Update quickwit/quickwit-parquet-engine/src/table_config.rs
g-talbot Apr 7, 2026
4d8b6b2
Merge quickwit-oss/main into gtt/phase-31-sort-schema
g-talbot Apr 8, 2026
1e67900
Merge branch 'gtt/phase-31-sort-schema' into gtt/phase-31-compaction-…
g-talbot Apr 8, 2026
85fcb2d
Merge branch 'gtt/phase-31-compaction-metadata' into gtt/phase-31-wri…
g-talbot Apr 8, 2026
06a6bf0
Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore
g-talbot Apr 8, 2026
9b95178
Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute
g-talbot Apr 8, 2026
397fc99
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
4481bef
style: rustfmt long match arm in default_sort_fields
g-talbot Apr 8, 2026
64c5d5f
Merge branch 'gtt/phase-31-sort-schema' into gtt/phase-31-compaction-…
g-talbot Apr 8, 2026
a8bf948
Merge branch 'gtt/phase-31-compaction-metadata' into gtt/phase-31-wri…
g-talbot Apr 8, 2026
caa9c3e
Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore
g-talbot Apr 8, 2026
42ab5c2
Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute
g-talbot Apr 8, 2026
35dd926
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
93e1cc7
fix: make parquet_file field backward-compatible in MetricsSplitMetadata
g-talbot Apr 8, 2026
b968085
Merge branch 'gtt/phase-31-compaction-metadata' into gtt/phase-31-wri…
g-talbot Apr 8, 2026
f7c89bf
fix: handle empty-column batches in accumulator flush
g-talbot Apr 8, 2026
1c99ddd
Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore
g-talbot Apr 8, 2026
bbfef12
Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute
g-talbot Apr 8, 2026
ca192e0
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
112f290
Merge quickwit-oss/main and address review comments
g-talbot Apr 8, 2026
e8c6be4
Merge branch 'gtt/phase-31-writer-wiring' into gtt/phase-31-pg-metastore
g-talbot Apr 8, 2026
71586e4
Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute
g-talbot Apr 8, 2026
19d67a8
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
f50f8f6
Merge quickwit-oss/main and address review feedback
g-talbot Apr 8, 2026
7dbd1ef
Merge branch 'gtt/phase-31-pg-metastore' into gtt/phase-31-execute
g-talbot Apr 8, 2026
6f1bf5b
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
4216827
Merge main, address review feedback, fix merge conflict
g-talbot Apr 8, 2026
4f118b3
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
501f014
style: rustfmt check_invariant macro argument
g-talbot Apr 8, 2026
cf37871
Merge branch 'gtt/phase-31-execute' into gtt/docs-claude-md
g-talbot Apr 8, 2026
03cf20f
Merge branch 'main' into gtt/docs-claude-md
g-talbot Apr 8, 2026
6e2bbe7
revert: move code changes to separate PR
g-talbot Apr 8, 2026
86e0881
remove ADR-004 from upstream PR β€” moving to DataDog/pomsky
g-talbot Apr 8, 2026
b901734
docs: rebrand for upstream β€” remove Datadog/Pomsky/Quickhouse references
g-talbot Apr 8, 2026
c0c127b
docs: remove ClickHouse references, track aspirational items
g-talbot Apr 8, 2026
e93bea7
docs: fix aspirational items β€” TLA+ specs and Stateright models exist
g-talbot Apr 9, 2026
a7e4f1f
chore: add .planning to .gitignore
g-talbot Apr 9, 2026
35fa272
revert: remove pomsky-specific Makefile changes
g-talbot Apr 9, 2026
4006b20
Merge branch 'main' into gtt/docs-claude-md
g-talbot Apr 9, 2026
b9bd5c8
docs: add pitfall rule against silently swallowing unexpected state
g-talbot Apr 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions .claude/skills/sesh-mode/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
description: "Verification-first development workflow with TLA+ specs, DST, and formal methods. Use when you want Claude to follow the rigorous plan→spec→test→implement sequence."
user-invocable: true
---

# Sesh Mode β€” Verification-First Development

Activate this mode when working on features that touch state machines, protocols, or critical data paths. This adds formal verification requirements on top of the base CLAUDE.md.

## Before Writing Any Code

**MUST** follow this sequence before implementation:

1. **Define the plan**: What are you doing and why? What invariants must hold?
2. **Check ADR/roadmap**: `docs/internals/adr/README.md` β†’ find relevant supplement
3. **Read the spec**: If touching state machines or protocols, read `docs/internals/specs/tla/*.tla`
4. **Write tests first**: DST tests define correctness, write them before code
5. **Only then**: Start implementation

## Three Engineering Pillars

Every code change **MUST** respect all three:

| Pillar | Location | Purpose |
|--------|----------|---------|
| **Code Quality** | [CODE_STYLE.md](../../../CODE_STYLE.md) + CLAUDE.md | Coding standards & reliability |
| **Formal Specs** | `docs/internals/specs/tla/`, `stateright_*.rs` | Protocol correctness |
| **DST** | DST crate (when created) | Fault tolerance |

**Priority**: Safety > Performance > Developer Experience

## The Verification Pyramid

All verification layers share the same invariants:

```
TLA+ Specs (docs/internals/specs/tla/*.tla)
β”‚ mirrors
Shared Invariants (invariants/) ← SINGLE SOURCE
β”‚ used by
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β–Ό β–Ό β–Ό
Stateright DST Tests Production Metrics
(exhaustive) (simulation) (Observability)
```

## Testing Through Production Path

**MUST NOT** claim a feature works unless tested through the actual network stack.

```bash
# 1. Start quickwit
cargo run -p quickwit-cli -- run --config ../config/quickwit.yaml

# 2. Ingest via OTLP
# (send logs/traces to localhost:4317)

# 3. Query via REST API
curl http://localhost:7280/api/v1/<index>/search -d '{"query": "*"}'
```

**Bypasses to AVOID**: Testing indexing pipeline without the HTTP/gRPC server, testing search without the REST API layer.

## DST (Deterministic Simulation Testing)

- DST tests define correctness for stateful components
- Write DST tests before implementation for new state machines
- Shared invariants are the single source of truth across all verification layers

## Architecture Evolution

Quickwit tracks architectural change through three lenses. See `docs/internals/adr/EVOLUTION.md` for the full process.

```
Architecture Evolution
β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β–Ό β–Ό β–Ό
Characteristics Gaps Deviations
(Proactive) (Reactive) (Pragmatic)
"What we need" "What we learned" "What we accepted"
```

| Lens | Location | When to Use |
|------|----------|-------------|
| **Characteristics** | `docs/internals/adr/` | Track cloud-native requirements |
| **Gaps** | `docs/internals/adr/gaps/` | Design limitation from incident/production |
| **Deviations** | `docs/internals/adr/deviations/` | Intentional divergence from ADR intent |

**Before implementing, check for**:
- Open gaps (design limitations to be aware of)
- Deviations (intentional divergence from ADRs)
- Characteristic status (what's implemented vs planned)

## Additional Commit Checklist (on top of CLAUDE.md MUST items)

These are expected unless justified:
- [ ] Functions under 70 lines
- [ ] Explanatory variables for complex expressions
- [ ] Documentation explains "why"
- [ ] ADR/roadmap updated if applicable
- [ ] DST test for new state transitions
- [ ] Integration test for new API endpoints
- [ ] Tests through production path (HTTP/gRPC)

## Reference Documentation

| Topic | Location |
|-------|----------|
| Verification & DST | [docs/internals/VERIFICATION.md](../../../docs/internals/VERIFICATION.md) |
| Verification philosophy | [docs/internals/VERIFICATION_STACK.md](../../../docs/internals/VERIFICATION_STACK.md) |
| Simulation workflow | [docs/internals/SIMULATION_FIRST_WORKFLOW.md](../../../docs/internals/SIMULATION_FIRST_WORKFLOW.md) |
| Benchmarking | [docs/internals/BENCHMARKING.md](../../../docs/internals/BENCHMARKING.md) |
| Rust style patterns | [docs/internals/RUST_STYLE.md](../../../docs/internals/RUST_STYLE.md) |
| ADR index | [docs/internals/adr/README.md](../../../docs/internals/adr/README.md) |
| Architecture evolution | [docs/internals/adr/EVOLUTION.md](../../../docs/internals/adr/EVOLUTION.md) |
| Compaction architecture | [docs/internals/compaction-architecture.md](../../../docs/internals/compaction-architecture.md) |
| Tantivy + Parquet design | [docs/internals/tantivy-parquet-architecture.md](../../../docs/internals/tantivy-parquet-architecture.md) |
| Locality compaction | [docs/internals/locality-compaction/](../../../docs/internals/locality-compaction/) |
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,6 @@ qwdata

# Generated by prost/tonic build
*_descriptor.bin

# GSD planning artifacts (local only)
.planning
Loading
Loading