Skip to content

docs: add SQL schema documentation - #8493

Draft
link2xt wants to merge 6 commits into
mainfrom
link2xt/sql-schema-docs
Draft

docs: add SQL schema documentation#8493
link2xt wants to merge 6 commits into
mainfrom
link2xt/sql-schema-docs

Conversation

@link2xt

@link2xt link2xt commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

There is currently no place to document up to date SQL schema. New database is initialized with src/sql/tables.sql and then updated by migrations,
So tracking down how some column is used requires grepping the code, looking for a migration adding it and corresponding commits.

The documentation is frozen at version 160.
Ideally it should be updated when adding a migration, but most of the value is documenting old stable tables.

There was an idea to automatically check
if the documentation corresponds to the database schema, but there is no tool to compare the schema.
SQLite itself stores database schema as the text
in the schema table, with columns appended later
without any formatting.
Even SQLite REPL which has ".schema --indent" option does not handle the comments properly
and produces invalid SQL
with closing parenthesis commented out.

Comment thread docs/schema.sql
starred INTEGER DEFAULT 0,
timestamp_sent INTEGER DEFAULT 0, -- Timestamp of the message as sent in the Date header.
timestamp_rcvd INTEGER DEFAULT 0,
hidden INTEGER DEFAULT 0,

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bad column, needs a good documentation.

In many cases the message should be trashed instead of hiding. Hiding is used for rare cases when you want the message to be "in the chat" so they are marked as seen. One is reactions, but there was something else, i think location streaming. Would be great to deprecate it somehow or at least discourage more usage of it. The problem with such hidden messages is that it is unclear when they are supposed to be cleaned up and they likely slow down chat loading if you have lots of reactions in the chat.

Comment thread docs/schema.sql Outdated
Comment thread docs/schema.sql
Comment thread docs/schema.sql Outdated
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 8e4c7f3 to 4e7ee48 Compare July 31, 2026 23:05
@hpk42

hpk42 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Quite like it. I didn't review the comments except in passing (maybe @r10s is also a review authority to reckon with there :), but did a PR #8514 to make the schema become tested in CI.

I wonder if we could evolve to introduce an env DC_DBCREATE_MIGRATIONS=1 or so which would retain today's behaviour, but the default could become creating the DB from the actual documented schema, basically bumping the tables.sql squash point from 68 to current. Not in this PR of course.

@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 606d56d to 8bafac5 Compare August 3, 2026 14:16
link2xt and others added 6 commits August 3, 2026 14:28
There is currently no place to document up to date SQL schema.
New database is initialized with src/sql/tables.sql
and then updated by migrations,
So tracking down how some column is used requires grepping the code,
looking for a migration adding it and corresponding commits.

The documentation is frozen at version 160.
Ideally it should be updated when adding a migration,
but most of the value is documenting old stable tables.

There was an idea to automatically check
if the documentation corresponds to the database schema,
but there is no tool to compare the schema.
SQLite itself stores database schema as the text
in the schema table, with columns appended later
without any formatting.
Even SQLite REPL which has ".schema --indent" option
does not handle the comments properly
and produces invalid SQL
with closing parenthesis commented out.
finds things like "exists in the db, but is not documented" or "documented but not in the db".
@link2xt
link2xt force-pushed the link2xt/sql-schema-docs branch from 8bafac5 to 7ac86fc Compare August 3, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants