sql-template.json

{
  "$schema": "../rpc-schema-draft.json",
  "type": "object",
  "added": "v23.02",
  "rpc": "sql",
  "title": "Command to do complex queries on list commands",
  "description": [
    "The **sql** RPC command runs the given query across a sqlite3 database created from various list commands.",
    "",
    "When tables are accessed, it calls the below commands, so it's no faster than any other local access (though it goes to great length to cache `listnodes` and `listchannels`) which then processes the results.",
    "",
    "It is, however faster for remote access if the result of the query is much smaller than the list commands would be.",
    "",
    "Note that you may need to use `-o` if you use queries which contain `=` (which make lightning-cli(1) default to keyword style)"
  ],
  "request": {
    "required": [
      "query"
    ],
    "additionalProperties": false,
    "properties": {
      "query": {
        "type": "string",
        "description": [
          "The standard sqlite3 query to run.",
          "Note that queries like \"SELECT *\" are fragile, as columns will change across releases; see lightning-listsqlschemas(7)."
        ]
      }
    }
  },
  "response": {
    "required": [
      "rows"
    ],
    "additionalProperties": false,
    "properties": {
      "rows": {
        "type": "array",
        "items": {
          "type": "array"
        }
      },
      "warning_db_failure": {
        "type": "string",
        "description": [
          "A message if the database encounters an error partway through."
        ]
      }
    },
    "pre_return_value_notes": [
      "On success, an object containing **rows** is returned. It is an array. Each array entry contains an array of values, each an integer, real number, string or *null*, depending on the sqlite3 type.",
      "",
      "The object may contain **warning_db_failure** if the database fails partway through its operation."
    ]
  },
  "treatment_of_types": [
    "The following types are supported in schemas, and this shows how they are presented in the database. This matters: a JSON boolean is represented as an integer in the database, so a query will return 0 or 1, not true or false.",
    "",
    "* *hex*. A hex string.",
    "  * JSON: a string",
    "  * sqlite3: BLOB",
    "",
    "* *hash*/*secret*/*pubkey*/*txid*: just like *hex*.",
    "",
    "* *msat*/*integer*/*u64*/*u32*/*u16*/*u8*. Normal numbers.",
    "  * JSON: an unsigned integer",
    "  * sqlite3: INTEGER",
    "",
    "* *boolean*. True or false.",
    "  * JSON: literal **true** or **false**",
    "  * sqlite3: INTEGER",
    "",
    "* *number*. A floating point number (used for times in some places).",
    "  * JSON: number",
    "  * sqlite3: REAL",
    "",
    "* *string*. Text.",
    "  * JSON: string",
    "  * sqlite3: TEXT",
    "",
    "* *short_channel_id*. A short-channel-id of form 1x2x3.  Stored as an integer internally for efficient indexing.",
    "  * JSON: string",
    "  * sqlite3: SCID (INTEGER affinity)",
    "",
    "You can use the `scid()` function to convert a short_channel_id string to its integer representation for queries, e.g. `WHERE in_channel = scid('1x2x3')`.  The `fmt_scid()` function converts back to string form."
  ],
  "permitted_sqlite3_functions": [
    "Writing to the database is not permitted, and limits are placed on various other query parameters.",
    "",
    "Additionally, only the following functions are allowed:",
    "",
    "* abs",
    "* avg",
    "* coalesce",
    "* count",
    "* date",
    "* datetime",
    "* julianday",
    "* hex",
    "* quote",
    "* length",
    "* like",
    "* lower",
    "* upper",
    "* min",
    "* max",
    "* strftime",
    "* sum",
    "* time",
    "* timediff",
    "* total",
    "* unixepoch",
    "* json_object",
    "* json_group_array",
    "* scid",
    "* fmt_scid"
  ],
  "tables": [
    "Note that tables which have a `created_index` field use that as the primary key (and `rowid` is an alias to this), otherwise an explicit `rowid` integer primary key is generated, whose value changes on each refresh.  This field is used for related tables to refer to specific rows in their parent. (sqlite3 usually has this as an implicit column, but we make it explicit as the implicit version is not allowed to be used as a foreign key).",
    ""
  ],
  "errors": [
    "On failure, an error is returned."
  ],
  "author": [
    "Rusty Russell [rusty@rustcorp.com.au](mailto:rusty@rustcorp.com.au) is mainly responsible."
  ],
  "see_also": [
    "lightning-listtransactions(7)",
    "lightning-listhtlcs(7)",
    "lightning-listinvoices(7)",
    "lightning-listoffers(7)",
    "lightning-listchannels(7)",
    "lightning-listpeers(7)",
    "lightning-listpeerchannels(7)",
    "lightning-listsendpays(7)",
    "lightning-listchainmoves(7)",
    "lightning-listchannelmoves(7)",
    "lightning-bkpr-listaccountevents(7)",
    "lightning-bkpr-listincome(7)",
    "lightning-listnodes(7)",
    "lightning-listforwards(7)"
  ],
  "resources": [
    "Main web site: [https://github.com/ElementsProject/lightning](https://github.com/ElementsProject/lightning)"
  ],
  "examples": [
    {
      "description": [
        "A simple peers selection query:"
      ],
      "request": {
        "id": "example:sql#1",
        "method": "sql",
        "params": {
          "query": "SELECT id FROM peers"
        }
      },
      "response": {
        "rows": [
          [
            "nodeid020202020202020202020202020202020202020202020202020202020202"
          ]
        ]
      }
    },
    {
      "description": [
        "A statement containing `=` needs `-o` in shell:"
      ],
      "request": {
        "id": "example:sql#2",
        "method": "sql",
        "params": [
          "SELECT label, description, status FROM invoices WHERE label='label inv_l12'"
        ]
      },
      "response": {
        "rows": [
          [
            "label inv_l12",
            "description inv_l12",
            "unpaid"
          ]
        ]
      }
    },
    {
      "description": [
        "If you want to get specific nodeid values from the nodes table:"
      ],
      "request": {
        "id": "example:sql#3",
        "method": "sql",
        "params": [
          "SELECT nodeid FROM nodes WHERE nodeid != x'nodeid030303030303030303030303030303030303030303030303030303030303'"
        ]
      },
      "response": {
        "rows": [
          [
            "nodeid020202020202020202020202020202020202020202020202020202020202"
          ],
          [
            "nodeid010101010101010101010101010101010101010101010101010101010101"
          ],
          [
            "nodeid040404040404040404040404040404040404040404040404040404040404"
          ]
        ]
      }
    },
    {
      "description": [
        "If you want to compare a BLOB column, `x'hex'` or `X'hex'` are needed:"
      ],
      "request": {
        "id": "example:sql#4",
        "method": "sql",
        "params": [
          "SELECT nodeid FROM nodes WHERE nodeid IN (x'nodeid010101010101010101010101010101010101010101010101010101010101', x'nodeid030303030303030303030303030303030303030303030303030303030303')"
        ]
      },
      "response": {
        "rows": [
          [
            "nodeid010101010101010101010101010101010101010101010101010101010101"
          ],
          [
            "nodeid030303030303030303030303030303030303030303030303030303030303"
          ]
        ]
      }
    },
    {
      "description": [
        "Related tables are usually referenced by JOIN:"
      ],
      "request": {
        "id": "example:sql#5",
        "method": "sql",
        "params": [
          "SELECT peer_id, to_us_msat, total_msat, peerchannels_status.status FROM peerchannels INNER JOIN peerchannels_status ON peerchannels_status.row = peerchannels.rowid"
        ]
      },
      "response": {
        "rows": [
          [
            "nodeid020202020202020202020202020202020202020202020202020202020202",
            490493792,
            1000000000,
            "CHANNELD_NORMAL:Channel ready for use."
          ]
        ]
      }
    },
    {
      "description": [
        "Simple function usage, in this case COUNT. Strings inside arrays need \", and ' to protect them from the shell:"
      ],
      "request": {
        "id": "example:sql#6",
        "method": "sql",
        "params": [
          "SELECT COUNT(*) FROM forwards"
        ]
      },
      "response": {
        "rows": [
          [
            9
          ]
        ]
      }
    },
    {
      "request": {
        "id": "example:sql#7",
        "method": "sql",
        "params": [
          "SELECT * from peerchannels_features"
        ]
      },
      "response": {
        "rows": [
          [
            28,
            23,
            0,
            "option_static_remotekey"
          ],
          [
            30,
            23,
            2,
            "option_anchors"
          ]
        ]
      }
    }
  ]
}