feat: add sqlserver__openquery macro and SQL Server best-practices guide - #790
Open
axellpadilla wants to merge 2 commits into
Open
feat: add sqlserver__openquery macro and SQL Server best-practices guide#790axellpadilla wants to merge 2 commits into
axellpadilla wants to merge 2 commits into
Conversation
Adds sqlserver__openquery for linked-server pass-through queries: escapes quotes, strips carriage returns, brackets the server name, and enforces the 8 KB OPENQUERY limit with a compiler error. Validation runs at execution phase so one invocation surfaces every bad model. Adds functional tests covering emitted SQL and live OPENQUERY execution against a loopback linked server in a single dbt run, and docs/sqlserver-best-practices.md covering RCSI, tempdb sizing, max server memory/MAXDOP, recovery model, and OPENQUERY vs 4-part-name joins.
Collaborator
Author
|
@joshmarkovic , @Benjamin-Knight please review this best-practices guide |
The openquery tests set up a loopback linked server, hardcoded for the engines
it was written against. CI passed on SQL Server 2019 and 2022 and failed on
2017 and 2025, for two unrelated reasons:
- 2017 rejected sp_addlinkedserver with Msg 7222, "Only a SQL Server provider
is allowed on this instance". MSOLEDBSQL only became a valid linked-server
provider on Linux in 2019; 2017 needs the native SQLNCLI name.
- 2025 created the linked server fine, then failed every query through it with
"SSL Provider: The handle specified is invalid (-2146893055)" - the engine's
own outbound connection as an OLE DB client, not dbt's. From 2025 the
provider negotiates encryption by default and the loopback presents the
instance's self-signed certificate, so the handshake fails unless the
certificate is trusted explicitly.
Both now derive from SERVERPROPERTY('ProductMajorVersion'): SQLNCLI below 2019,
and @provstr = 'TrustServerCertificate=Yes' from 2025 on. The provider string
is sent only where needed, so 2019 and 2022 generate byte-identical SQL and
cannot regress. The SQLNCLI branch carries a note to drop it, with the 2017 CI
leg, once 2017 leaves extended support on 2027-10-12.
The failure also took out the six tests that need no linked server - the
compiled-SQL assertions and the four validation errors - because they hang off
the same class-scoped fixture. Left as is: with the setup fixed they run
everywhere, and a real linked-server regression should fail loudly, not skip.
axellpadilla
force-pushed
the
feat/openquery-macro-and-best-practices
branch
from
August 1, 2026 05:42
64cc887 to
ac6732a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
sqlserver__openqueryhelper for linked-server pass-through queries, itstests, and a SQL Server configuration guide. 3 files, +515, no existing code
touched.
Macro
dbt/include/sqlserver/macros/utils/openquery.sql{{ sqlserver__openquery('LOCALLOOP', "select id from RemoteDb.dbo.customers where active = 1") }} -- OPENQUERY([LOCALLOOP], 'select id from RemoteDb.dbo.customers where active = 1')OPENQUERYlimit — theerror reports the actual escaped length and points at
EXEC(...) ATor aremote view
Validation is execution-phase (
{% if execute %}) rather than parse-time, so onerun reports every bad model instead of aborting on the first.
Tests
tests/functional/adapter/mssql/test_openquery.py— eight models built by asingle
dbt run. Three executeOPENQUERYagainst aLOCALLOOPloopback linkedserver (
MSOLEDBSQL) and assert both the emitted SQL and the rows the serverreturned; the rest cover the length boundary and the validation errors. The
linked server is created and dropped inside the fixture.
Requires a live SQL Server and
sp_addlinkedserverrights. Verified9 passedon SQL Server 2022 (16.0.4265.3), mssql-python backend.
Guide
docs/sqlserver-best-practices.md— SQL Server defaults that are wrong for a dbtworkload, each with Microsoft's recommended value and why it matters for this
adapter:
table_refresh_method: dmlthreads > 1; RCSI's version store lives here too0threadsFULLSELECT INTO, how the adapter materializes tables, is only minimally logged outsideFULLPlus OPENQUERY over 4-part names: stage remote reads as models rather than
joining across the boundary, which also avoids error 3988 under the adapter's
default
XACT_ABORT ON.A closing section covers
cost threshold for parallelismandoptimize for ad hoc workloads— both marked as having no published target value and unmeasuredagainst a dbt workload, framed as candidates to test rather than settings to
apply.
All claims cited to Microsoft Learn; adapter-specific claims link to the macro or
materialization they come from.