Go: Add overlay annotations from script

[owen-mc]

#20623 enabled basic overlay support for Go. Performance testing showed that this led to a significant speedup for larger projects, but for smaller projects it was causing a small slowdown. This PR adds some overlay annotations to the CodeQL Go library by running the script config/add-overlay-annotations.py. This increases the number of calculations that do not have to be redone. Performance testing shows that this is a significant improvement. Not all projects have a speedup, but even when limiting to small repos (which I'm defining as <100s analysis time) we see a speedup on average.

[Copilot]

Pull request overview

This PR expands Go’s overlay support by adding overlay annotations and explicit module; declarations across the Go CodeQL library, aiming to reduce recomputation and improve performance (especially on smaller projects) under overlay analysis.

Changes:

• Add overlay[local?] + module; declarations to many Go library .qll files so they participate in overlay-aware compilation/caching.
• Add overlay[caller?] annotations to selected inline/local flow & taint helper predicates to enable inlining across the overlay frontier.
• Apply the same overlay/module pattern to Go inline-expectations test implementation library.

Reviewed changes

Copilot reviewed 188 out of 188 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
go/ql/lib/utils/test/internal/InlineExpectationsTestImpl.qll	Add overlay/module header for overlay-aware test library compilation.
go/ql/lib/semmle/go/security/ZipSlipCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/ZipSlip.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/Xss.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/XPathInjectionCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/XPathInjection.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/WeakSensitiveDataHashingCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/UrlConcatenation.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/UnsafeUnzipSymlinkCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/UnsafeUnzipSymlink.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/UncontrolledAllocationSizeCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/UncontrolledAllocationSize.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/TaintedPathCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/TaintedPath.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/StringBreakCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/StringBreak.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/StoredXssCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/StoredXss.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/StoredCommand.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/SqlInjectionCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/SqlInjection.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/SensitiveActions.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/Sanitizers.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/SafeUrlFlowCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/SafeUrlFlow.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/RequestForgeryCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/RequestForgery.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/ReflectedXssCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/ReflectedXss.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/OpenUrlRedirectCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/OpenUrlRedirect.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/MissingJwtSignatureCheckCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/MissingJwtSignatureCheck.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/LogInjectionCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/LogInjection.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/InsecureRandomnessCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/InsecureRandomness.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/InsecureFeatureFlag.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/IncorrectIntegerConversionLib.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/HardcodedCredentials.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/FlowSources.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/ExternalAPIs.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CookieWithoutSecure.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CookieWithoutHttpOnly.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CommandInjectionCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CommandInjection.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CleartextLoggingCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/CleartextLogging.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/AllocationSizeOverflowCustomizations.qll	Add overlay/module header.
go/ql/lib/semmle/go/security/AllocationSizeOverflow.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Unsafe.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/TextTemplate.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/TextTabwriter.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Syscall.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Strings.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Strconv.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Regexp.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Reflect.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/PathFilepath.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Path.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Os.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/NetTextproto.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/NetHttpHttputil.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/NetHttp.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Net.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/MimeQuotedprintable.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/MimeMultipart.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Log.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/IoFs.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Io.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/HtmlTemplate.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Html.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Fmt.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Errors.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingXml.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingPem.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingJson.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingGob.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingCsv.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/EncodingAsn1.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/DatabaseSql.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/CryptoTls.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/CompressZlib.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/CompressLzw.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/CompressGzip.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/CompressFlate.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/Bufio.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/ArchiveZip.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/stdlib/ArchiveTar.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Zap.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Yaml.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/XPath.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/XNetHtml.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/WebSocket.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Twirp.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Testing.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/SystemCommandExecutors.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Stdlib.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Squirrel.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Spew.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/SQL.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/RsCors.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Revel.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Protobuf.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/NoSQL.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Mux.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Macaron.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Logrus.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/K8sIoClientGo.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/K8sIoApimachineryPkgRuntime.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/K8sIoApiCoreV1.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Jwt.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Gqlgen.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Gorqlite.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/GoMicro.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/GoKit.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/GoJose.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Glog.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/GinCors.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Gin.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Fasthttp.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Encoding.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Email.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/ElazarlGoproxy.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Echo.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/CryptoLibraries.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Couchbase.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Bun.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/BeegoOrm.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Beego.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/AwsLambda.qll	Add overlay/module header.
go/ql/lib/semmle/go/frameworks/Afero.qll	Add overlay/module header.
go/ql/lib/semmle/go/dependencies/SemVer.qll	Add overlay/module header.
go/ql/lib/semmle/go/dependencies/Dependencies.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/TaintTrackingUtil.qll	Add overlay/module header; mark selected inline predicates overlay[caller?].
go/ql/lib/semmle/go/dataflow/internal/TaintTrackingImplSpecific.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/FlowSummaryImpl.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/ExternalFlowExtensions.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowUtil.qll	Add overlay/module header; mark local flow predicate overlay[caller?].
go/ql/lib/semmle/go/dataflow/internal/DataFlowPrivate.qll	Add overlay/module header; mark inline helper overlay[caller?].
go/ql/lib/semmle/go/dataflow/internal/DataFlowNodes.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowImplSpecific.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowImplConsistency.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowImplCommon.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowImpl.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/internal/DataFlowDispatch.qll	Add overlay/module header; mark parameter/arg helpers overlay[caller?].
go/ql/lib/semmle/go/dataflow/internal/ContainerFlow.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/barrierguardutil/UrlCheck.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/barrierguardutil/RegexpCheck.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/barrierguardutil/RedirectCheckBarrierGuard.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/TaintTracking.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/SsaImpl.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/SSA.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/Properties.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/GlobalValueNumbering.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/FunctionInputsAndOutputs.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/FlowSummary.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/ExternalFlow.qll	Add overlay/module header.
go/ql/lib/semmle/go/dataflow/DataFlow.qll	Add overlay/module header.
go/ql/lib/semmle/go/controlflow/IR.qll	Add overlay/module header.
go/ql/lib/semmle/go/controlflow/ControlFlowGraphImpl.qll	Add overlay/module header.
go/ql/lib/semmle/go/controlflow/ControlFlowGraph.qll	Add overlay/module header; mark dominance helper overlay[caller?].
go/ql/lib/semmle/go/controlflow/BasicBlocks.qll	Add overlay/module header.
go/ql/lib/semmle/go/concepts/HTTP.qll	Add overlay/module header.
go/ql/lib/semmle/go/concepts/GeneratedFile.qll	Add overlay/module header.
go/ql/lib/semmle/go/VariableWithFields.qll	Add overlay/module header.
go/ql/lib/semmle/go/Util.qll	Add overlay/module header.
go/ql/lib/semmle/go/Types.qll	Add overlay/module header.
go/ql/lib/semmle/go/StringOps.qll	Add overlay/module header.
go/ql/lib/semmle/go/Stmt.qll	Add overlay/module header.
go/ql/lib/semmle/go/Scopes.qll	Add overlay/module header.
go/ql/lib/semmle/go/PrintAst.qll	Add overlay/module header.
go/ql/lib/semmle/go/Packages.qll	Add overlay/module header.
go/ql/lib/semmle/go/Locations.qll	Add overlay/module header.
go/ql/lib/semmle/go/HTML.qll	Add overlay/module header.
go/ql/lib/semmle/go/GoMod.qll	Add overlay/module header.
go/ql/lib/semmle/go/Files.qll	Add overlay/module header.
go/ql/lib/semmle/go/Expr.qll	Add overlay/module header.
go/ql/lib/semmle/go/Errors.qll	Add overlay/module header.
go/ql/lib/semmle/go/DiagnosticsReporting.qll	Add overlay/module header.
go/ql/lib/semmle/go/Decls.qll	Add overlay/module header.
go/ql/lib/semmle/go/Concepts.qll	Add overlay/module header.
go/ql/lib/semmle/go/Comments.qll	Add overlay/module header.
go/ql/lib/semmle/go/Architectures.qll	Add overlay/module header.
go/ql/lib/semmle/go/AST.qll	Add overlay/module header.
go/ql/lib/ideContextual.qll	Add overlay/module header.
go/ql/lib/go.qll	Add overlay/module header.
go/ql/lib/Customizations.qll	Add overlay/module header.

[owen-mc]

I've had to rebase on main to get the tests to run. The overlay DCA runs were done on older commits. I don't think this affects their validity, as I don't think anything relating to this has changed recently.

[owen-mc]

Ah, running DCA (not in overlay mode) highlighted that maybe these annotations have introduced some bad join orders. Let me investigate those before this gets reviewed. I'll move it into draft.