Skip to content

feat(logs): add tail and until retrieval filters#1764

Open
stephenlclarke wants to merge 10 commits into
apple:mainfrom
stephenlclarke:logs-tail-until-options
Open

feat(logs): add tail and until retrieval filters#1764
stephenlclarke wants to merge 10 commits into
apple:mainfrom
stephenlclarke:logs-tail-until-options

Conversation

@stephenlclarke

@stephenlclarke stephenlclarke commented Jun 20, 2026

Copy link
Copy Markdown

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Motivation and Context

This is a draft stacked follow-up to #1592. Chris's #1592 introduces the initial ContainerLogOptions API shape for since and timestamps; this draft extends that direction with the remaining line-retrieval filters needed by container logs and external API clients: tail and until.

The motivation is to keep Docker-style log retrieval semantics close to the service that owns the log files, rather than requiring every caller to replay and filter the complete stream locally. This is useful for the container logs CLI and for external orchestrators, while keeping Compose-specific formatting and policy out of apple/container.

Draft/stacking note: this branch is based on full-chaos/container:feat/chaos-1322-log-options from #1592. Because GitHub PRs to apple/container target main, this diff includes #1592 plus the follow-up commits below. The intended review delta for this draft is:

  • ece5d5d feat(logs): add tail and until retrieval filters
  • bd178e8 fix(logs): harden tail replay filters
  • acab75a test(logs): cover byte-preserving log replay
  • 881f9e9 fix(logs): fail closed when filtering logs
  • b8d7675 test(logs): close filtered log handles
  • f4e2723 test(logs): close command log handles
  • 03c2594 docs(logs): clarify followed time filter limitation
  • 4884d25 fix(logs): create filtered logs with private permissions
  • 897665a fix(logs): require unlinking filtered log files

What Changed

  • Extends ContainerLogOptions with tail and until while preserving the existing zero-option behavior.
  • Plumbs tail, since, and until through ContainerClient, XPC, the harness, and ContainersService.
  • Applies tail after service-side line filtering, with Docker-compatible negative-tail behavior treated as "all".
  • Adds a bounded service-side tail scan for static tail-only requests instead of reading the whole log into memory.
  • Adds byte-preserving local output for followed raw logs so negative tail and non-UTF-8 log bytes do not crash or decode incorrectly.
  • Adds container logs --tail long-form support and --since / --until CLI arguments.
  • Preserves unparseable and non-UTF-8 raw log bytes while filtering what can be safely filtered.
  • Fails closed when filtered log replay cannot create the temporary filtered handle, so callers do not silently receive unfiltered logs.
  • Creates filtered replay files with exclusive owner-only permissions and requires successful unlink before returning the replay handle.
  • Closes filtered and command log handles explicitly in focused tests.
  • Clarifies that followed time filters are not supported for this static-replay draft.
  • Updates the command reference for the new CLI options and the current followed time-filter limitation.

Relationship to Existing Work

Non-Goals

  • This does not add structured log records.
  • This does not add timestamp rendering for raw logs.
  • This does not add Compose-specific behavior.
  • This does not support followed --since / --until yet; those filters are static replay only in this draft.

Testing

  • Tested locally
  • Added/updated tests
  • Added/updated docs

Local verification:

swift format lint --strict --configuration .swift-format-nolint Sources/ContainerCommands/LogFileOutput.swift Sources/ContainerCommands/Container/ContainerLogs.swift Sources/Services/ContainerAPIService/Server/Containers/ContainersService.swift Tests/ContainerCommandsTests/ContainerLogsCommandTests.swift Tests/ContainerAPIServiceTests/ContainerLogsTests.swift Tests/ContainerResourceTests/ContainerLogOptionsTests.swift
git diff --check
swift build
swift test --filter ContainerLogsCommandTests
swift test --filter ContainerLogsTests
swift test --filter ContainerLogOptionsTests

Result: focused validation passed locally. The focused tests ran 25 tests across ContainerLogsCommandTests, ContainerLogsTests, and ContainerLogOptionsTests.

chrisgeo and others added 2 commits May 23, 2026 13:14
…ient.logs

Adds an additive overload to ContainerClient.logs that accepts a
ContainerLogOptions { since: Date?, timestamps: Bool } and plumbs the
two parameters through XPC into the daemon's existing log-handle path.

Motivation
----------

External orchestrators that drive the API server (the canonical use
case is a Compose-spec orchestrator implementing 'compose logs --since
<timestamp>' and '--timestamps') need to retrieve only recent log
output and optionally annotate lines with timestamps. Today
ContainerClient.logs(id:) returns raw containerLog + bootlog file
handles unconditionally; consumers either replay the entire log buffer
or implement their own line-by-line filter on the client. Surfacing
the parameters at the API boundary keeps the line scanning where the
file lives — server-side — and matches the docker-compose UX users
expect.

What this PR changes
--------------------

- Sources/ContainerResource/Container/ContainerLogOptions.swift (new):
  the ContainerLogOptions struct, Sendable + Codable, with a static
  '.default' equivalent to the original logs(id:) zero-config call.
- Sources/Services/ContainerAPIService/Client/XPC+.swift: two new
  XPCKeys cases ('logSince', 'logTimestamps') for the optional
  parameters.
- Sources/Services/ContainerAPIService/Client/ContainerClient.swift:
  new logs(id:options:) overload. The existing logs(id:) is retained
  as a thin wrapper over .default for source compatibility.
- Sources/Services/ContainerAPIService/Server/Containers/ContainersService.swift:
  matching logs(id:options:) overload; when options.since is provided,
  applies a private 'filterFileHandleSince' that parses ISO-8601
  timestamps from line starts and drops older lines (lines without a
  parseable timestamp pass through unchanged).
- Sources/Services/ContainerAPIService/Server/Containers/ContainersHarness.swift:
  parses the two optional XPC keys, builds ContainerLogOptions, and
  forwards to the new service overload.

Wire compatibility
------------------

XPC calls without the new keys decode as 'sinceRaw.timeIntervalSince1970
== 0' (the harness treats this as 'no since filter') and 'timestamps =
false', so older clients hitting a newer server see unchanged behavior.
Newer clients hitting an older server send the keys; an old harness
will silently drop them and run the original codepath.

Known limitations (intentional, follow-up work)
-----------------------------------------------

- The 'options.timestamps' parameter is plumbed end-to-end but the
  daemon does not currently decorate raw log lines that lack a
  timestamp prefix. Line decoration is a deliberate follow-up; keeping
  the parameter on the API surface today avoids a second wire-format
  break later.
- 'filterFileHandleSince' currently slurps the file into memory then
  returns a Pipe-backed FileHandle. For the typical container log
  size this is fine; for very large logs a streaming line iterator
  would be a worthwhile optimization. Filed as a known-issue for the
  follow-up timestamp-decoration PR.

Verification
------------

Full 'swift build' clean on macOS 26 / Apple silicon (release config,
all targets including downstream consumers of ContainerClient.logs).
@stephenlclarke

stephenlclarke commented Jun 20, 2026

Copy link
Copy Markdown
Author

@chrisgeo, I tried to add you as a reviewer but GitHub would not let me request review from this fork. These draft follow-ups are stacked on your log retrieval options work in #1592. No pressure for a full review yet; early guidance on API shape and whether this fits your intended direction would be really useful before I polish them for maintainer review.

@stephenlclarke stephenlclarke marked this pull request as ready for review June 20, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants