Skip to content

Fix empty labels in Stackdriver log IO for Airflow 3 Supervisor#68292

Open
23tae wants to merge 2 commits into
apache:mainfrom
23tae:fix-gcl-empty-labels
Open

Fix empty labels in Stackdriver log IO for Airflow 3 Supervisor#68292
23tae wants to merge 2 commits into
apache:mainfrom
23tae:fix-gcl-empty-labels

Conversation

@23tae

@23tae 23tae commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

This PR adds fallback logic to StackdriverRemoteLogIO to extract task labels directly from the structlog context dictionary when record.task_instance is missing, addressing the empty labels issue in the Airflow 3 Supervisor.

Description

This PR addresses Bug 1 described in #68240.

In the Airflow 3 Task SDK Supervisor environment, the task_instance object may be stripped from log records. Without it, the Stackdriver handler failed to attach identifying labels, resulting in untraceable "empty label" logs in Google Cloud.

As suggested by @ashb in the issue, rather than manually parsing log file paths, this PR leverages the structlog context variables. The Task SDK inherently binds task details to the structlog context, making them naturally available in the event dictionary.

Key changes

  • Fallback Label Extraction: Added fallback logic in StackdriverRemoteLogIO.processors.proc. If record.task_instance is missing, it safely extracts dag_id, task_id, run_id, try_number, and map_index directly from the event dictionary and applies them as labels.

Verification Results

I have verified the changes using prek and breeze.

  • Static Checks (Prek): Passed
  • Unit Tests (Breeze): Passed

related: #68240


Was generative AI tooling used to co-author this PR?
  • Yes

Generated-by: Antigravity following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@23tae 23tae requested a review from shahar1 as a code owner June 9, 2026 15:06
@boring-cyborg boring-cyborg Bot added area:logging area:providers provider:google Google (including GCP) related issues labels Jun 9, 2026
@23tae 23tae changed the title Fix empty labels in Stackdriver Log IO for Airflow 3 Supervisor Fix empty labels in Stackdriver log IO for Airflow 3 Supervisor Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:logging area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant