Add team_name tag to Celery executor metrics - #69092
Merged
ferruzzi merged 1 commit intoJul 7, 2026
Merged
Conversation
SameerMesiah97
force-pushed
the
68996-Celery-Team-Name-Metrics
branch
2 times, most recently
from
June 28, 2026 13:07
bcf37a6 to
36e5e52
Compare
SameerMesiah97
marked this pull request as ready for review
June 28, 2026 14:20
SameerMesiah97
requested review from
dheerajturaga and
hussein-awala
as code owners
June 28, 2026 14:20
shivaam
approved these changes
Jul 1, 2026
Contributor
Author
|
I was hoping you could review this. |
Contributor
|
Ah, I didn't see it until you tagged me, I'll check it out. |
ferruzzi
reviewed
Jul 6, 2026
ferruzzi
left a comment
Contributor
There was a problem hiding this comment.
One trivial nit in the tests and one actual request for backcompat in the code; reasoning for skipping celery.execute_command.failure seems sound.
metric to improve per-team observability in multi-team deployments. Add unit test coverage for the task timeout retry path, verifying that the metric is emitted with and without the team_name tag.
SameerMesiah97
force-pushed
the
68996-Celery-Team-Name-Metrics
branch
from
July 6, 2026 20:50
36e5e52 to
8bbf10b
Compare
Contributor
Author
|
Thanks for the review. I have responded to your comments and have just rebased the branch. |
ferruzzi
approved these changes
Jul 6, 2026
o-nikolas
approved these changes
Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change adds the
team_nametag to thecelery.task_timeout_errormetric emitted by the Celery executor when retrying workload publication after anAirflowTaskTimeout.The
celery.execute_command.failuremetric was intentionally left unchanged. That metric is only emitted byexecute_command(), which is used on Airflow versions prior to 3.0, whereas multi-team support (andteam_name) was introduced in Airflow 3.1. As a result, there is no supported execution path wherecelery.execute_command.failurecan be associated with a team.Rationale
The
celery.task_timeout_errormetric is used to identify workload publication timeouts. Adding theteam_nametag allows these events to be attributed to the owning team in multi-team deployments, making it easier to diagnose publish failures and identify affected teams.Tests
Added a unit test covering the previously untested workload publication timeout retry path. The test verifies that
celery.task_timeout_erroris emitted with the expected tags both with and withoutteam_name.Backwards Compatibility
This change is additive only. The
team_nametag is only emitted when available; otherwise, the metric is emitted unchanged.Related: #68996
Was generative AI tooling used to co-author this PR?
Generated-by: [GPT 5.5] following the guidelines