Skip to content

#2525 spark metrics in depends on prometheus #2529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 29, 2025

Conversation

blcksrx
Copy link
Contributor

@blcksrx blcksrx commented May 8, 2025

Purpose of this PR

Proposed changes:

  • <Change 1>
  • <Change 2>
  • <Change 3>

Change Category

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Checklist

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

#2525

@ChenYi015 ChenYi015 linked an issue May 12, 2025 that may be closed by this pull request
1 task
@nabuskey
Copy link
Contributor

A bit confused here. This PR doesn't seem to address the issue linked. The functions modified are called when building prometheus configurations only. This means it doesn't allow for other monitoring solutions to work like mentioned in the issue.

@blcksrx
Copy link
Contributor Author

blcksrx commented May 14, 2025

I understand the confusion, it just naming and I have avoid to touch/rename more than that to make it simple
Although the function name is configPrometheusMonitoring but it builds the configMap file for metrics.properties whetever your MetricProperties or MetricsPropertiesFile are

https://github.com/kubeflow/spark-operator/blob/master/internal/controller/sparkapplication/monitoring_config.go#L125

@nabuskey
Copy link
Contributor

Hmm as far as I can tell, configPrometheusMonitoring is called if the prometheus field is not nil. So this means you have to put something in the prometheus field. I think writings tests here to show your intentions and results help here.

@blcksrx
Copy link
Contributor Author

blcksrx commented May 14, 2025

yes Exactly, my intention was to write the test but I saw that all the monitoring_test has been commented out

@google-oss-prow google-oss-prow bot added size/XL and removed size/XS labels May 19, 2025
@blcksrx blcksrx force-pushed the blcksrx/2525 branch 2 times, most recently from 6b8fe14 to 4cdbb25 Compare May 19, 2025 14:54
@google-oss-prow google-oss-prow bot added size/M and removed size/XL labels May 19, 2025
@blcksrx
Copy link
Contributor Author

blcksrx commented May 19, 2025

@nabuskey I've rebased my PR with your recent fix and added my test case there

Signed-off-by: Hossein Torabi <blcksrx@pm.me>
@blcksrx blcksrx requested a review from nabuskey May 20, 2025 08:36
@blcksrx
Copy link
Contributor Author

blcksrx commented May 26, 2025

@nabuskey I've reflected the comments,
Appreciate the review again

@nabuskey
Copy link
Contributor

/lgtm

Copy link
Contributor

@vara-bonthu vara-bonthu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vara-bonthu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 7b81972 into kubeflow:master May 29, 2025
15 checks passed
@ChenYi015
Copy link
Contributor

/cherrypick release-2.2

@google-oss-robot
Copy link

@ChenYi015: new pull request created: #2543

In response to this:

/cherrypick release-2.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spark metrics depends on Prometheus
5 participants