Skip to content

[DOCS-9950] Add limitNOFILE to OP troubleshooting #29077

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 8, 2025

Conversation

maycmlee
Copy link
Contributor

@maycmlee maycmlee commented Apr 30, 2025

What does this PR do? What is the motivation?

Add entry to OP troubleshooting.

Merge instructions

Merge readiness:

  • Ready for merge

For Datadog employees:
Merge queue is enabled in this repo. Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

To have your PR automatically merged after it receives the required reviews, add the following PR comment:

/merge

Additional notes

@maycmlee maycmlee added the WORK IN PROGRESS No review needed, it's a wip ;) label Apr 30, 2025
@maycmlee maycmlee requested a review from a team as a code owner April 30, 2025 19:29
Copy link
Contributor

Preview links (active after the build_preview check completes)

Modified Files

@maycmlee maycmlee removed the WORK IN PROGRESS No review needed, it's a wip ;) label May 7, 2025
@@ -65,6 +65,10 @@ If you try to install a new version of the Worker in an instance that is running

If you do not see Worker logs in [Log Explorer][12], make sure they are not getting excluded in your log pipelines. Worker logs must be indexed in Log Management for optimal functionality. The logs provide deployment information, such as Worker status, version, and any errors, that is shown in the Observability Pipelines UI. The logs are also helpful for troubleshooting Worker or pipelines issues. All Worker logs have the tag `source:op_worker`.

## Too many files error

If you see the error `Too many files` and the Worker processes continually to restart, it could be due to a low file descriptor limit on the host. To resolve this issue for Linux environments, set `LimitNOFILE` in the systemd service configuration to `65,536` to increase the file descriptor limit.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the phrase "the Worker processes continually to restart" right? I'm having trouble understanding it. Does it mean that the Worker processes repeatedly restart?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I updated to say "the Worker processes repeatedly restart". Thanks!

@maycmlee maycmlee requested a review from urseberry May 7, 2025 21:12
@@ -65,6 +65,10 @@ If you try to install a new version of the Worker in an instance that is running

If you do not see Worker logs in [Log Explorer][12], make sure they are not getting excluded in your log pipelines. Worker logs must be indexed in Log Management for optimal functionality. The logs provide deployment information, such as Worker status, version, and any errors, that is shown in the Observability Pipelines UI. The logs are also helpful for troubleshooting Worker or pipelines issues. All Worker logs have the tag `source:op_worker`.

## Too many files error

If you see the error `Too many files` and the Worker processes repeatedly restarts, it could be due to a low file descriptor limit on the host. To resolve this issue for Linux environments, set `LimitNOFILE` in the systemd service configuration to `65,536` to increase the file descriptor limit.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's still a plural error. It should either be "processes restart" or "process restarts".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@maycmlee maycmlee requested a review from urseberry May 7, 2025 21:23
@maycmlee
Copy link
Contributor Author

maycmlee commented May 7, 2025

/merge

@dd-devflow
Copy link

dd-devflow bot commented May 7, 2025

View all feedbacks in Devflow UI.

2025-05-07 21:25:32 UTC ℹ️ Start processing command /merge


2025-05-07 21:25:37 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2025-05-08 01:26:04 UTC ⚠️ MergeQueue: This merge request was unqueued

devflow unqueued this merge request: It did not become mergeable within the expected time

@maycmlee
Copy link
Contributor Author

maycmlee commented May 8, 2025

/merge

@dd-devflow
Copy link

dd-devflow bot commented May 8, 2025

View all feedbacks in Devflow UI.

2025-05-08 16:05:24 UTC ℹ️ Start processing command /merge


2025-05-08 16:05:31 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2025-05-08 16:35:16 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in master is approximately 21m (p90).


2025-05-08 16:52:03 UTC ℹ️ MergeQueue: This merge request was merged

@dd-mergequeue dd-mergequeue bot merged commit 91a4477 into master May 8, 2025
16 of 23 checks passed
@dd-mergequeue dd-mergequeue bot deleted the may/add-limitnofile-op-troubleshooting branch May 8, 2025 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants