Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAOS-16366 test: Use agent/server config files from test directory #14944

Merged
merged 21 commits into from
Aug 28, 2024
Merged

DAOS-16366 test: Use agent/server config files from test directory #14944

merged 21 commits into from
Aug 28, 2024

Conversation

phender
Copy link
Contributor

@phender phender commented Aug 15, 2024 •
edited
Loading

Use agent, control, and server config files placed in the common test directory instead of /etc/daos with a systemctl override configuration file.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

@phender
DAOS-16366 test: Use agent/server config files from test directory
ef9c05e 
Initial support for utilizing agnet, control,a nd server config files
placed in the common test directory.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 15, 2024 •
edited
Loading

Ticket title is 'ftest: run agent/server commands with config files sourced from test directory'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-16366

phender added 5 commits August 15, 2024 23:29
@phender
Import workaround.
ddeb132 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Minor updates.
2bd6663 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Fix import issue.
52c3434 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Merge branch 'master' into pahender/DAOS-16366
2971065
@phender
Update imports.
c54382f 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/5/execution/node/798/log

@phender
Additional changes.
f22f93a 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/6/execution/node/798/log

phender added 3 commits August 16, 2024 16:27
@phender
FIx systemctl daemon-reload
ddfc597 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true
Test-tag: test_setup

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Additional updates.
9438c7b 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Fix removal of systemctl override config file.
9e69cfc 
Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/10/execution/node/799/log

phender added 3 commits August 19, 2024 09:06
@phender
Merge branch 'master' into pahender/DAOS-16366
9c92b0a
@phender
Fix ddb.py import.
8be16a2 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Fix config file collection.
380f83a 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/12/execution/node/798/log

@phender
Ensure the config file directories exist for all tests.
19ea367 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Copy link
Contributor Author

phender commented Aug 21, 2024

Tests passed in https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-14944/13/testReport/

@phender phender marked this pull request as ready for review August 21, 2024 01:52
@phender phender requested review from a team as code owners August 21, 2024 01:52
@phender phender added the forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed. label Aug 21, 2024
mjean308
mjean308 previously approved these changes Aug 21, 2024
View reviewed changes
daltonbohning
daltonbohning reviewed Aug 21, 2024
View reviewed changes
Copy link
Contributor

@daltonbohning daltonbohning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't thought of this entirely from a high level yet so I will need another pass and will need to test on Aurora and/or boro

@phender
Applying feedback.
e608092 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender phender requested a review from daltonbohning August 23, 2024 14:32
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/16/execution/node/1099/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/16/execution/node/1145/log

@phender
Fix support_collect_log().
4c3f320 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse DestroyTests DaosAgentSupportCollectLogTest DaosServerSupportCollectLogTest

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/17/execution/node/1065/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/17/execution/node/1086/log

daltonbohning
daltonbohning reviewed Aug 26, 2024
View reviewed changes
Copy link
Contributor

@daltonbohning daltonbohning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this running locally

sudo -n chown dbohning:dbohninx /var/run/daos_agent

When calling verify_socket_directory we should do
verify_socket_directory(self.job.manager.certifiate_owner)

@phender
Applying feedback.
3949777 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse DestroyTests DaosAgentSupportCollectLogTest DaosServerSupportCollectLogTest

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/18/execution/node/1010/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/18/execution/node/1107/log

@phender phender requested a review from daltonbohning August 27, 2024 17:36
@phender
Copy link
Contributor Author

phender commented Aug 27, 2024

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/18/execution/node/1107/log

Failure is known issue https://daosio.atlassian.net/browse/DAOS-16367

phender added 2 commits August 27, 2024 14:11
@phender
Remove extra import.
e2fccb6 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse DestroyTests DaosAgentSupportCollectLogTest DaosServerSupportCollectLogTest

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@phender
Merge branch 'master' into pahender/DAOS-16366
9a40074 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse DestroyTests DaosAgentSupportCollectLogTest DaosServerSupportCollectLogTest

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
daltonbohning
daltonbohning approved these changes Aug 27, 2024
View reviewed changes
Copy link
Contributor

@daltonbohning daltonbohning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I haven't tested on Aurora and I'm not sure if this would have issues on GCP

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Large completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/20/execution/node/1106/log

@daosbuild1
Copy link
Collaborator

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/20/execution/node/1152/log

@phender
Copy link
Contributor Author

phender commented Aug 28, 2024 •
edited
Loading

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/20/execution/node/1152/log

Theses failures were caused by an intermittent ib1 link on wolf-221:

    Aug 28 03:42:30 wolf-221.wolf.hpdd.intel.com kernel: IPv6: ADDRCONF(NETDEV_UP): ib1: link is not ready
    Aug 28 03:42:30 wolf-221.wolf.hpdd.intel.com kernel: IPv6: ADDRCONF(NETDEV_UP): ib1: link is not ready
    Aug 28 03:42:30 wolf-221.wolf.hpdd.intel.com kernel: IPv6: ADDRCONF(NETDEV_CHANGE): ib1: link becomes ready

it affected 02-./daos_test/suite.py:DaosCoreTest.test_daos_management and 04-./daos_test/suite.py:DaosCoreTest.test_daos_container:

08/28-03:53:00.05 wolf-221 DAOS[109403/0/3178] external ERR  # [1362.010237] mercury->msg: [error] /builddir/build/BUILD/mercury-2.4.0rc4/src/na/na_ofi.c:5920
 # na_ofi_tag_send(): fi_tsend() failed, rc: -99 (Cannot assign requested address), buf=0x7fd3d0078098, len=1163, desc=0x7fd3cc5a9a50, dest_addr=7, tag=4294968796, context=0x7fd3cc5b4960
08/28-03:53:00.05 wolf-221 DAOS[109403/0/3178] external ERR  # [1362.010262] mercury->msg: [error] /builddir/build/BUILD/mercury-2.4.0rc4/src/na/na_ofi.c:8360
 # na_ofi_msg_send_unexpected(): Could not post msg send
08/28-03:53:00.05 wolf-221 DAOS[109403/0/3178] external ERR  # [1362.010278] mercury->rpc: [error] /builddir/build/BUILD/mercury-2.4.0rc4/src/mercury_core.c:4096
 # hg_core_forward_na(): Could not post send for input buffer (NA_ADDRNOTAVAIL)
08/28-03:53:00.05 wolf-221 DAOS[109403/0/3] hg   ERR  src/cart/crt_hg.c:1358 crt_hg_req_send_cb(0x7fd3cdda0ea0) [opc=0xf4020001 (CART:CRT_OPC_IV_UPDATE) rpcid=0x1afe5149000006f3 rank:tag=6:0] RPC failed; rc: DER_HG(-1020): 'Transport layer mercury error'
08/28-03:53:00.05 wolf-221 DAOS[109403/0/3178] server DBUG src/engine/server_iv.c:1042 iv_op_internal() class_id 10 opc 2 rc -1020
08/28-03:53:00.05 wolf-221 DAOS[109403/0/3178] server INFO src/engine/server_iv.c:1098 _iv_op() ns 12 retry for class 10 opc 2 rank 6/6: DER_HG(-1020): 'Transport layer mercury error'

but all other test variants passed.

https://daosio.atlassian.net/browse/SRE-2450

@phender phender requested a review from a team August 28, 2024 19:54
@phender
Copy link
Contributor Author

phender commented Aug 28, 2024 •
edited
Loading

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-14944/20/execution/node/1152/log

Does not seem to be related to these changes, created https://daosio.atlassian.net/browse/DAOS-16464

@daltonbohning daltonbohning merged commit 20f7361 into master Aug 28, 2024
46 of 49 checks passed
@daltonbohning daltonbohning deleted the pahender/DAOS-16366 branch August 28, 2024 21:28
phender added a commit that referenced this pull request Aug 29, 2024
@phender
DAOS-16366 test: Use agent/server config files from test directory (#…
555085a 
…14944)

Use agent, control, and server config files placed in the common test directory instead of /etc/daos with a systemctl override configuration file.

Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: pr test_setup_vm DdbTest DaosCoreTestDfuse DestroyTests DaosAgentSupportCollectLogTest DaosServerSupportCollectLogTest

Required-githooks: true

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
jolivier23 pushed a commit that referenced this pull request Aug 29, 2024
@phender @jolivier23
DAOS-16366 test: Use agent/server config files from test directory (#…
070d0cb 
…14944)

Use agent, control, and server config files placed in the common test directory instead of /etc/daos with a systemctl override configuration file.

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
daltonbohning pushed a commit that referenced this pull request Aug 30, 2024
@phender
DAOS-16366 test: Use agent/server config files from test directory (#…
cf7322a 
…14944) (#15033)

Use agent, control, and server config files placed in the common test directory instead of /etc/daos with a systemctl override configuration file.

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
@jolivier23 jolivier23 mentioned this pull request Sep 4, 2024
Merged
@mjmac mjmac mentioned this pull request Nov 13, 2024
Closed
@mjmac mjmac mentioned this pull request Mar 18, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daltonbohning daltonbohning daltonbohning approved these changes

@mjean308 mjean308 mjean308 approved these changes

Assignees
No one assigned
Labels
forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed.
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@phender @daosbuild1 @daltonbohning @mjean308