Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Jvolivie/fix spdk #16098

Draft
wants to merge 86 commits into
base: jvolivie/remove_raft_rpm
Choose a base branch
from
Draft

Jvolivie/fix spdk #16098

wants to merge 86 commits into from

Conversation

jolivier23
Copy link
Contributor

@jolivier23 jolivier23 commented Mar 14, 2025 •
edited
Loading

Work in progress that depends on a few PRs in-flight

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

wangdi1 and others added 30 commits March 4, 2025 04:01
@wangdi1
DAOS-17207 daos: upgrade to SPDK 24.09
08c9a40 
Upgrade to SPDK24.09

Signed-off-by: Di Wang <ddiwang@google.com>
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@wangdi1
DAOS-17207 build: fix style
6cda205 
fix style

Signed-off-by: Di Wang <ddiwang@google.com>
@wangdi1
DAOS-17027 bio: fix style
d5bf1b4 
fix style and add those binary in app

Signed-off-by: Di Wang <ddiwang@google.com>
@wangdi1
DAOS-17207 build: fix style issue
5a8186a 
fix style issue.

Signed-off-by: Di Wang <ddiwang@google.com>
@JohnMalmberg
SRE-2832 Build: Improve CI portability.
d03e14f 
Update Jenkinsfile to support testing SCRIPT_SPLITTING_TRANSFORMATION per
https://issues.jenkins.io/browse/JENKINS-56500.

Commented out for now the Ubuntu 20.04 build as Ubuntu 20.04 is
shortly going out of support, and the Ubuntu build process needs
a rewrite for proper support.

ci/provisioning/post_provision_config_common_functions.sh:
  Fix cleanup of old repos not be Jenkins hostname specific.

ci/provisioning/post_provision_config_nodes_EL_8.sh
  Fix to use environment variable for repository URL.

utils/githooks/README.md:
  Add packages that need or are recommended to be installed.
  Fix markuplint flagged issues.

utils/githooks/pre-commit.d:
  Use JENKINS_HOST for specifying the Jenkins host and report it.
  Remove unneeded and confusing CURL_PROXY code.
  cURL uses standard HTTPS_PROXY

utils/scripts/helpers/install_maldet.ch :
  Fix defaults to be more implementation neutral.

utils/scripts/helpers/repo-helper-*.sh :
  Optionally install CA needed to validate local repository mirrors.

Dockerfiles modified to support intalling the CA.

Dockerfile.code_scanning modified to allow specifying the fedora
version used.

Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@wangdi1
DAOS-17207 bio: update the patch
651a8cd 
Use subsys_init_cb directly.

Signed-off-by: Di Wang <ddiwang@google.com>
@wangdi1
DAOS-17207 bio: update patch
d878b6e 
Update the function and fix more jason file.

Signed-off-by: Di Wang <ddiwang@google.com>
@JohnMalmberg
SRE-2832 Fix Ubuntu DAOS builds
34a8dc8 
Fix to not to depend on curl not being installed to know to install
other needed software products.

Fix to only install PPA for golang for Ubuntu 20.04 as newer versions
should have golang available from the official distribution.

Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 build Ubuntu needs locales package
9c47136 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
Merge branch 'master' into malmberg/sre-2832
05e982c
@JohnMalmberg
SRE-2832 Fix Ubuntu install
300140f 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Another Ubuntu fix
a60d21d 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Put the PPA back in for now.
5fea176 
It looks like the PPA may be adding in the long end of life Java-8
as current versions of Ubuntu do not have a Java-11 package.

Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Remove PPA again No java-8 available
00a9bb4 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Ubuntu container missing python3
195bc9d 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Add python3-venv package for Ubuntu
ec85e8d 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Ubuntu Dockerfile expects GCC.
e20c19c 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Ubuntu needs python3-dev
ff78854 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Need more UBUNTU packages
244161f 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Use ubuntu-install.sh
4356c5a 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Fix path for Ubuntu install
c6cd889 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@JohnMalmberg
SRE-2832 Still need the PPA for golang
61f8e1e 
Signed-off-by: John E. Malmberg <john.malmberg@hpe.com>
@grom72
SRE-2832 Still need https_proxy for proper build
b384bf9 
Disabble also ARM build as we do not have ARM self-hosted runners.

Doc-only: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72
SRE-2832 Disable ARM builds
ff81a2e 
Disabble ARM builda as we do not have ARM self-hosted runners.
Doc-only: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72
Revert "SRE-2832 Disable ARM builds"
b6e95be 
Execute dummy failed build to get PR validation faster

Doc-only: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72
SRE-2832 Build: Skip ARM64 builds in CI
a38cce1 
Disabble ARM builds as we do not have ARM64 self-hosted runners.

Priority: 2

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@jolivier23
DAOS-17253 build: Fix client only build
972141c 
* When tests are specified, some server components are built.
Argobots is still built for some mocks but this is acceptable
for now.
* Add conditional build to rpm spec
* Remove server side build from Ubuntu packaging

Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@wangdi1
DAOS-17207 bio: fix typo
bea407e 
fix typo

Signed-off-by: Di Wang <ddiwang@google.com>
@jolivier23
Merge remote-tracking branch 'comm/master' into jvolivie/client_only
fea0cdf
@jolivier23
Empty commit for testing
81c7e7d 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@tanabarr
Copy link
Contributor

I think there could be 2 related issues resulting in the operation not permitted errors, think there is probably no hugepages being allocated and this is probably a result of permission issues with scripts and binaries. Maybe shared library is also not accessible

I think this is the first problem:

ERROR: /usr/bin/daos_server_helper ERROR: pbin: code = 5 description = "resetting device bindings: reset: spdk setup failed (): stat /usr/share/daos/control/setup_spdk.sh: no such file or directory"

we used to copy them across in the source build

maybe something is not matching up with site_scons/components/init.py L363 and the changes in utils/setup_daos_server_helper.sh

ryon-jensen and others added 2 commits March 18, 2025 06:26
@ryon-jensen
Make rpmlint warnings errors
c685e22
@jolivier23
push another patch
16ae529 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/39/execution/node/342/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/39/execution/node/345/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/39/execution/node/338/log

@daosbuild1
Copy link
Collaborator

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/39/execution/node/333/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/40/execution/node/287/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/40/execution/node/310/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/40/execution/node/313/log

@jolivier23
Merge branch 'jvolivie/remove_raft_rpm' into jvolivie/fix_spdk
9d5f217 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@jolivier23
Get recursion to work
743930e 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@jolivier23
Merge remote-tracking branch 'comm/jvolivie/remove_raft_rpm' into jvo…
9846186 
…livie/fix_spdk
@jolivier23
Fix archive.sh for recursive submodules
3a185a3 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@jolivier23
Add recursive submodule support
560d154 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@jolivier23 jolivier23 changed the base branch from master to jvolivie/remove_raft_rpm March 19, 2025 03:15
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/42/execution/node/311/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/42/execution/node/288/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/42/execution/node/314/log

@jolivier23
Merge branch 'jvolivie/remove_raft_rpm' into jvolivie/fix_spdk
a1b7a45 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status UNSTABLE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/43/execution/node/287/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/43/execution/node/313/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/43/execution/node/300/log

@jolivier23
A couple more changes
e599df3 
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/44/execution/node/287/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/44/execution/node/313/log

@daosbuild1
Copy link
Collaborator

Test stage Build RPM on Leap 15.5 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-16098/44/execution/node/310/log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanabarr tanabarr tanabarr left review comments

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@jolivier23 @daosbuild1 @tanabarr @wangdi1 @JohnMalmberg @grom72 @ryon-jensen