Skip to content

Add ubuntu 20.04 image for linux-riscv64 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 11, 2023

Conversation

luhenry
Copy link
Contributor

@luhenry luhenry commented Nov 7, 2023

No description provided.

@luhenry
Copy link
Contributor Author

luhenry commented Nov 7, 2023

I've also verified that the resulting image can build rocksdb with:

podman run --rm -it --platform linux/riscv64 -v /path/to/rocksdb:/rocksdb -w /rocksdb -v ~/.m2:/root/.m2 localhost/evolvedbinary/rocksjava:ubuntu20_riscv64-be bash -ex -c "make rocksdbjava"

@adamretter
Copy link
Member

@luhenry This looks like a great addition, but we don't have suitable hardware to be able to test this on. Also I see it uses JDK 11, but we require JDK 8 for building RocksJava.

@luhenry
Copy link
Contributor Author

luhenry commented Nov 8, 2023

we don't have suitable hardware to be able to test this on

We could be using QEMU on x86_64 or aarch64 machines to test the functionality. I could get that working if you are interested.

I see it uses JDK 11, but we require JDK 8 for building RocksJava.

JDK 8 is not available on RISC-V, and I don't expect it to be ever. The backport would is just too much work, and RISC-V is a new enough platform that people stuck on JDK 8 are very unlikely to switch to RISC-V. Is there any other concern on using JDK 11 on RISC-V specifically?

@adamretter
Copy link
Member

We could be using QEMU on x86_64 or aarch64 machines to test the functionality

We have a bunch of servers running KVM and QEMU, however in the past when we have tried to emulate RISC-V, we were able to get a Ubuntu installed, however we then could not get Docker running on RISC-V as that appeared unsupported, at least at that time under Ubuntu - has that changed now?

JDK 8 is not available on RISC-V, and I don't expect it to be ever

Okay understood.

Is there any other concern on using JDK 11 on RISC-V specifically?

In practice probably not. Just to say that we don't code for it or test with it yet. As RocksDB is a widely used library we aim for a lowest common denominator of both libc (or muslc) library ABI, and JLS version. This allows the widest audience possible to use the binaries of the library without having to compile it themselves.
Ideally we target Java 1.8, but if that isn't supported on RISC-V, then a newer version could be used to produce RISC-V binaries.

@luhenry
Copy link
Contributor Author

luhenry commented Nov 8, 2023

We have a bunch of servers running KVM and QEMU, however in the past when we have tried to emulate RISC-V, we were able to get a Ubuntu installed, however we then could not get Docker running on RISC-V as that appeared unsupported, at least at that time under Ubuntu - has that changed now?

I've been running QEMU on Ubuntu on Github Actions CI on multiple projects. https://github.com/uraimo/run-on-arch-action/ is an example of how to do it on GHA. I'm happy to reproduce something similar on CircleCI.

@adamretter
Copy link
Member

@luhenry Okay I now have a Ubuntu 22.04 emulated RISC-V 64 VM running on KVM/QEMU on Ubuntu 22.04 x86_64, and I have managed to install Docker 24.0.5 on there. See - https://blog.adamretter.org.uk/ubuntu-riscv64-guest-on-kvm/

I am surprised by quite how slow the emulation is though!

I will now try and test out your Docker image PR...

@adamretter
Copy link
Member

@luhenry I have now tested this and it appears to work well, thank you very much for your PR, and sorry for the delay in being able to test this.

@adamretter adamretter merged commit 338aad0 into evolvedbinary:main Dec 11, 2023
@adamretter
Copy link
Member

I have now also pushed the image to Docker Hub - evolvedbinary/rocksjava:ubuntu20_riscv64-be

@luhenry
Copy link
Contributor Author

luhenry commented Dec 11, 2023

I am surprised by quite how slow the emulation is though!

It's not fast indeed :) Usually around 10x slower per-core. It's important to use multithreading/multiprocessing as much as you can just to hide that slowness

@luhenry
Copy link
Contributor Author

luhenry commented Dec 11, 2023

Thank you for merging the PR, I'll submit the follow up changes for rocksdb in the coming days.

facebook-github-bot pushed a commit to facebook/rocksdb that referenced this pull request Dec 14, 2023
Summary:
Following evolvedbinary/docker-rocksjava#2, we can now build rocksdb on riscv64.

I've verified this works as expected with `make rocksdbjavastaticdockerriscv64`.

Also fixes #10500 #11994

Pull Request resolved: #12139

Reviewed By: jaykorean

Differential Revision: D52128098

Pulled By: akankshamahajan15

fbshipit-source-id: 706d36a3f8a9e990b76f426bc450937a0cd1a537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants