Integrate system operations into setup-ruby

[MSP-Greg]

Thinking about what 's needed for some of the repos I've worked in, I'm wondering if integrating some of the MSYS2 related things I've done in msys2-action and actions-ruby could be integrated here.

For instance, inputs could be added like:

with:

  pkg: ragel
  macOS:
  mingw:
  ubuntu:

EDIT: for instance, all the code needed for ragel

      - name: ubuntu & macos - install ragel
        if:   startsWith(matrix.os, 'ubuntu') || startsWith(matrix.os, 'macos')
        run:  |
          if [ "${{ matrix.os }}" == "macos" ]; then
            brew install ragel
          else
            sudo apt-get -qy install ragel
          fi

'ragel' is a package with a cross platform name, so code could install it based on the platform used. For packages without common names, the OS specific names could be used in the respective inputs.

I recall somewhere seeing something like 'brew update all' or a similar command, which for Windows/MSYS2 might be the equivalent of updating the gcc tools. Just some general thoughts...

[eregon]

I've been thinking about this too, it's quite verbose in GitHub Actions to install a package across platforms, even if it has the same name.

I'm not sure this fits in this action though, wouldn't it fit better in some install-packages (or setup-packages maybe) action?

- uses: unknown/install-packages
  with:
    packages: ragel openssl

A quick search shows https://github.com/marketplace/actions/get-package which does something a bit like that.

[eregon]

I think pkg is the name of some package manager, so I think it's better to use the full name.

[MSP-Greg]

I'm not sure this fits in this action though

1. Using two different actions means two steps, two repos to download, two fully packaged js files, each with very similar dependency code.

2. If no packages are needed, it's no different than what's used now.

Maybe look at it as the one action needed to configure Ruby and the system for the CI task?

Also, I've had a few workflows with a lot of steps, so the idea of keeping them to a minimum is good.

It's not something I fell strongly about. Really, all people care about is the equivalent of rake test or any failing previous steps...

[eregon]

I think the overhead of using a separate action is very small in practice, probably less than 1 second.

I've made my mind, I think such an action would be useful in general, not just for Ruby and so I think it should be a separate action. This action is already not that simple anymore, I wouldn't want to add not-so-directly related features to it.

So, I think someone should create a install-packages or setup-packages action, which just needs to shell out to the respective package manager.

[eregon]

I forgot to mention, adding such feature in this action would also clearly no longer meet the efficiency claim, as installing packages can take far more than 5 seconds.

[MSP-Greg]

Okay by me. Maybe setup-ruby-packages?

I think making it clear that it's Ruby specific, and it would be helpful to have the action used after Ruby is setup/installed.

Also, as you mentioned, code expands. I think maybe the idea of a global 'packages' input will just cause problems.

Maybe have the following for allowable inputs:

- uses: ruby/setup-ruby-packages
  with:
    ubuntu: ragel openssl
    macos:  ragel openssl
    mingw:  ragel openssl
    msys2:
    mswin:  ragel openssl
    jruby:  ragel openssl

[eregon]

Why do you say it's Ruby-specific? Aren't system packages global and so could also be used e.g., for some Python library's CI?

For the names I'd suggest apt/brew/choco/pacman/etc, following the pattern of https://github.com/postmodern/ruby-install/blob/402cbdeaffb99d105ecd9b93c75828b32f0f382f/share/ruby-install/truffleruby/dependencies.txt

Then it's immediately clear by which package manager the package is installed, which is particularly useful if there are multiple package managers on that platform.
Of course the trade-off is then it might be less clear which package manager is used on which platform, but I think that's OK.

What does jruby: ragel openssl do?

[MSP-Greg]

What does jruby: ragel openssl do?

I just threw that in. I don't know if JRuby has a package concept or not.

Why do you say it's Ruby-specific?

First of all, because supporting generic package installation might be a support nightmare. We're Ruby people.

As to Ruby specific, the main reason I started msys2-action and then actions-ruby is OpenSSL. To build against Windows Ruby versions 2.2 thru head/master mingw & mswin takes four different packages. I help ruby/openssl, puma, & eventmachine, and all compile against OpenSSL. I didn't want workflow spaghetti code in all three.

Without having the Ruby binary installed first, the action has no way to know what version/platform of OpenSSL to install, unless we add another input. Also, having the Ruby binary installed makes things like RUBY_DESCRIPTION, RUBY_VERSION, the ABI version, etc, available in the action script.

I don't think I've mentioned it, but I don't think the RubyGems --user-install folder's bin directory is in PATH. Not common in CI, but I saw some discussion of making it the default in some (or all) situations. Finding it's location is much easier if Ruby exists. Not applicable to the package action, but somewhat an example...

[eregon]

Reopening to keep the discussion here, although I think this should be a separate action, not part of this action.

[eregon]

Re jruby: I don't think there is a package manager for JRuby.
If e.g. one wants to run mvn package for some Java project, I don't think that belongs to setup-packages.

First of all, because supporting generic package installation might be a support nightmare. We're Ruby people.

I think it's pretty simple if it just shells out the package manager and does nothing more.

Without having the Ruby binary installed first, the action has no way to know what version/platform of OpenSSL to install, unless we add another input.

Can the version be part of the package name? I think it is, e.g. libssl1.0-dev for apt.

It seems what you'd like is msys2: openssl to automatically find the correct version of openssl to install, as in
https://github.com/puma/puma/blob/adb6170b77ce4a4ea156f0198678444f61b39255/.github/workflows/ruby.yml#L81
I think that's too much magic.
AFAIK only openssl has this problem that the latest version is not compatible with older Rubies.

Also, does installing openssl even have any effect when using Ruby? Maybe on Windows?
I believe on non-Windows the C extension is compiled against the build-time openssl, and whatever openssl is available at runtime is ignored, it always use the same as the one from build time (as in ABI version, such as libssl.so.1.1, so typically the same absolute path at runtime).

[MSP-Greg]

I think that's too much magic.

Puma and ruby/openssl are using it, and EventMachine probably will when they update their CI. If the logic isn't available, it makes for very messy workflows. I suppose I could have a ruby-win31-openssl action...

[eregon]

What about the point that the runtime openssl is mostly ignored? Is that not true on Windows?

[MSP-Greg]

What about the point that the runtime openssl is mostly ignored? Is that not true on Windows?

You've just hit on another Windows concept, and part of the reason I always build with OpenSSL versions matching the version Ruby was built with.

Windows exe/dll files have a concept often referred to as 'manifests'. It involves xml embedded in exe or dll files that tell them where their support dll's are located. Using manifests does not affect PATH.

RI2 and ruby-loco (mingw & mswin) all have manifests. So, anything using OpenSSL dll's, like Puma, EM or ruby/openssl, will use the files defined by the manifest first, which are the files packaged with Ruby.

Note that when testing from build in ruby/ruby, the manifest doesn't exist, so the path resolution for the dll's is different. That is the reason some OpenSSL files need renaming in ruby/ruby, but not elsewhere.

Another reason why I test from install...

[eregon]

Thanks for the explanation. If I understand that right, it means it shouldn't be necessary to install openssl for puma, EM, etc, because it's already part of the shipped Ruby?

Why is openssl updated then as part of Puma's CI?
Is it purely to test against the latest available MSYS2 openssl package, but would otherwise not be needed?

[MSP-Greg]

because it's already part of the shipped Ruby?

Good thought, but unlike the builds you've created for Ubuntu & macOS, only the dll's are included with shipped Ruby.

Why is openssl updated then as part of Puma's CI?

Actually, to keep it smaller, the RI2 embedded MSYS2 installs don't have the OpenSSL package included.

[eregon]

Ah, so it's to install the openssl.h, etc headers, right.

The headers are typically pretty small in size, it sounds like shipping them with builds of Ruby on Windows would simplify that quite a bit for a probably very small increase in download size.

[MSP-Greg]

The actual package is about 4.77 MB. Some of that is man/html pages, some is shared/static, some the dll's already included in the Ruby package. It's about the same size as the Ruby mswin package, half the size of mingw.

It's not a bad idea, but we also don't want to to stray too far from what a local install would be. Setting up cert files & paths is one thing, adding dependent packages into the build package is another.

[MSP-Greg]

I should add that Ruby mingw builds have a history.

With mswin builds we can do whatever we want. As I think I've stated, I created it so ruby/ruby had a means of testing with it in std-lib/'default gem' repos.

[MSP-Greg]

Did some testing with the MSYS2 gcc issue. Below is gcc for recent Ruby releases, the suffix on the gcc version is the MSYS2 release:

gcc \ ruby	2.4	2.5	2.6	2.7
8.3.0-2	2.4.6	2.5.5	2.6.3
9.2.0-1	2.4.7	2.5.6	2.6.4
9.2.0-2	2.4.9	2.5.7	2.6.5	2.7.0

Note that all the current Rubies were built with the current MSYS2 gcc (9.2.0-2).

A particular gem was known to cause an issue, so I updated gcc to 9.2.0-2, then installed the gem on all Ruby versions shown above, except 2.7.0.

All of the Rubies using gcc 8.3.0-2 and 9.2.0-1 failed. So, it appears that being able to install either current gcc or 9.2.0-1 is the answer to supporting all Rubies & extension gems. JFYI, I originally tried installing current versions of json & psych, and both installed.

Re the 'setup-ruby-packages' idea, maybe it should just be separate *.js files by OS/platform? Have a shared/common runner that loads/runs whichever is needed?

[MSP-Greg]

Closing since 'off-topic'

See https://github.com/MSP-Greg/setup-ruby-pkgs

[eregon]

Great, thanks for creating that action.