Skip to content

Set up CI to do split legacy build for macOS 10.9-10.12 #1331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 2, 2022

Conversation

ychin
Copy link
Member

@ychin ychin commented Oct 31, 2022

SDK for the new macOS 13 (released as part of Xcode 14.1) increased the minimum deployed OS from 10.9 to 10.13. Since we still want to support a minimum OS requirement of 10.9, while building against the latest SDK to get the most up-to-date features, we have to split the build process to build two separate binaries. A regular MacVim that targets the macOS 13 SDK (with min OS target 10.13), and a "legacy" MacVim that targets the macOS 12 SDK (with min OS target 10.9).

Change the GitHub Action CI config to have two separate matrix entries that publish, with one having a "legacy" flag set, which will use the older version of Xcode and the correct env vars set. It will also only build for x86-64 (no arm64) since currently all Apple Silicon hardware can run the latest versions of macOS and would have no need to run legacy builds.

Also, fix some scripting issues:

  • fix Python 2 library path set incorrectly
  • Update Perl to 5.30 from 5.18, as 5.18 is no longer installed in macOS 13. We could potentially change Perl to use the Homebrew version instead similar to how Python and other scripting languages are done but for now this works fine (except it won't work on older macOS versions). Perl is a rarely used language for Vim plugins anyway.

See #1288

@ychin
Copy link
Member Author

ychin commented Oct 31, 2022

Will probably let this sit a while until Xcode 14.1 actually comes out (currently it's only in release candidate stage).

@ychin ychin mentioned this pull request Nov 1, 2022
Comment on lines 282 to 291
if ${{ matrix.legacy == true }}; then
mv src/MacVim/build/Release/MacVim.dmg src/MacVim/build/Release/MacVim_10.9.dmg
fi

- name: Upload MacVim (legacy) image
if: startsWith(github.ref, 'refs/tags/') && matrix.publish && matrix.legacy
uses: actions/upload-artifact@v3
with:
name: MacVim_10.9.dmg
path: src/MacVim/build/Release/MacVim_10.9.dmg
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need to rename to MacVim_10.9.dmg at test step?
I think can set name to MacVim_10.9.dmg and path to src/MacVim/build/Release/MacVim.dmg at upload step.
Additionally, can aggregate upload steps as setting name to MacVim${{ matrix.legacy ? "_10.9" : "" }}.dmg
(but I don't make sure it works); does it feel complicated?

Copy link
Member Author

@ychin ychin Nov 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need to rename to MacVim_10.9.dmg at test step?
I think can set name to MacVim_10.9.dmg and path to src/MacVim/build/Release/MacVim.dmg at upload step.

The name part of upload-artifact only defines how the artifact is named in GitHub Actions. Once you download the file (either through the web, or the gh command-line tool), it uses the original file name. So basically if we didn't do the rename, the downloaded file will still show up as MacVim.dmg. I couldn't find a way to remap the file name in the available parameters. I personally have to sign / notarize the app off CI anyway (since I never set up secrets sharing with GitHub Actions, which I'm not sure if I want to do), so I can still rename them when I do that, but it feels like the output of the CI build step should result in a file called MacVim_10.9.dmg.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, can aggregate upload steps as setting name to MacVim${{ matrix.legacy ? "_10.9" : "" }}.dmg
(but I don't make sure it works); does it feel complicated?

Huh. I can give it a try. I think I couldn't get the syntax to quite work out but I don't think I tried the conditional that you showed. I do dislike how we have to have two separate step for uploading artifacts. I'm not sure if it will work though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name part of upload-artifact only defines how the artifact is named in GitHub Actions. Once you download the file (either through the web, or the gh command-line tool), it uses the original file name. So basically if we didn't do the rename, the downloaded file will still show up as MacVim.dmg. I couldn't find a way to remap the file name in the available parameters.

I see, thank you.

Additionally, can aggregate upload steps as setting name to MacVim${{ matrix.legacy ? "_10.9" : "" }}.dmg (but I don't make sure it works); does it feel complicated?

Huh. I can give it a try. I think I couldn't get the syntax to quite work out but I don't think I tried the conditional that you showed. I do dislike how we have to have two separate step for uploading artifacts. I'm not sure if it will work though.

Sorry, GitHub Actions does not compatible with ternary operator, but I confirmed can use MacVim${{ matrix.legacy && '_10.9' || '' }}.dmg.

Copy link
Member Author

@ychin ychin Nov 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should just make a new matrix variable called matrix.file_postfix. This way we just define it in one place instead of having to paste "_10.9" all over the place, and less reliance on complicated template logic.

Copy link
Member Author

@ychin ychin Nov 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright just pushed an update to use a matrix variable instead, so it's easier to modify later. Here's an example run (tests were removed to make it run faster): https://github.com/ychin/macvim/actions/runs/3366660777

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you have more comments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel the variable matrix.legacy is less necessary due to matrix.publish_postfix.

Copy link
Member Author

@ychin ychin Nov 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I see what you mean. Functionally yes, but I think I prefer keeping them separate for now, as they have different meanings. For example, let's say I actually want to publish MacVim as MacVim_10.13.dmg instead I can just add publish_postfix to the matrix and it would work, without touching the legacy parts.

@ychin ychin added this to the Release 175 milestone Nov 1, 2022
@ychin ychin added Infrastructure Non-app infrastructure issues, e.g. CI Legacy macOS Issues related to legacy OS support (macOS 10.9 - 10.12) labels Nov 1, 2022
@ychin
Copy link
Member Author

ychin commented Nov 1, 2022

Also see #1332 which upgrades to Sparkle 2 from Sparkle 1. Because Sparkle 2 requires macOS 10.13, we will be enabling it only in normal builds, with legacy builds still using Sparkle 1.

@ychin ychin force-pushed the split-ci-macos10.13-10.9-legacy branch 3 times, most recently from 34fa914 to 5846910 Compare November 1, 2022 05:20
@ychin
Copy link
Member Author

ychin commented Nov 1, 2022

Ugh, a new Python version (e.g. 3.11) got released recently and just got installed to GitHub's CI runners, so it's breaking the build, but only when using 14.0 but not 14.1 (which seems to work as configure can find the correct 3.10 path, somehow). Will need to find a way to fix this.

@ychin ychin force-pushed the split-ci-macos10.13-10.9-legacy branch from 8477c66 to 5dfda82 Compare November 1, 2022 06:09
@ichizok
Copy link
Member

ichizok commented Nov 1, 2022

Current Homebrew's default python is 3.10, thus need to specify version as brew install python@3.11.

@ychin
Copy link
Member Author

ychin commented Nov 1, 2022

Current Homebrew's default python is 3.10, thus need to specify version as brew install python@3.11.

Actually looking more into it I think there's some issues with GitHub CI's setup. It seems to be inconsistent, so I'm guessing that they are doing a rolling upgrade of their VMs which makes it really frustrating to test. But essentially there are some VMs (see this run) that have Python 3.11 installed, but they are installed not using Homebrew, but to /Library/Frameworks instead (the default Python installation folder). Incidentally the build then hit the HAVE_DUP issue that you filed at vim/vim#11484. Either way this seems wrong, and I'm looking into forcing the CI to use the Homebrew version for consistency (since that's the one we set the dynamic linkage to anyway), probably just need a brew link --overwrite.

@ychin ychin force-pushed the split-ci-macos10.13-10.9-legacy branch 6 times, most recently from e4e63d9 to 1e80ead Compare November 1, 2022 07:59
SDK for the new macOS 13 (released as part of Xcode 14.1) increased the
minimum deployed OS from 10.9 to 10.13. Since we still want to support
a minimum OS requirement of 10.9, while building against the latest SDK
to get the most up-to-date features, we have to split the build process
to build two separate binaries. A regular MacVim that targets the macOS
13 SDK (with min OS target 10.13), and a "legacy" MacVim that targets
the macOS 12 SDK (with min OS target 10.9).

Change the GitHub Action CI config to have two separate matrix entries
that publish, with one having a "legacy" flag set, which will use the
older version of Xcode and the correct env vars set. It will also only
build for x86-64 (no arm64) since currently all Apple Silicon hardware
can run the latest versions of macOS and would have no need to run
legacy builds.

Also, fix some scripting issues:

- fix Python 2 library path set incorrectly
- Update Perl to 5.30 from 5.18, as 5.18 is no longer installed in macOS
  13. We could potentially change Perl to use the Homebrew version
  instead similar to how Python and other scripting languages are done
  but for now this works fine (except it won't work on older macOS
  versions). Perl is a rarely used language for Vim plugins.

See macvim-dev#1288
@ychin ychin force-pushed the split-ci-macos10.13-10.9-legacy branch from 1e80ead to a988b6e Compare November 2, 2022 11:24
@ychin ychin merged commit 88dc6f7 into macvim-dev:master Nov 2, 2022
@ychin ychin deleted the split-ci-macos10.13-10.9-legacy branch November 2, 2022 22:15
@ychin
Copy link
Member Author

ychin commented Nov 2, 2022

This is now merged, since Xcode 14.1 is now officially out (it's the same build as Release Candidate 2). Seems like GitHub's runners are still using RC1, but I don't know of any features / fixes that we need in RC2 anyway.

ychin added a commit that referenced this pull request Feb 7, 2023
Updated to Vim 9.0.1276

Features
====================

Dictionary lookup
--------------------

You can now use Force Touch or Cmd-Ctrl-D to look up definitions of word
under the cursor (or selected text in visual mode). This will also
preview URLs, and support data types such as phone numbers and
addresses. #1312 #1313

This feature can also be invoked programmatically from VimScript (see
`:h macvim-lookup`). #1315

Tool bar / Touch Bar / menu icons
--------------------

You can now use SF Symbols for Tool bar and Touch Bar icons, including
using different symbol styles such as "palette" or "multicolor". Menu
items can now also use the `icon=` syntax to specify icons as well. See
`:help macvim-toolbar-icon` for details. #1329

The default tool bar also has updated icons to look similar to SF
Symbols used by newer macOS versions. #1214 by @sfsam

Window management actions
--------------------

There are new `macaction`'s for managing the MacVim window. The new
`zoomLeft`/`zoomRight` actions allow you to pin the window to the
left/right of the screen, and there are also new actions for interacting
with Stage Manager (requires macOS 13+). See `:h macvim-actions` for
details. #1330

Pre-release updates / Sparkle 2
--------------------

MacVim now supports pre-release software builds. It's sometimes hard for
us to release frequent updates due to the desire to pick a stable
upstream Vim version, needing to test the release on multiple OS
versions, making sure there aren't half-complete or buggy features, and
other reasons.

This new feature now allows us to push pre-release beta builds out in a
more frequent fashion, which could be useful if there are particular
features or fixes that you would like to try out before the next
official release. Pre-release builds will be released depending on bug
fixes and features instead of a fixed cadence. Do note that these
pre-release builds may not be as well-validated and may have half-baked
features.

If you are using the built-in auto-updater to update MacVim, you can
turn this on by going to Advanced settings pane, and enable "Enable
pre-release software updates".

This feature is only available for macOS 10.13 or above.

The auto-updater has also been updated from Sparkle 1.27.1 to 2.3.0 for
10.13+ builds. Legacy (10.9-10.12) builds are still using Sparkle 1.

See #1332.

New Vim features
--------------------

New `smoothscroll` option allows you to scroll through a long wrapped
line (using Ctrl-E or mouse wheel) without immediately jumping to the
next line. (v9.0.0640)

`splitscroll` option has been renamed `splitkeep`, with more flexibility
than before. (v9.0.0647)

Sound playback on macOS is now supported. You can use `has('sound')` to
check. See `help sound` for details. (v9.0.0694)

Terminals now support `:confirm` for `:q`, etc, which also means
MacVim's Cmd-W will work properly for terminal windows. (v9.0.0710)

Virtual text had numerous bugs fixed.

General
====================

Legacy build for 10.9 - 10.12
--------------------

Per a previous announcement (#1271), the default MacVim binary will now
require macOS 10.13 or above. Users of macOS 10.9 - 10.12 can use a
separate "legacy" build which will still be supported. The legacy binary
will still have the latest versions of Vim and be supported, but may not
have all the latest features (e.g. pre-release builds).

If you are using the auto-updater (Sparkle) to update MacVim, it should
"just work" and find the best version for you. If you are downloading
MacVim from the website, there is also a link to download the legacy
version marked for 10.9+ as well. If you download the normal binary
marked for 10.13+ from the website, it won't work on these older macOS
versions.

See #1331.

Fixes
====================

CoreText Renderer clipping and rendering bugs
--------------------

Unicode characters with multiple composing characters (e.g. "x⃗") will
now render correctly. #1172

Texts (e.g. Tibetan, Zalgo texts) that are taller than the line height
will no longer be clipped inappropriately. You can use a new setting
`MMRendererClipToRow` to re-enable clipping if the tall texts are
distracting. #995 / #1356

Tab crash
--------------------

Fixed a crash when opening new tabs that seems to only occur in macOS 13
Ventura. #1333

Other bugs
--------------------

- Fixed non-native full screen not working well with the notch on newer
  MacBook's when set to not show menu bar. You can also use
  `MMNonNativeFullScreenSafeAreaBehavior` to force MacVim to use the
  notch area as well if you don't mind some content being obscured. Note
  that the previous release also claimed it fixed this, but because the
  binary was built against an old macOS SDK (Big Sur), the fix did not
  work in the binary release. #1261
- Allow "Open untitled window: never" and "After last window closes:
  Quit MacVim" to be set together again. Added safeguards to make sure
  doing so won't immediately close the app. #1338
- Edit.Cut / Copy menu items will now be properly disabled when there
  isn't selected text. #1308
- Fixed potential `:emenu` crash when the menu is associated with an
  action in a non-valid mode. #1305
- Fixed bug where just bringing up the right-click (or the
  MacVim→Services) menu would somehow copy the selected texts to the
  system clipboard. #1300
- Fixed a Japanese input method bug where using left/right arrow to move
  to a different section of the input text would previously result in
  the candidate list not showing up at the correct position. #1312
- Fix non-CoreText renderer not handling text styles like strikethrough
  correctly (note: this renderer has been deprecated for a while and you
  should not use it). #1296
- This release uses an older sh/bash syntax file because the latest one
  in Vim has a bug. #1358

Misc
====================

New settings:

- "No drop shadows" (Appearance). #1301
- "Treat Ctrl-click as right-click" (Input) (#1326). This was previously
  configurable via command-line, but now also possible in the settings
  pane under the new "Input" category.

"About MacVim" now reports the version number in a clearer way with
clearly specified release number vs Vim version.

Known Issues
====================

Printing
--------------------

Printing using File→Print or `:hardcopy` is currently not working under
macOS 13 Ventura due to its removal of PostScript support in the Preview
app. This will be fixed in a later release. See the issue for
workarounds. #1347

Scripting
====================

- Scripting languages versions:
    - Perl is now built against 5.30, up from 5.18.
    - Ruby is now built against 3.2, up from 3.1.

Compatibility
====================

Requires macOS 10.9 or above. (10.9 - 10.12 requires downloading a
separate legacy build)

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.30
- Python2 2.7
- Python3 3.10
- Ruby 3.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Non-app infrastructure issues, e.g. CI Legacy macOS Issues related to legacy OS support (macOS 10.9 - 10.12)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants