Skip to content

Reland "[windows][toolchain] Enable builtins and sanitizers" #78630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

weliveindetail
Copy link
Member

Profile and builtin libraries from compiler-rt are static and do not require a link step. We can enable them in our CMake caches and we can build and install them in the unified LLVM build step.

Sanitizers contain dynamic libraries. We need the target-specific Visual Studio shell to perform the link step. This patch adds extra build steps in build.ps1 that reconfigure, build and install the target-specific nested build-trees in the LLVM runtimes directory through their respective Visual Studio shells.

My previous attempt in #77770 failed, because it didn't handle cross-compile cases properly. Since the just-built compiler cannot be executed in a cross-build, the LLVM build step was falling back to the host compiler. On Windows this resolves to MSVC, which doesn't work.

This PR aims to fix that and reland the feature. We have to override the compiler in each nested build-tree manually, to select the stage-1 Clang compiler that we use for the LLVM build step itself.

armv7-unknown-linux-androideabi)
endif()

set(LLVM_BUILTIN_TARGETS ${default_targets} CACHE STRING "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally speaking, mutating lists is generally considered bad form with CMake. I think that it might be better to just have the duplication here which would also make it easier to follow.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in Windows-x86_64. If it works in CI, I will replicate it here.

Comment on lines +84 to +85
set(BUILTINS_${target}_CMAKE_C_COMPILER_TARGET "${target}21" CACHE STRING "")
set(BUILTINS_${target}_CMAKE_CXX_COMPILER_TARGET "${target}21" CACHE STRING "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about re-using ${BUILTINS_${target}_CMAKE_ANDROID_API} instead of replicating 21 here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added explicit CMAKE_ANDROID_API with a note to keep it in sync with build.ps1 (which is unfortunate, but I don't see how to do it better)

@@ -1,3 +1,5 @@
// UNSUPPORTED: OS=windows-msvc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this due to the missing functionality in the driver?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that was the case back in November. I had found a similar case when trying to build the toolchain with sanitizers enabled here: weliveindetail@5551c86

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you file an issue to track re-enabling the set of tests that have been disabled here? The UNSUPPORTED: OS=windows-msvc means that we cannot identify when the test starts to work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, let's see if they can be XFAIL as well

@weliveindetail
Copy link
Member Author

@swift-ci please test Windows platform

@weliveindetail
Copy link
Member Author

The build failed while testing sourcekit-lsp for arch x86_64. This is the very last build step in build.ps1.

compiler-rt changes can have a wide range of side-effects, but this really seems unrelated:

sourcekit-lsp\Tests\SourceKitLSPTests\PullDiagnosticsTests.swift:466:
  error: PullDiagnosticsTests.testDiagnosticsWhenOpeningProjectFromSymlink : XCTUnwrap failed:
    expected non-nil value of type "Diagnostic" - 
Test Case 'PullDiagnosticsTests.testDiagnosticsWhenOpeningProjectFromSymlink' failed (3.248 seconds)

@ahoppen
Copy link
Member

ahoppen commented Jan 15, 2025

That looks like a nondeterministic failure I’m seeing for the first time. I’m starting to understand the problem and am working on a solution. Re-running CI should resolve the failure for now.

@ahoppen
Copy link
Member

ahoppen commented Jan 15, 2025

@swift-ci Please test Windows

…oss-compilation

LLVM Runtimes contain various compiler-rt components like builtins, crt, profile and sanitizer libraries, etc.
The LLVM project spawns a nested build-tree for each configured target architecture.
For Windows we aim to build builtins, profile and sanitizer for x86_64, aarch64 and i686.

In regular builds LLVM selects the just-built Clang compiler for cross-compilation in the nested build-trees.
In cross-builds, it falls back to the default host compiler. On Windows this resolves to MSVC, which doesn't work.

We have to override the compiler in each nested build-tree manually, to select the stage-1 Clang compiler,
that we use for the LLVM Project build itself.
@weliveindetail weliveindetail force-pushed the reland-windows-toolchain-sanitizers branch from 8cc55b6 to 16d8eff Compare January 21, 2025 12:59
@weliveindetail
Copy link
Member Author

@swift-ci Please test Windows

@weliveindetail
Copy link
Member Author

That worked. Let's make sure we can turn UNSUPPORTED into XFAIL and check the other platforms as well.

@swift-ci Please test

@weliveindetail
Copy link
Member Author

The Linux bot failed during checkout. Previous PR builds are running fine, so it might be a temporary problem.

@weliveindetail
Copy link
Member Author

It's too hard to build Android SDKs that way. Let's do for standalone builds everywhere: #78861

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants