Remove RWC runner and related infrastructure #55187
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've previously talked about this one; this infra is a pain to maintain, isn't visible to the public (we have to copy/paste and self-censor the internal repos we show), and we can't import tests into it anymore post-modules (as the mechanism used to "instrument"
tsc
was removed). I've already removed all of the other "special" runners (user, docker, DT, test262), so after this PR we'll only have the tests that exist fully in this repo, and those in https://github.com/microsoft/TypeScript-error-deltas.A downside is that there may be things tested in RWC that aren't tested in our in-repo test suite or in user/topXYZ tests.
Notably, RWC baselines emit while our other external test cases do not, but from being the one to update those baselines when
main
fails, it doesn't really seem like we actually get much out of it as the emit changes are always very clearly reflected in tests.Additionally, RWC is one of the only places with existing errors, which do provide a mechanism to observe how error messages are changing. It's likely that user/topXYZ tests don't contain any errors except the new ones we add in a release. But it's a double edged sword; the reason the RWC projects have errors is because they're all 3+ years old and are never updated. Most of them contain 5.0 option deprecation errors!
I'm not sure we're "ready" for this, but I do want to discuss it. I'm finding it increasingly annoying to fix RWC up; we've broken the test runner without noticing multiple times this year, its ability to push PRs to the internal repo is currently broken, and often its ability to post github replies also breaks. It's just annoying.