[mono] MiniJIT System.Formats.Nrbf.Tests getting killed

[matouskozak]

The System.Formats.Nrbf.Tests are getting ocassinally killed on CI. Affecting mostly MonoJIT linux_x64_release_mono_minijit_librariestests job and occasionally MonoInterpreterlinux_x64_release_mono_interpreter_librariestests.

7 failures in the last 30 days as per

Log:

Starting:    System.Formats.Nrbf.Tests (parallel test collections = on [2 threads], stop on fail = off)
./RunTests.sh: line 176:    21 Killed                  "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Formats.Nrbf.Tests.runtimeconfig.json --depsfile System.Formats.Nrbf.Tests.deps.json xunit.console.dll System.Formats.Nrbf.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
cat: /proc/sys/kernel/coredump_filter: No such file or directory
dmesg: read kernel buffer failed: Operation not permitted
+ export _commandExitCode=137
+ python /root/helix/work/correlation/reporter/run.py https://dev.azure.com/dnceng-public/ public 21132138 eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IjdSd2F5dmlYRHFoZnN6MTZSNmxPbXNXWWxTQSJ9.eyJuYW1laWQiOiJjNzczZjJjMi01MTIwLTQyMDctYWZlMi1hZmFmMzVhOGJjMGEiLCJzY3AiOiJhcHBfdG9rZW4iLCJhdWkiOiJiMWU0ZDg3Ni0yZjIwLTQ1NjctODgyMi05OWQ0YTNhZGYxNGEiLCJzaWQiOiI4ODdhMTkyYy0yNTQ3LTQ4YTUtYTEwMS0yOTc4ZmJiNzI0ZWIiLCJCdWlsZElkIjoiY2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5OzgxNTI0OSIsIkRlZklkIjoiMTI5Iiwiam9icmVmIjoiNDgyZjk0ZWQtYjFjZi00Mjk4LWJmMmMtMDY3NGIyNTRjNzgxOjFkMTA2YTY1LTI2OWItNTI4Mi1lZjMxLWUzMzc4YTVhMTNmMiIsInBwaWQiOiJ2c3RmczovLy9CdWlsZC9CdWlsZC84MTUyNDkiLCJvcmNoaWQiOiI0ODJmOTRlZC1iMWNmLTQyOTgtYmYyYy0wNjc0YjI1NGM3ODEuYnVpbGQuYnVpbGRfbGludXhfeDY0X3JlbGVhc2VfbW9ub19pbnRlcnByZXRlcl9saWJyYXJpZXN0ZXN0cy5fX2RlZmF1bHQiLCJyZXBvSWRzIjoiIiwiaXNzIjoiYXBwLnZzdG9rZW4udmlzdWFsc3R1ZGlvLmNvbSIsImF1ZCI6ImFwcC52c3Rva2VuLnZpc3VhbHN0dWRpby5jb218dnNvOjZmY2M5MmU1LTczYTctNGY4OC04ZDEzLWQ5MDQ1YjQ1ZmIyNyIsIm5iZiI6MTcyNzAzNDk0MSwiZXhwIjoxNzI3MDY0OTQxfQ.om2rr6irEACnRYKhRuUB2ECACuJKmQKGl3wctZ4IVZo8LtGCigyhva_yyKjvZRX5n3xl6SQgNlv01Cl2uKRaVGws1zizT9WsUwKKkLm390V5dSHg3eb1VyDHMM2LTO9mM5Ve4dfn4XhgeUid4xkpH-V6vXrIMu6E3hW-B30oH7xH1kdj8Ftw1sYuL5rsNJRAXjnd3hZgODSSXgK3_dij7BqeH-AZZBqv3ErGWdUYuVd0BRv4M8P51EJPjp8u4CC1jP_rjKRMw8nZm4VvCEL7CyR4X4SbVNqp5KP2F8Kpjnpus_0rMPNyLnyE2KOI1totQRV_WmguZZq7w9V8KMb5FQ
/root/helix/work/workitem/e
----- end Sun 22 Sep 2024 08:43:19 PM UTC ----- exit code 137 ----------------------------------------------------------
exit code 137 means SIGKILL Killed either due to out of memory/resources (see /var/log/messages) or by explicit kill.

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["System.Formats.Nrbf.Tests", "exit code 137 means SIGKILL Killed either due to out of memory/resources (see /var/log/messages) or by explicit kill"],
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Report

Build	Definition	Test	Pull Request
998368	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
998126	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
996982	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
996954	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
996279	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
995599	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
995061	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
993463	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
986560	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
986493	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
985306	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
984635	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution	#113576
984583	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
984385	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
983588	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
982817	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
979574	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
976902	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
973408	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
973385	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
972238	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
971108	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
970793	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
970536	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
967749	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution
966471	dotnet/runtime	System.Formats.Nrbf.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	8	26

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=825800
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 10/3/2024 4:08:45 PM UTC

[dotnet-policy-service]

Tagging subscribers to this area: @lambdageek, @steveisok
See info in area-owners.md if you want to be subscribed.

[lambdageek]

There are some interesting "attack tests" that do things with cyclic graphs and potentially large memory usage that might be behaving pathologically on Mono. (for example due to conservative stack scanning)

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Formats.Nrbf/tests/AttackTests.cs

[matouskozak]

No longer failing. Unclear what fix it as the test was a bit flaky.

[matouskozak]

Failed again, reopening https://dev.azure.com/dnceng-public/public/_build/results?buildId=914493&view=results