Skip to content

[Support] Recycler: Enforce minimum allocation size #121425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

optimisan
Copy link
Contributor

Recycler uses reinterpret_cast to an internal structure of size 8.
Invalid write occurs if Recycler is used for objects with sizes less
than 8.

Copy link
Contributor Author

optimisan commented Jan 1, 2025

@optimisan optimisan force-pushed the users/Akshat-Oke/01-01-_support_recycler_enforce_minimum_allocation_size branch from 600fd4c to 4496358 Compare January 1, 2025 06:54
@optimisan optimisan changed the base branch from users/Akshat-Oke/12-16-_newpm_codegen_record_parameterized_machine_pass_names_to_pic to users/Akshat-Oke/12-19-_codegen_liveregmatrix_use_allocator_through_a_unique_ptr January 1, 2025 06:54
@optimisan optimisan force-pushed the users/Akshat-Oke/01-01-_support_recycler_enforce_minimum_allocation_size branch from 4496358 to ac8a44c Compare January 1, 2025 07:06
@optimisan optimisan marked this pull request as ready for review January 1, 2025 08:26
@llvmbot
Copy link
Member

llvmbot commented Jan 1, 2025

@llvm/pr-subscribers-llvm-support

Author: Akshat Oke (optimisan)

Changes

Recycler uses reinterpret_cast to an internal structure of size 8.
Invalid write occurs if Recycler is used for objects with sizes less
than 8.


Full diff: https://github.com/llvm/llvm-project/pull/121425.diff

3 Files Affected:

  • (modified) llvm/include/llvm/Support/Recycler.h (+2)
  • (modified) llvm/unittests/Support/CMakeLists.txt (+1)
  • (added) llvm/unittests/Support/RecyclerTest.cpp (+46)
diff --git a/llvm/include/llvm/Support/Recycler.h b/llvm/include/llvm/Support/Recycler.h
index bbd9ae321ae30c..8cc882ea5fa058 100644
--- a/llvm/include/llvm/Support/Recycler.h
+++ b/llvm/include/llvm/Support/Recycler.h
@@ -85,6 +85,8 @@ class Recycler {
                   "Recycler allocation alignment is less than object align!");
     static_assert(sizeof(SubClass) <= Size,
                   "Recycler allocation size is less than object size!");
+    static_assert(Size >= sizeof(FreeNode) &&
+                  "Recycler size must be atleast 8");
     return FreeList ? reinterpret_cast<SubClass *>(pop_val())
                     : static_cast<SubClass *>(Allocator.Allocate(Size, Align));
   }
diff --git a/llvm/unittests/Support/CMakeLists.txt b/llvm/unittests/Support/CMakeLists.txt
index d64f89847aa8e7..6de81658264420 100644
--- a/llvm/unittests/Support/CMakeLists.txt
+++ b/llvm/unittests/Support/CMakeLists.txt
@@ -69,6 +69,7 @@ add_llvm_unittest(SupportTests
   PerThreadBumpPtrAllocatorTest.cpp
   ProcessTest.cpp
   ProgramTest.cpp
+  RecyclerTest.cpp
   RegexTest.cpp
   ReverseIterationTest.cpp
   ReplaceFileTest.cpp
diff --git a/llvm/unittests/Support/RecyclerTest.cpp b/llvm/unittests/Support/RecyclerTest.cpp
new file mode 100644
index 00000000000000..8cd763c0b83f8a
--- /dev/null
+++ b/llvm/unittests/Support/RecyclerTest.cpp
@@ -0,0 +1,46 @@
+//===--- unittest/Support/RecyclerTest.cpp --------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/Recycler.h"
+#include "llvm/Support/AllocatorBase.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+namespace {
+
+struct Object1 {
+  char Data[1];
+};
+
+class DecoratedMallocAllocator : public MallocAllocator {
+public:
+  int DeallocCount = 0;
+
+  template <typename T> void Deallocate(T *Ptr) {
+    DeallocCount++;
+    MallocAllocator::Deallocate(Ptr);
+  }
+};
+
+TEST(RecyclerTest, RecycleAllocation) {
+  DecoratedMallocAllocator Allocator;
+  // Recycler needs size to be atleast 8 bytes.
+  Recycler<Object1, 8, 8> R;
+  Object1 *A1 = R.Allocate(Allocator);
+  Object1 *A2 = R.Allocate(Allocator);
+  R.Deallocate(Allocator, A2);
+  Object1 *A3 = R.Allocate(Allocator);
+  EXPECT_EQ(A2, A3); // reuse the deallocated object.
+  R.Deallocate(Allocator, A1);
+  R.Deallocate(Allocator, A3);
+  R.clear(Allocator); // Should deallocate A1 and A3.
+  EXPECT_EQ(Allocator.DeallocCount, 2);
+}
+
+} // end anonymous namespace

Base automatically changed from users/Akshat-Oke/12-19-_codegen_liveregmatrix_use_allocator_through_a_unique_ptr to main January 1, 2025 09:24
Recycler uses reinterpret_cast to an internal structure of size 8.
Invalid write occurs if Recycler is used for objects with sizes less
than 8.
@optimisan optimisan force-pushed the users/Akshat-Oke/01-01-_support_recycler_enforce_minimum_allocation_size branch from ac8a44c to 5e6aefd Compare January 1, 2025 11:43
R.clear(Allocator); // Should deallocate A1 and A3.
EXPECT_EQ(Allocator.DeallocCount, 2);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to test the static_assert fails in the < 8 case?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think we have any infrastructure in LLVM for "this should fail to compile". If it were SFINAE'd then we could static_assert that something fails to compile, but static_asserts aren't themselves SFINAE, so it'd mean adding more metaprogramming infrastructure that's presumably not otherwise needed (& wouldn't test the static_assert itself, but the SFINAE trap)

Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com>
Copy link
Contributor Author

optimisan commented Jan 6, 2025

Merge activity

  • Jan 6, 1:22 AM EST: A user started a stack merge that includes this pull request via Graphite.
  • Jan 6, 1:23 AM EST: A user merged this pull request with Graphite.

@optimisan optimisan merged commit 34e8aff into main Jan 6, 2025
8 checks passed
@optimisan optimisan deleted the users/Akshat-Oke/01-01-_support_recycler_enforce_minimum_allocation_size branch January 6, 2025 06:23
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jan 6, 2025

LLVM Buildbot has detected a new failure on builder llvm-clang-aarch64-darwin running on doug-worker-4 while building llvm at step 2 "checkout".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/190/builds/12239

Here is the relevant piece of the build log for the reference
Step 2 (checkout) failure: update (failure)
git version 2.39.3 (Apple Git-146)
fatal: unable to access 'https://github.com/llvm/llvm-project.git/': Could not resolve host: github.com
fatal: unable to access 'https://github.com/llvm/llvm-project.git/': Could not resolve host: github.com

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jan 6, 2025

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux-fast running on sanitizer-buildbot4 while building llvm at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/169/builds/7088

Here is the relevant piece of the build log for the reference
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure)
...
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 88102 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
FAIL: LLVM-Unit :: Support/./SupportTests/135/174 (78782 of 88102)
******************** TEST 'LLVM-Unit :: Support/./SupportTests/135/174' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/unittests/Support/./SupportTests-LLVM-Unit-2979769-135-174.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=174 GTEST_SHARD_INDEX=135 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/unittests/Support/./SupportTests
--

Note: This is test shard 136 of 174.
[==========] Running 8 tests from 8 test suites.
[----------] Global test environment set-up.
[----------] 1 test from CastingTest
[ RUN      ] CastingTest.isa_and_nonnull
Classof: 0x5d8b472373e0
Classof: 0x5d8b472373e0
[       OK ] CastingTest.isa_and_nonnull (0 ms)
[----------] 1 test from CastingTest (0 ms total)

[----------] 1 test from EndianStream
[ RUN      ] EndianStream.WriteArrayLE
[       OK ] EndianStream.WriteArrayLE (0 ms)
[----------] 1 test from EndianStream (0 ms total)

[----------] 1 test from HashBuilderTest/3, where TypeParam = <type>
[ RUN      ] HashBuilderTest/3.HashUserDefinedStruct
[       OK ] HashBuilderTest/3.HashUserDefinedStruct (0 ms)
[----------] 1 test from HashBuilderTest/3 (0 ms total)

[----------] 1 test from OverflowTest/0, where TypeParam = <type>
[ RUN      ] OverflowTest/0.MulNoOverflowToMax
[       OK ] OverflowTest/0.MulNoOverflowToMax (0 ms)
[----------] 1 test from OverflowTest/0 (0 ms total)

[----------] 1 test from RecyclerTest
[ RUN      ] RecyclerTest.RecycleAllocation
=================================================================
==1610643==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x5020000112d0 in thread T0:
  object passed to delete has wrong type:
  size of the allocated type:   8 bytes;
  size of the deallocated type: 1 bytes.
Step 10 (stage2/asan_ubsan check) failure: stage2/asan_ubsan check (failure)
...
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using lld-link: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/lld-link
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using ld64.lld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/ld64.lld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/llvm/config.py:506: note: using wasm-ld: /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/bin/wasm-ld
llvm-lit: /home/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 88102 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
FAIL: LLVM-Unit :: Support/./SupportTests/135/174 (78782 of 88102)
******************** TEST 'LLVM-Unit :: Support/./SupportTests/135/174' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/unittests/Support/./SupportTests-LLVM-Unit-2979769-135-174.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=174 GTEST_SHARD_INDEX=135 /home/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/unittests/Support/./SupportTests
--

Note: This is test shard 136 of 174.
[==========] Running 8 tests from 8 test suites.
[----------] Global test environment set-up.
[----------] 1 test from CastingTest
[ RUN      ] CastingTest.isa_and_nonnull
Classof: 0x5d8b472373e0
Classof: 0x5d8b472373e0
[       OK ] CastingTest.isa_and_nonnull (0 ms)
[----------] 1 test from CastingTest (0 ms total)

[----------] 1 test from EndianStream
[ RUN      ] EndianStream.WriteArrayLE
[       OK ] EndianStream.WriteArrayLE (0 ms)
[----------] 1 test from EndianStream (0 ms total)

[----------] 1 test from HashBuilderTest/3, where TypeParam = <type>
[ RUN      ] HashBuilderTest/3.HashUserDefinedStruct
[       OK ] HashBuilderTest/3.HashUserDefinedStruct (0 ms)
[----------] 1 test from HashBuilderTest/3 (0 ms total)

[----------] 1 test from OverflowTest/0, where TypeParam = <type>
[ RUN      ] OverflowTest/0.MulNoOverflowToMax
[       OK ] OverflowTest/0.MulNoOverflowToMax (0 ms)
[----------] 1 test from OverflowTest/0 (0 ms total)

[----------] 1 test from RecyclerTest
[ RUN      ] RecyclerTest.RecycleAllocation
=================================================================
==1610643==ERROR: AddressSanitizer: new-delete-type-mismatch on 0x5020000112d0 in thread T0:
  object passed to delete has wrong type:
  size of the allocated type:   8 bytes;
  size of the deallocated type: 1 bytes.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jan 6, 2025

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/2009

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/38/87' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-2520-38-87.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=87 GTEST_SHARD_INDEX=38 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants