Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] jni_util.cc:79: Failed to update reservation while freeing bytes: Java Exception: java.lang.IncompatibleClassChangeError #8963

Open
beliefer opened this issue Mar 11, 2025 · 1 comment
Labels
bug Something isn't working triage

Comments

@beliefer
Copy link
Contributor

Backend

VL (Velox)

Bug description

When I commit the PR #8962.
I received an error message int test run-spark-test-spark35-scala213
The error message show below.

E20250311 12:08:25.004997  1998 Exceptions.h:66] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2043, Function:terminate, Expression:  Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE
25/03/11 12:08:25 WARN GlutenFallbackReporter: Validation failed for plan: Scan text , due to: 
 - Unsupported file format UnknownFormat.
/opt/gluten/ep/_ep/arrow_ep/java/dataset/src/main/cpp/jni_util.cc:79: Failed to update reservation while freeing bytes: Java Exception: java.lang.IncompatibleClassChangeError

/tmp/jnilib-1622654969009870607.tmp(+0x11b6338)[0x7efc5908e338]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow4util8ArrowLogD1Ev+0xed)[0x7efc5908e76d]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow7dataset3jni31ReservationListenableMemoryPool4FreeEPhll+0x47d)[0x7efc5870358d]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow10PoolBufferD0Ev+0x47)[0x7efc593336e7]
/tmp/jnilib-1622654969009870607.tmp(_ZNSt23_Sp_counted_ptr_inplaceIN5arrow9ArrayDataESaIS1_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv+0x10e)[0x7efc5890458e]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow17SimpleRecordBatchD1Ev+0x10e)[0x7efc591bc94e]
/tmp/jnilib-1622654969009870607.tmp(_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE10_M_releaseEv+0x3a)[0x7efc586fa46a]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_11RecordBatchEEE20WrapResultOnComplete8CallbackINSC_14ThenOnCompleteIZNS_23DefaultIfEmptyGeneratorISB_EclEvEUt_NSC_17PassthruOnFailureISI_EEEEEEED0Ev+0x2e)[0x7efc5874effe]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x1a4)[0x7efc590a8494]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureISt10shared_ptrINS_11RecordBatchEEE12MarkFinishedENS_6ResultIS3_EE+0x96)[0x7efc5876f526]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_11RecordBatchEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISC_SC_Lb0ELb0EEEEEE6invokeES4_+0x55)[0x7efc5876f895]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x40)[0x7efc590a81c0]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xe5)[0x7efc5909eb35]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt8functionIFNS8_ISt10shared_ptrINS_11RecordBatchEEEEvEEE20WrapResultOnComplete8CallbackINSG_14ThenOnCompleteIZNS_20FutureFirstGeneratorISC_EclEvEUlRKSF_E_NSG_17PassthruOnFailureISO_EEEEEEE6invokeES4_+0x10c)[0x7efc5876fdbc]
- csv scan v1
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureISt8functionIFNS0_ISt10shared_ptrINS_11RecordBatchEEEEvEEE12MarkFinishedENS_6ResultIS7_EE+0x117)[0x7efc587792a7]
/tmp/jnilib-1622654969009870607.tmp(+0x85c160)[0x7efc58734160]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureISt10shared_ptrINS_3csv15StreamingReaderEEE12MarkFinishedENS_6ResultIS4_EE+0x10c)[0x7efc587617bc]
/tmp/jnilib-1622654969009870607.tmp(+0x8484ea)[0x7efc587204ea]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_3csv15StreamingReaderEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISD_SD_Lb0ELb0EEEEEE6invokeES4_+0x15b)[0x7efc58753a5b]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(+0x115f4e4)[0x7efc590374e4]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureINS_8internal5EmptyEE12MarkFinishedIS2_vEEvNS_6StatusE+0xc2)[0x7efc5874c742]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureINS0_5EmptyEE21WrapStatusyOnComplete8CallbackINS_6detail16MarkNextFinishedISA_SA_Lb1ELb1EEEEEE6invokeES4_+0x28)[0x7efc5874d5b8]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x40)[0x7efc590a81c0]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xe5)[0x7efc5909eb35]
/tmp/jnilib-1622654969009870607.tmp(+0x11664e1)[0x7efc5903e4e1]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureISt10shared_ptrINS_6BufferEEE12MarkFinishedENS_6ResultIS3_EE+0xef)[0x7efc58ab561f]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_6BufferEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISC_SC_Lb0ELb0EEEEEE6invokeES4_+0x55)[0x7efc58ab58c5]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x40)[0x7efc590a81c0]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xe5)[0x7efc5909eb35]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_6BufferEEE20WrapResultOnComplete8CallbackINSC_14ThenOnCompleteIZNS_21TransformingGeneratorISB_SB_E26TransformingGeneratorStateclEvEUlRKSB_E_NSC_17PassthruOnFailureISL_EEEEEEE6invokeES4_+0xa6)[0x7efc5904fff6]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x196)[0x7efc590a8486]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow6FutureISt10shared_ptrINS_6BufferEEE12MarkFinishedENS_6ResultIS3_EE+0xef)[0x7efc58ab561f]
/tmp/jnilib-1622654969009870607.tmp(_ZN5arrow8internal6FnOnceIFvvEE6FnImplIZZNS0_8Executor10DoTransferISt10shared_ptrINS_6BufferEENS_6FutureIS9_EENS_6ResultIS9_EEEENSA_IT_EESF_bENUlRKSD_E0_clESH_EUlvE_E6invokeEv+0x54)[0x7efc58ab5744]
/tmp/jnilib-1622654969009870607.tmp(+0x11b18d4)[0x7efc590898d4]
/lib64/libstdc++.so.6(+0xc2ba3)[0x7efd1cb8aba3]
/lib64/libpthread.so.0(+0x817a)[0x7efd1e8cc17a]
/lib64/libc.so.6(clone+0x43)[0x7efd1dfd0dc3]

[INFO] Reactor Summary for Gluten Parent Pom 1.4.0-SNAPSHOT:
[INFO] 
[INFO] Gluten Parent Pom .................................. SUCCESS [  0.563 s]
[INFO] Gluten Ras ......................................... SUCCESS [  6.092 s]
[INFO] Gluten Ras Common .................................. SUCCESS [ 45.847 s]
[INFO] Gluten Shims ....................................... SUCCESS [  0.628 s]
[INFO] Gluten Shims Common ................................ SUCCESS [ 11.825 s]
[INFO] Gluten Shims for Spark 3.5 ......................... SUCCESS [ 11.079 s]
[INFO] Gluten UI .......................................... SUCCESS [  4.652 s]
[INFO] Gluten Core ........................................ SUCCESS [ 30.474 s]
[INFO] Gluten Substrait ................................... SUCCESS [ 51.413 s]
[INFO] Gluten Arrow ....................................... SUCCESS [ 12.724 s]
[INFO] Gluten Celeborn .................................... SUCCESS [  4.569 s]
[INFO] Gluten Iceberg ..................................... SUCCESS [ 11.282 s]
[INFO] Gluten DeltaLake ................................... SUCCESS [ 11.070 s]
[INFO] Gluten Backends Velox .............................. FAILURE [01:09 min]
[INFO] Gluten Package ..................................... SKIPPED
[INFO] Gluten Ras Planner ................................. SKIPPED
[INFO] Gluten Unit Test Parent ............................ SKIPPED
[INFO] Gluten Unit Test Common ............................ SKIPPED
[INFO] Gluten Unit Test ................................... SKIPPED
[INFO] Gluten Unit Test Spark35 ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:33 min
[INFO] Finished at: 2025-03-11T12:08:28Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.scalatest:scalatest-maven-plugin:2.2.0:test (test) on project backends-velox: There are test failures -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Error:  
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :backends-velox
Error: Process completed with exit code 1.

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

@beliefer beliefer added bug Something isn't working triage labels Mar 11, 2025
@beliefer beliefer changed the title jni_util.cc:79: Failed to update reservation while freeing bytes: Java Exception: java.lang.IncompatibleClassChangeError [VL] jni_util.cc:79: Failed to update reservation while freeing bytes: Java Exception: java.lang.IncompatibleClassChangeError Mar 11, 2025
@beliefer
Copy link
Contributor Author

ping @zhouyuan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant