[v 3.2.0] Not Working: Exception Assault using CM4SB (chaos monkey for spring boot) in Spring Data JPA Repository

[pratik-chauhan]

Hello folks,

I am trying to inject java.sql.SQLException exception assault in spring data jpa repository method considering below configuration. But, it is not injectiong the exception assault at repository method.

Applied Assault configuration

Applied Watcher configuration

Read Assault-Watcher configuration

API Endpoint response

chaos-monkey-assault-watcher-configuration.json

As per the pom.xml, SpringData JPA dependecny is present in classpath
pom.xml

I tried to reproduce this issue in chaos-monkey-demo-app for http://localhost:8080/dbgreet API Endpoint

Thank you !!

[pratik-chauhan]

Hello folks,

I can see same is working fine with v2.2.0 which was demostrated in Chaos Monkey for Spring Boot - John Fletcher & Manuel Wessner @ Spring I/O Bridge

Applied Assault configuration

Applied Watcher configuration

** Read Assault-Watcher configuration**

API Endpoint response

Demo Application console logs

chaos-monkey-assault-watcher-configuration_v2.2.0.json

[pratik-chauhan]

Hello @jens-kaiser @denniseffing @WtfJoke
I would like to request to help on it.
Please suggest if I am doing something wrong at assault-watcher configuration.
Appreciate your efforts !!
Thank you.

[pratik-chauhan]

Hello folks,
Please share your thoughts on this issue.
Appreciate your efforts !!!
Thanks !!

[WtfJoke]

We see your issue and comments.
However we usually dont work 24/7 on this project. There limited people who address issues in the project.
We most likely work fridays on this project (but not guaranteed). So usually it takes some time to get an answer from us, I hope you understand that.

For the sake of the issue, it looks like this what you show it should work. Can you also share the code of your JPA-Repository (or even better a small reproduction sample project or were you able to reproduce in the chaos-monkey-demo-app?), that would help to solve the issue.

[jens-kaiser]

Your assault configuration is wrong.

"watchedCustomService": ["com.example.chaos.monkey.chaosdemo.repo.HelloRepo.save"]

It must be

"watchedCustomService": ["com.example.chaos.monkey.chaosdemo.repo.HelloRepo#save"]

Or leave it out, as in the working example.

[pratik-chauhan]

Hello @jens-kaiser @WtfJoke - First of all sorry to bother you. I did not aware about the process. Feel free to reply and address to this issue whenever you gets chance. I would like to share my finding around this issue.

@WtfJoke - As you suggested, I tried to reproduce this issue by creating junit test case. I found that junit test case is working as expected meaning it is able to inject fault at repository method, but same is not working when applying assault-watcher through REST Endpoint for repository method.
Please refer this PR - Junit test case(#558)
Please refer recording - Recording

I would request that whenever you get time, please look into it and suggest if I am doing something wrong or really there is a scope of improvement.

Appreaciate your inputs and efforts !!

[pratik-chauhan]

Added logs to check if RepositoryAnnotatedClassFilter is printing logs for eligible repository proxy classes during applying fault through assault-watcher REST calls

When exception assault applied through REST API calls

When exception assault applied through Junit Test case

[pratik-chauhan]

Hello folks,

Should you please at least direct me on -- wheather I am doing something wrong or if it is really something breaking. Please provide direction on breaking area in the code where I can start looking into it and try to contribute on fixing this issue in case if it is really an issue.

Thank you so much !!!

[WtfJoke]

Have you seen @jens-kaiser comment? There is a typo in the config;

Your assault configuration is wrong.

"watchedCustomService": ["com.example.chaos.monkey.chaosdemo.repo.HelloRepo.save"]

It must be

"watchedCustomService": ["com.example.chaos.monkey.chaosdemo.repo.HelloRepo#save"]

Or leave it out, as in the working example.

[pratik-chauhan]

Hi @WtfJoke - I have seen that comment. But, I tried to apply fault for all repository operations instead of specific operation ( as per above comment - i.e. HelloRepo#save ) assuming if it applys to falult for all repository operations, it should consider specific operation also. Considering that understanding, I have tried and share findings in my previous comment.

Let me try for specific operation also.

Thanks !!!

[pratik-chauhan]

Hi @WtfJoke - I tried applying HelloRepo#save but no luck.

Please see applied assault-watcher configuration as below:
{
"chaosMonkeyProperties": {
"enabled": true,
"lastEnabledToggleTimestamp": 1743952773136,
"togglePrefix": "chaos.monkey"
},
"assaultProperties": {
"level": 1,
"deterministic": true,
"latencyRangeStart": 1000,
"latencyRangeEnd": 3000,
"latencyActive": false,
"exceptionsActive": true,
"exception": {
"type": "java.lang.RuntimeException",
"method": "",
"arguments": [
{
"type": "java.lang.String",
"value": "Chaos Monkey - RuntimeException"
}
]
},
"killApplicationActive": false,
"killApplicationCronExpression": "OFF",
"memoryActive": false,
"memoryMillisecondsHoldFilledMemory": 90000,
"memoryMillisecondsWaitNextIncrease": 1000,
"memoryFillIncrementFraction": 0.15,
"memoryFillTargetFraction": 0.25,
"memoryCronExpression": "OFF",
"cpuActive": false,
"cpuMillisecondsHoldLoad": 90000,
"cpuLoadTargetFraction": 0.9,
"cpuCronExpression": "OFF",
"watchedCustomServices": [
"com.example.chaos.monkey.chaosdemo.repo.HelloRepo#save"
]
},
"watcherProperties": {
"controller": false,
"restController": false,
"service": false,
"repository": true,
"component": false,
"restTemplate": false,
"webClient": false,
"actuatorHealth": false,
"beans": [],
"beanClasses": [],
"excludeClasses": []
}
}

Response of http://localhost:8080/dbgreet API endpoint where it internally calls HelloRepo.save method.

Note: There is no source code changes made in chaos-monkey-demo-app

Thank you for replying and taking interest. Please suggest if I am doing wrong in configuration part.

[WtfJoke]

Its also reproducible with just chaos.monkey.watcher.repository=true

It sounds really similar to #335 and there is even an ChaosMonkeyRepositoryWatcherIntegrationTest in place to prevent a reintroduction of that issue.

Full application.properties

spring.profiles.active=chaos-monkey
chaos.monkey.enabled=true
chaos.monkey.assaults.exceptions-active=true
chaos.monkey.watcher.repository=true
chaos.monkey.assaults.level=1

@pratik-chauhan currently I think its not a configuration issue. We'll have a look. Sorry for the wrong closing. Please have patience.

[pratik-chauhan]

Hey @WtfJoke @jens-kaiser - Just wanted to gently follow up on the issue. Were you able to take a look at it?

Thanks !!!

[WtfJoke]

I haven’t found what’s the cause of the issue on the demo app. Also tried to spot the difference between the demo app and the integration test, but were not yet successful. One repo gets proxied the other dont. Need to give it another shot, but currently time is very limited.

[pratik-chauhan]

Hey @WtfJoke - Added jdk.proxy4. as match criteria for jdkproxiedClass in ChaosMonkeyBaseClassFilter
which enables to inject fault at repository. Please review and verify it.

Thanks !!!

[pratik-chauhan]

Hi @WtfJoke @jens-kaiser - I have raised PR-562 and provided fix for this issue.

I know you are running with very limited time for this project. I tried to identify and provided fix for this issue. It would be great if you guys can spend time to review PR-562 and release this change if all looks fine from your end.

Appreciate your efforts !!!

Thank You !!

[WtfJoke]

Hey @pratik-chauhan

We were able to have a look at it today.
We confirmed, that your pull-request solves the issue (startsWith("jdk.proxy")).
We also identified in our analysis (thanks for your analysis and proposed fix, that helped a lot!) that the NUMBER in jdk.proxyNUMBER differs based of the jdk version (see #564).

However we decided to not rely on guess-working of internal jdk proxy class names and instead rely on Proxy.isProxyClass(clazz). Performance wise this could be a slightly worser decision but its the more sustainable/stable solution and makes it less likely that we reintroduce the issue. That's why we created and merged #564, we kept you in the changelog, I hope thats fine for you :)

We will release soon a new version, which includes the fix.

[WtfJoke]

Is fixed with v3.2.1 and should be soon available in maven central

[pratik-chauhan]

Thanks @WtfJoke and @denniseffing for make the issue fix as part of release.
Appreciate your efforts !!