-
Notifications
You must be signed in to change notification settings - Fork 17
Add default server restart to each test run #309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
501d14f
to
fde42a6
Compare
Typo: tarantol -> tarantool. |
fde42a6
to
4a082f9
Compare
Found that the root cause of the issues happened with vinyl tests were backside effects of the not correct test 'vinyl/gh.test.lua' which leaved Tarantool worker process in inconsistent state. After it any other next test on the same Tarantool worker process could fail on running testings with snapshots calls, like tarantool/tarantool-qa#126: error: Snapshot is already in progress Either restarting Tarantool worker process could fail on stopping it, like #261 and tarantool/tarantool#5141: E> failed to process vylog record: delete_slice{slice_id=115, } E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered Decided to avoid of such situations for now and in the future all tests must be run after Tarantool worker default server process was restarted. This patch moves this server restart call from check if the test failed to the common part of the tests run loop. Fixes #260 Fixes #261 Needed for tarantool/tarantool#5089 Closes tarantool/tarantool-qa#126
4a082f9
to
650dd6f
Compare
650dd6f
to
fd3e5f2
Compare
Please, look at the comment in TarantoolServer.pretest_clean(). It becomes non-actual. In fact, xlog/snap are removed between tests, but in very non-obvious way (and it does not depend on the pretest_clean suite.ini option anyhow). Please, at least update the comment. Maybe it also worth to drop the option at all and always perform clean up between tests. Actually the following code removes xlog/snap: test-run/lib/tarantool_server.py Lines 715 to 717 in fd3e5f2
I propose to spin around the code for a hour and decide about some 'good enough' amount of changes we'll do now (postpone everything else a bit). |
b5d9279
to
a9ecfa5
Compare
After the previous commit in the current patchset we don't use the same tarantool server process from test to test and we don't need to use any scripts like 'pretest_clean.lua' to cleanup it between tests. So this patch removes this script.
Found that all currently used suite.ini files use this option enabled. Also there is no need to have any suite of tests where snap either log files whould be used by the next test from the previous, due to such test schema can be easily implemented within single test file.
a9ecfa5
to
5c6417f
Compare
I've tried with and w/o clean and saw that this function is still needed. Also checked that it calls on tests starts with 'core = tarantool' suite. Found that there is no any test suites suite.ini file that use this option. Also there is no need to have any suite of tests where snap either log files whould be used by the next test from the previous, due to such test schema can be easily implemented within single test file. It means that this option 'pretest_clean' is a dead code and not needed at all. So I've added tiny patch to remove it. |
To be honest, I'm not very happy about the code: it is more complex then it would be. However, it seems, everything works as expected, so it looks as a way to go. |
This update offers two changes: * More intelligent error reporting for non UTF-8 TAP13 output ([1], [2]). * Restart tarantool server before each test (to avoid unexpected and non-obvious dependencies between tests, see [3]). Removed the pretest_clean suite.ini option: test-run does not read it anymore. [1]: tarantool/test-run#293 [2]: tarantool/test-run#297 [3]: tarantool/test-run#309
This update offers two changes: * More intelligent error reporting for non UTF-8 TAP13 output ([1], [2]). * Restart tarantool server before each test (to avoid unexpected and non-obvious dependencies between tests, see [3]). Removed the pretest_clean suite.ini option: test-run does not read it anymore. [1]: tarantool/test-run#293 [2]: tarantool/test-run#297 [3]: tarantool/test-run#309 (cherry picked from commit 7fa70b6)
This update offers two changes: * More intelligent error reporting for non UTF-8 TAP13 output ([1], [2]). * Restart tarantool server before each test (to avoid unexpected and non-obvious dependencies between tests, see [3]). Removed the pretest_clean suite.ini option: test-run does not read it anymore. [1]: tarantool/test-run#293 [2]: tarantool/test-run#297 [3]: tarantool/test-run#309 (cherry picked from commit 7fa70b6)
This update offers two changes: * More intelligent error reporting for non UTF-8 TAP13 output ([1], [2]). * Restart tarantool server before each test (to avoid unexpected and non-obvious dependencies between tests, see [3]). Removed the pretest_clean suite.ini option: test-run does not read it anymore. [1]: tarantool/test-run#293 [2]: tarantool/test-run#297 [3]: tarantool/test-run#309 (cherry picked from commit 7fa70b6)
Updated the test-run submodule in tarantool in 2.9.0-104-g7fa70b661, 2.8.1-75-g029d66355, 2.7.2-62-g6f9ccad83, 1.10.10-25-g9b4acec8f. |
Now it is an irrelevant feature for us because we restart Tarantool server before each test [1]. So the order of tests shouldn't affect anything. This patch just disables reproduce content printing. It is planned to drop the related code later. [1] #309
Now it is an irrelevant feature for us because we restart Tarantool server before each test [1]. So the order of tests shouldn't affect anything. This patch just disables reproduce content printing. It is planned to drop the related code later. [1] #309
Now it is an irrelevant feature for us because we restart Tarantool server before each test [1]. So the order of tests shouldn't affect anything. This patch just disables reproduce content printing. It is planned to drop the related code later. [1] #309
Now it is an irrelevant feature for us because we restart Tarantool server before each test [1]. So the order of tests shouldn't affect anything. This patch just disables reproduce content printing. It is planned to drop the related code later. [1] #309
Patchset of 3 patches:
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like test: flaky vinyl/bloom.test.lua fail because of previous vinyl/gh.test.lua on Snapshot is already in progress tarantool-qa#126:
Either restarting Tarantool worker process could fail on stopping it,
like ./test-run --gdb is broken in 1.6 tarantool#261 and test: flaky vinyl/gh.test.lua test tarantool-qa#205:
Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.
Part of resources: disk bound tests need to run in special way tarantool-qa#97
Fixes Problems with installing tarantool-modules on master tarantool#260
Fixes ./test-run --gdb is broken in 1.6 tarantool#261
Needed for test: flaky vinyl/deferred_delete.test.lua tarantool#5089
Closes test: flaky vinyl/bloom.test.lua fail because of previous vinyl/gh.test.lua on Snapshot is already in progress tarantool-qa#126
After the previous commit in the current patchset we don't use the same
tarantool server process from test to test and we don't need to use any
scripts like 'pretest_clean.lua' to cleanup it between tests. So this
patch removes this script.
Found that all currently used suite.ini files use this option enabled.
Also there is no need to have any suite of tests where snap either
log files whould be used by the next test from the previous, due to
such test schema can be easily implemented within single test file.