Skip to content

test: flaky vinyl/bloom.test.lua fail because of previous vinyl/gh.test.lua on Snapshot is already in progress #126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
avtikhon opened this issue Jun 7, 2021 · 0 comments · Fixed by tarantool/test-run#309
Closed

test: flaky vinyl/bloom.test.lua fail because of previous vinyl/gh.test.lua on Snapshot is already in progress #126

avtikhon opened this issue Jun 7, 2021 · 0 comments · Fixed by tarantool/test-run#309
Assignees
@avtikhon

Comments

@avtikhon
Copy link
Contributor

avtikhon commented Jun 7, 2021 •
edited
Loading

Tarantool 2.9.0-57-gea0b126ff
Target: Linux-x86_64-Debug
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON
Compiler: /usr/bin/cc /usr/bin/c++
C_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -Werror
CXX_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror

Fail:

[001] Test failed! Result content mismatch:
[001] --- vinyl/bloom.result	Wed Apr  7 07:45:20 2021
[001] +++ var/rejects/vinyl/bloom.reject	Mon Jun  7 08:23:17 2021
[001] @@ -15,7 +15,7 @@
[001]  ...
[001]  box.snapshot()
[001]  ---
[001] -- ok
[001] +- error: Snapshot is already in progress
[001]  ...
[001]  for i = 1, 10 do s:get{i} end
[001]  ---
[001] 
[001] Last 15 lines of Tarantool Log file [Instance "vinyl"][/home/ubuntu/tarantool/test/var/001_vinyl/vinyl.log]:
[001] 2021-06-07 08:23:17.225 [1424727] vinyl.dump.0/102/task I> writing `/home/ubuntu/tarantool/test/var/001_vinyl/vinyl/539/0/00000000000000000118.index'
[001] 2021-06-07 08:23:17.225 [1424727] snapshot/101/main I> saving snapshot `/home/ubuntu/tarantool/test/var/001_vinyl/vinyl/00000000000000021501.snap.inprogress'
[001] 2021-06-07 08:23:17.230 [1424727] main/108/vinyl.scheduler I> 539/0: dump completed
[001] 2021-06-07 08:23:17.230 [1424727] main/108/vinyl.scheduler I> dumped 49338 bytes in 0.0 s, rate 3.6 MB/s
[001] 2021-06-07 08:23:17.231 [1424727] snapshot/101/main I> done
[001] 2021-06-07 08:23:17.231 [1424727] main/123/console/unix/: I> vinyl checkpoint completed
[001] 2021-06-07 08:23:17.240 [1424727] main I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/00000000000000020342.snap
[001] 2021-06-07 08:23:17.240 [1424727] main I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/00000000000000019325.vylog
[001] 2021-06-07 08:23:17.240 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000096.index
[001] 2021-06-07 08:23:17.241 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000096.run
[001] 2021-06-07 08:23:17.241 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000102.index
[001] 2021-06-07 08:23:17.241 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000102.run
[001] 2021-06-07 08:23:17.241 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000104.index
[001] 2021-06-07 08:23:17.241 [1424727] main/105/gc I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/535/0/00000000000000000104.run
[001] 2021-06-07 08:23:17.243 [1424727] wal I> removed /home/ubuntu/tarantool/test/var/001_vinyl/vinyl/00000000000000020342.xlog

Reproducer:

echo >a.rep <<EOF
- [vinyl/gh.test.lua, null]
- [vinyl/bloom.test.lua, null]
EOF

c=0 ; while ./test-run.py --reproduce a.rep --force ; do rm -rf var/ ; c=$(($c+1)) ; echo "================ $c" ; done

Found that running tests in the same Tarantool worker process next tests after 'vinyl/gh.test.lua' test may get it in non consistence state when some snapshot internal processes not yet completed. To avoid of it 'vinyl/gh.test.lua' test must be fixed either each test must be run in standalone new Tarantool worker process.

To be sure that there is no big impact on time of the testings:

Parallel Host No restarts With restarts Diff
-j1 MCS Github 14m40s 17m20s +15%
-j4 (default) MCS Github 9m32s 10m26s +9%
-j8 MCS Github 9m06s 10m28s +13%
-j16 Laptop 5m59s 7m02s +14%

More parallel values not interesting due to Github Actions hosts and MCS local hosts use 2 cores hosts with 4 test threads by default and bigger parallel values doesn't change the timings in general.
@avtikhon avtikhon mentioned this issue Jun 9, 2021
Closed
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
3ff5659 
Found that thr root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent status. After which
any other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before to be able to run with
other tests.

Part of tarantool/tarantool-qa#97
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
51c0568 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
3dd02e8 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
@avtikhon avtikhon mentioned this issue Jun 9, 2021
Merged
@avtikhon avtikhon changed the title test: flaky vinyl/bloom.test.lua fail on Snapshot is already in progress test: flaky vinyl/bloom.test.lua fail because of previous vinyl/gh.test.lua on Snapshot is already in progress Jun 9, 2021
@avtikhon avtikhon mentioned this issue Jun 9, 2021
Closed
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
928200f 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #5089
This was referenced Jun 9, 2021
Closed
Closed
Closed
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
f494b3b 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #5089
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 9, 2021
@avtikhon
test: unblock vinyl tests from fragile list
73144d3 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5089
Closes #5187
avtikhon added a commit to tarantool/test-run that referenced this issue Jun 10, 2021
@avtikhon
Add default server restart to each test run
2da1b8b 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like #261 and tarantool/tarantool#5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.

Part of tarantool/tarantool-qa#97
Needed for tarantool/tarantol#5089
Closes tarantool/tarantool-qa#126
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 10, 2021
@avtikhon
test: unblock vinyl tests from fragile list
20dcdfc 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
@avtikhon avtikhon mentioned this issue Jun 10, 2021
Merged
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 10, 2021
@avtikhon
test: unblock vinyl tests from fragile list
d7c8ae3 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 10, 2021
@avtikhon
test: unblock vinyl tests from fragile list
52ca7a0 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
avtikhon added a commit to tarantool/test-run that referenced this issue Jun 11, 2021
@avtikhon
Add default server restart to each test run
41e7a26 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like #261 and tarantool/tarantool#5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.

Part of tarantool/tarantool-qa#97
Needed for tarantool/tarantol#5089
Closes tarantool/tarantool-qa#126
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 12, 2021
@avtikhon
test: unblock vinyl tests from fragile list
6d73df1 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 12, 2021
@avtikhon
test: unblock vinyl tests from fragile list
e4ca354 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 13, 2021
@avtikhon
test: unblock vinyl tests from fragile list
bbd862d 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4979
Closes #4984
Closes #4993
Closes #5187
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 16, 2021
@avtikhon
test: unblock vinyl tests from fragile list
601df71 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4168
Closes #4309
Closes #4346
Closes #4572
Closes #4979
Closes #4984
Closes #4985
Closes #4993
Closes #5141
Closes #5197
Closes #5336
Closes #5338
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes #5408
Closes #5539
Closes #5584
Closes #5586
@avtikhon avtikhon self-assigned this Jun 16, 2021
@avtikhon avtikhon added the teamQ label Jun 16, 2021
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 16, 2021
@avtikhon
test: unblock vinyl tests from fragile list
4e5912d 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes #5408
Closes #5584
Closes #5586
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 18, 2021
@avtikhon
test: unblock vinyl tests from fragile list
890299c 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes #5408
Closes #5584
Closes #5586
@avtikhon avtikhon mentioned this issue Jun 18, 2021
Closed
avtikhon added a commit to tarantool/test-run that referenced this issue Jun 18, 2021
@avtikhon
Add default server restart to each test run
10a3a82 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like #261 and tarantool/tarantool#5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.

Part of tarantool/tarantool-qa#97
Fixes #260
Fixes #261
Needed for tarantool/tarantool#5089
Closes tarantool/tarantool-qa#126
avtikhon added a commit to tarantool/test-run that referenced this issue Jun 18, 2021
@avtikhon
Add default server restart to each test run
17bee72 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like #261 and tarantool/tarantool#5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.

Fixes #260
Fixes #261
Needed for tarantool/tarantool#5089
Closes tarantool/tarantool-qa#126
Totktonada pushed a commit to tarantool/test-run that referenced this issue Jun 21, 2021
@avtikhon @Totktonada
Add default server restart to each test run
33fbace 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like #261 and tarantool/tarantool#5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to avoid of such situations for now and in the future all tests
must be run after Tarantool worker default server process was restarted.
This patch moves this server restart call from check if the test failed
to the common part of the tests run loop.

Fixes #260
Fixes #261
Needed for tarantool/tarantool#5089
Closes tarantool/tarantool-qa#126
@avtikhon avtikhon mentioned this issue Jun 22, 2021
Closed
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 22, 2021
@avtikhon
test: unblock vinyl tests from fragile list
1e5a8e4 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes #5408
Closes #5584
Closes #5586
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 22, 2021
@avtikhon
test: remove fixed tests from fragile lists
d276d8d 
Checked and found that:

  #4353: engine/ddl.test.lua fixed in #6102.
  #4926, #115: box/alter_limits.test.lua fixed
    in tarantool/tarantool-qa#126.
  #5547: box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583: box/net.box_methods_gh-3107.test.lua
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes #4353
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes #5547
Closes tarantool/tarantool-qa#22
Closes #5583
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: unblock vinyl tests from fragile list
e33ff22 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

Part of tarantool/tarantool-qa#97
Closes #4346
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes #5408
Closes #5584
Closes #5586
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: remove fixed tests from fragile lists
baea518 
Checked and found that:

  #4353: engine/ddl.test.lua fixed in #6102.
  #4926, #115: box/alter_limits.test.lua fixed
    in tarantool/tarantool-qa#126.
  #5547: box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583: box/net.box_methods_gh-3107.test.lua
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes #4353
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes #5547
Closes tarantool/tarantool-qa#22
Closes #5583
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: unblock vinyl tests from fragile list
d3987a1 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: remove fixed tests from fragile lists
c53b566 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: unblock vinyl tests from fragile list
e9fb3dc 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19
avtikhon added a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon
test: remove fixed tests from fragile lists
9bf3203 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: unblock vinyl tests from fragile list
f0f53a3 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: remove fixed tests from fragile lists
4053a35 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: unblock vinyl tests from fragile list
ae86d6f 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19

(cherry picked from commit f0f53a3)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: remove fixed tests from fragile lists
cadf352 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22

(cherry picked from commit 4053a35)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: unblock vinyl tests from fragile list
a650359 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19

(cherry picked from commit f0f53a3)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: remove fixed tests from fragile lists
c080c4c 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22

(cherry picked from commit 4053a35)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: unblock vinyl tests from fragile list
1c4a4f0 
Found that the root cause of the issues happened with vinyl tests were
backside effects of the not correct test 'vinyl/gh.test.lua' which
leaved Tarantool worker process in inconsistent state. After it any
other next test on the same Tarantool worker process could fail on
running testings with snapshots calls, like tarantool/tarantool-qa#126:

  error: Snapshot is already in progress

Either restarting Tarantool worker process could fail on stopping it,
like tarantool/test-run#261 and #5141:

  E> failed to process vylog record: delete_slice{slice_id=115, }
  E> ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Slice 115 deleted but not registered

Decided to remove all vinyl tests from 'fragile' list except test
'gh.test.lua', which should be improved before, to be able to run it
with the other tests. And 'gh-5141-invalid-vylog-file.test.lua' test
which checks this issue and can be removed after the fix will be done.

The following issues were moved to tarantool/tarantool-qa repository:
  #4346 -> tarantool/tarantool-qa#11
  #5408 -> tarantool/tarantool-qa#73
  #5584 -> tarantool/tarantool-qa#21
  #5586 -> tarantool/tarantool-qa#19

Part of tarantool/tarantool-qa#97
Closes tarantool/tarantool-qa#11
Closes #4572
Closes #4979
Closes #4984
Closes #5336
Closes #5356
Closes #5377
Closes #5378
Closes #5383
Closes tarantool/tarantool-qa#73
Closes tarantool/tarantool-qa#21
Closes tarantool/tarantool-qa#19

(cherry picked from commit f0f53a3)
kyukhin pushed a commit to tarantool/tarantool that referenced this issue Jun 23, 2021
@avtikhon @kyukhin
test: remove fixed tests from fragile lists
a01caf8 
Checked and found that:

  #4353 -> tarantool/tarantool-qa#13:
    engine/ddl.test.lua fixed in #6102.
  #4926, #115:
    box/alter_limits.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5547 -> tarantool/tarantool-qa#50:
    box/net.box_schema_change_gh-2666.test.lua fixed in
    tarantool/tarantool-qa#126.
  #5583 -> tarantool/tarantool-qa#22:
    box/net.box_methods_gh-3107.test.lua fixed in
    tarantool/tarantool-qa#126.

Closes tarantool/tarantool-qa#13
Closes tarantool/tarantool-qa#115
Closes #4926
Closes tarantool/tarantool-qa#50
Closes tarantool/tarantool-qa#22

(cherry picked from commit 4053a35)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@avtikhon avtikhon

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add default server restart to each test run tarantool/test-run
1 participant
@avtikhon