Binary value in result leads to crashing of sql-tap tests

[ImeevMA]

How to reproduce:

1. Add a new test that returns binary value to sql-tap tests. For example:

diff --git a/test/sql-tap/sql-errors.test.lua b/test/sql-tap/sql-errors.test.lua
index 469193a2b..0ca58c2ff 100755
--- a/test/sql-tap/sql-errors.test.lua
+++ b/test/sql-tap/sql-errors.test.lua
@@ -1,6 +1,6 @@
 #!/usr/bin/env tarantool
 local test = require("sqltester")
-test:plan(72)
+test:plan(73)
 
 test:execsql([[
 	CREATE TABLE t0 (i INT PRIMARY KEY, a INT);
@@ -780,4 +780,12 @@ test:do_catchsql_test(
 		-- </sql-errors-2.9>
 	})
 
+test:do_execsql_test(
+	"sql-errors-3.0",
+	[[
+		SELECT RANDOMBLOB(5);
+	]], {
+		1
+	})
+
 test:finish_test()

2. Run test. For example:

./test-run.py sql-tap/sql-errors.test.lua -j1

Result:

Started ./test-run.py sql-tap/sql-errors.test.lua -j1
Running in parallel with 1 workers

Timeout options:
-------------------
REPLICATION_SYNC_TIMEOUT: 100
TEST_TIMEOUT:             110
NO_OUTPUT_TIMEOUT:        120

Collecting tests in 'app'            (Found 0   tests): application server tests.
Collecting tests in 'app-tap'        (Found 0   tests): application server tests (TAP).
Collecting tests in 'box'            (Found 0   tests): Database tests.
Collecting tests in 'box-py'         (Found 0   tests): legacy python tests.
Collecting tests in 'box-tap'        (Found 0   tests): Database tests with #! using TAP.
Collecting tests in 'engine'         (Found 0   tests): tarantool multiengine tests.
Collecting tests in 'engine_long'    (Found 0   tests): tarantool engine stress tests.
Collecting tests in 'long_run-py'    (Found 0   tests): long running tests.
Collecting tests in 'replication'    (Found 0   tests): tarantool/box, replication.
Collecting tests in 'replication-py' (Found 0   tests): tarantool/box, replication.
Collecting tests in 'small'          (Found 0   tests): libsmall unit tests.
Collecting tests in 'sql'            (Found 0   tests): sql tests.
Collecting tests in 'sql-tap'        (Found 2   tests): Database tests with #! using TAP.
Collecting tests in 'swim'           (Found 0   tests): SWIM tests.
Collecting tests in 'unit'           (Found 0   tests): unit tests.
Collecting tests in 'vinyl'          (Found 0   tests): vinyl integration tests.
Collecting tests in 'wal_off'        (Found 0   tests): tarantool/box, wal_mode = none.
Collecting tests in 'xlog'           (Found 0   tests): tarantool write ahead log tests.
Collecting tests in 'xlog-py'        (Found 0   tests): legacy python tests.

Tarantool server information
 | Found executable at /home/mergen/work/dev/src/tarantool
 | Found tarantoolctl at /home/mergen/work/dev/extra/dist/tarantoolctl

 | Tarantool 2.8.0-232-g4e828bae9
 | Target: Linux-x86_64-Debug
 | Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON
 | Compiler: /usr/bin/cc /usr/bin/c++
 | C_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -Werror
 | CXX_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror


======================================================================================
WORKR TEST                                            PARAMS          RESULT
---------------------------------------------------------------------------------
[001] sql-tap/sql-errors.test.lua                     vinyl           
[001] Worker "001_sql-tap" received the following error; stopping...
[001] Traceback (most recent call last):
[001]   File "/home/mergen/work/dev/test-run/lib/worker.py", line 315, in run_task
[001]     short_status, result_checksum = self.suite.run_test(
[001]   File "/home/mergen/work/dev/test-run/lib/test_suite.py", line 272, in run_test
[001]     short_status, result_checksum = test.run(server)
[001]   File "/home/mergen/work/dev/test-run/lib/test.py", line 223, in run
[001]     is_tap, is_ok = self.check_tap_output()
[001]   File "/home/mergen/work/dev/test-run/lib/test.py", line 345, in check_tap_output
[001]     content = f.read()
[001]   File "/usr/lib/python3.8/codecs.py", line 322, in decode
[001]     (result, consumed) = self._buffer_decode(data, self.errors, final)
[001] UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 1568: invalid start byte
[001] 
[001] Exception: 'utf-8' codec can't decode byte 0x81 in position 1568: invalid start byte
---------------------------------------------------------------------------------
[Internal test-run error] The following tasks were dispatched to some worker task queue, but were not reported as done (does not matters success or fail):
- [sql-tap/sql-errors.test.lua, vinyl]
- [sql-tap/sql-errors.test.lua, memtx]

[Totktonada]

AFAIR, it can be reproduced with test-run's test test-unit/broken_unicode.test if we'll drop broken_unicode.result.

[Totktonada]

We decided to provide a better error message instead of the stacktrace. We'll NOT support non UTF-8 data in TAP13 parsing, at least not now: I don't see much value in this functionality.

After PR #297 test-run's error reporting should become less confusing:

$ cd ${TARANTOOL_REPO}/test-run
$ rm test/test-unit/broken_unicode.result
$ ./test/test-run.py broken_unicode
<...>
[002] test-unit/broken_unicode.test                                   
[002] TAP13 parse failed ('utf-8' codec can't decode byte 0xc2 in position 27: invalid continuation byte).
[002] 
[002] No result file (test-unit/broken_unicode.result) found.
[002] Run the test with --update-result option to write the new result file.
[002] [ fail ]
<...>

So, if you have non UTF-8 data in the test output, there are the following options:

1. Change the test to make its output human readable.
2. Call test-run with --update-result and store the result file with non UTF-8 data.