Big5 encoding/decoding support

[r12a]

Results for a series of tests for Big5 encoding/decoding can be found at
https://www.w3.org/International/tests/repo/results/encoding-dbl-byte.en#big5

The tests can be run from that page (select the link in the left-most column) or get the tests from the WPT repo. There is a PR at
web-platform-tests/wpt#3197

The tests check whether:

1. the browser produces the expected byte sequences for all characters in the big5 encoding after 0x9F when encoding bytes for a URL produced by a form, using the encoder steps in the specification.
2. the browser produces percent-escaped character references for a URL produced by a form when encoding miscellaneous characters that are not in the big5 encoding. (tests for several ranges)
3. same two types of test when writing characters to an href value
4. the browser decodes all characters as expected from a file generated by encoding all pointers in the big5 encoding per the encoder steps in the specification.
5. the browser decodes all characters as expected from a file generated by encoding all pointers less than 5024 in the big5 encoding per the encoder steps in the specification.
6. the browser decodes characters that are not recognised from the big5 encoding as replacement characters.

The following summarises the current situation according to my testing, for major desktop browsers. (I will be adding nightly results and perhaps other browsers in time.) The table lists the number of characters that were NOT successfully converted by the test.

Notes:

• Edge fails all href encode tests because characters are not converted to percent-escapes in the href attribute.
• Firefox fails all href encode tests for characters not in the encoding because it converts characters to percent-escaped Unicode values instead.

Can we please investigate the failures to ascertain whether:

1. the browser needs to be changed
2. the spec needs to be changed
3. the test is at fault

The following tool may be helpful for investigating issues. It converts between byte sequences and characters for all encodings in the Encoding spec. http://r12a.github.io/apps/encodings/

[r12a]

List of bugs raised:

• https://bugs.webkit.org/show_bug.cgi?id=159890
• https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/8202219/

[jungshik]

Again, in case of Chromium's form(misc) failures, it has the same root cause as #62, #59, #61: See
https://bugs.chromium.org/p/chromium/issues/detail?id=647568

[r12a]

Today and yesterday i updated the results at https://www.w3.org/International/tests/repo/results/encoding-dbl-byte.en#big5 for Firefox, FNightly, Chrome, and Canary. The latest summary is:

[hsivonen]

Thank you. The Big5 tests LGTM for merging into WPT. /cc @domenic

[domenic]

Let's close this as web-platform-tests/wpt#6254 is ready to merge.

[domenic]

Reopening per #61 (comment)

[annevk]

Now that Firefox passes all these tests and a year has passed, I'm happy to consider this done. A new issue would also be less noisy at this point, were one warranted.