std::io::Read::read_to_end does not specialize for &[u8] or Take<&'a [u8]>

[fulmicoton]

read_to_end currently includes some logic to do exponentially large .resize / .read operations.

For &[u8] or Take<&'a [u8]>, the problem is much simpler as we know the size that will be required :

v.extend_from_slice(&data);
let n = v.len();
data = &data[n..];

The manual implementation is typically between 3 and 5 times faster on my computer.

Assuming what seems a non-pathological case : no resize required, the Vec has a sufficient capacity to begin with, and running the following benchmark https://gist.github.com/fulmicoton/a1fb87c3f3578f118552917636c95933
yields the following result :

test tests::bench_read_manual ... bench:           9 ns/iter (+/- 1)
test tests::bench_read_std    ... bench:          35 ns/iter (+/- 6)

[fulmicoton]

A simple solution would be to specialize .read_to_end for those types.

Alternatively, we could also expose a public trait similar in spirit to TrustedLen for iterators, and implement the specialization for this new trait. It seems a bit overkill to expose another Trait for this though.

Currently, libstd does not enable on the #![feature(specialization)] feature.
Other crates of rust (liballoc for instance), already enable it. Is there a case against doing it yet?

[sfackler]

No, there's no reason we can't enable specialization in libstd.

I don't think specialization is required for the &[u8] case though.

[fulmicoton]

@sfackler By which you mean it is not used / not worth it?

[sfackler]

No - you can just add a read_to_end implementation in the impl<'a> Read for &'a [u8] { ... }.

[fulmicoton]

Oh yes that is true :)