Skip to content

Commit 29380ff

Browse files
optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (redis#10587)
All commands / use cases that heavily rely on double to a string representation conversion, (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ), could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures 100% coverage of conversion. This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries ( fmtlib ) that use the optimized double to string conversion underneath. The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). test suite changes: Despite being compatible, in some cases it produces a different result from printf, and some tests had to be adjusted. one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000` instead of 5e+9, which sounds like a bug? In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding issues (`expr 0.8 == 0.79999999999999999`)
1 parent 9ab873d commit 29380ff

18 files changed

+662
-23
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ deps/lua/src/lua
2929
deps/lua/src/luac
3030
deps/lua/src/liblua.a
3131
deps/hdr_histogram/libhdrhistogram.a
32+
deps/fpconv/libfpconv.a
3233
tests/tls/*
3334
.make-*
3435
.prerequisites

deps/Makefile

+7
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ distclean:
4040
-(cd lua && $(MAKE) clean) > /dev/null || true
4141
-(cd jemalloc && [ -f Makefile ] && $(MAKE) distclean) > /dev/null || true
4242
-(cd hdr_histogram && $(MAKE) clean) > /dev/null || true
43+
-(cd fpconv && $(MAKE) clean) > /dev/null || true
4344
-(rm -f .make-*)
4445

4546
.PHONY: distclean
@@ -66,6 +67,12 @@ hdr_histogram: .make-prerequisites
6667

6768
.PHONY: hdr_histogram
6869

70+
fpconv: .make-prerequisites
71+
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
72+
cd fpconv && $(MAKE)
73+
74+
.PHONY: fpconv
75+
6976
ifeq ($(uname_S),SunOS)
7077
# Make isinf() available
7178
LUA_CFLAGS= -D__C99FEATURES__=1

deps/fpconv/LICENSE.txt

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
Boost Software License - Version 1.0 - August 17th, 2003
2+
3+
Permission is hereby granted, free of charge, to any person or organization
4+
obtaining a copy of the software and accompanying documentation covered by
5+
this license (the "Software") to use, reproduce, display, distribute,
6+
execute, and transmit the Software, and to prepare derivative works of the
7+
Software, and to permit third-parties to whom the Software is furnished to
8+
do so, all subject to the following:
9+
10+
The copyright notices in the Software and this entire statement, including
11+
the above license grant, this restriction and the following disclaimer,
12+
must be included in all copies of the Software, in whole or in part, and
13+
all derivative works of the Software, unless such copies or derivative
14+
works are solely in the form of machine-executable object code generated by
15+
a source language processor.
16+
17+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
18+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
19+
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
20+
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
21+
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
22+
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
23+
DEALINGS IN THE SOFTWARE.

deps/fpconv/Makefile

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
STD=
2+
WARN= -Wall
3+
OPT= -Os
4+
5+
R_CFLAGS= $(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS)
6+
R_LDFLAGS= $(LDFLAGS)
7+
DEBUG= -g
8+
9+
R_CC=$(CC) $(R_CFLAGS)
10+
R_LD=$(CC) $(R_LDFLAGS)
11+
12+
AR= ar
13+
ARFLAGS= rcs
14+
15+
libfpconv.a: fpconv_dtoa.o
16+
$(AR) $(ARFLAGS) $@ $+
17+
18+
fpconv_dtoa.o: fpconv_dtoa.h fpconv_dtoa.c
19+
20+
.c.o:
21+
$(R_CC) -c $<
22+
23+
clean:
24+
rm -f *.o
25+
rm -f *.a
26+
27+

deps/fpconv/README.md

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
libfpconv
2+
3+
----------------------------------------------
4+
5+
Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm[1].
6+
7+
This port contains a subset of the 'C' version of Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm available at [github.com/night-shift/fpconv](https://github.com/night-shift/fpconv)).
8+
9+
[1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf

0 commit comments

Comments
 (0)