cmd/internal/ld: dwarf generation slow

[mwhudson]

I don't know if this is known or expected, but it surprised me: passing -w to 6l (to disable dwarf generation) can make it almost twice as fast (e.g. when linking the standard library into a shared library, adding -w to the command line makes the link go from ~4.5 s to ~2.2s).

[mwhudson]

(adding GOGC=off takes it down to 1.7s)

[josharian]

IIRC from a couple of weeks ago, a significant amount of the allocations in the linker occur during DWARF generation.

[bradfitz]

(Part of #10571 -- all.bash is too slow)

[mwhudson]

Staring at pprof a bit, I think the layers of IO seem to be hurting -- a stupid hack to cache the offset in the output file (d7d3500) makes the link of libstd.so 25% faster! (4.41->3.31s best of 5 times). I think it's particularly bad for external linking, but linking godoc still improves from 1.97s to 1.79s (about 9%, again best of 5). The change makes almost no difference when -w is passed to disable dwarf.

I don't know that this bug contributes especially much to the slowness of all.bash as dwarf is not produced when you run go test pkg. It does add a few seconds to the make.bash part I guess.

[bradfitz]

Your change makes pos not flush. That's probably the real win.

[mwhudson]

Ah yes, good point. CL coming up.

[thanm]

Closing out this old bug. Much has changed in linker dwarf generation since 2015.