Add AccThunk to avoid prematurely unthunking thunks

[pxl-th]

• Add AccThunk type (as suggested in Don't create nested thunks when accumulating #1555) that holds thunks to-be-accumulated without unthunking prematurely.
• Add reduced MWE from Infinite time of gradient Flux.jl#2585 as test, that leads to infinite compilation without AccThunk.

PR Checklist

• Tests are added

[pxl-th]

Test failures are unrelated.

[ToucheSir]

Coming from #1555 (comment), I think we need at least one test that shows we aren't introducing new type instabilities when multiple paths try to accumulate different (un)thunked types.

[pxl-th]

Coming from #1555 (comment), I think we need at least one test that shows we aren't introducing new type instabilities when multiple paths try to accumulate different (un)thunked types.

IIUC you mean the case when AccThunk holds thunks of different type and if that leads to more type-instability -> worse performance?

I benchmarked GaussianSplatting.jl and its ~2x faster with AccThunk than without.

[ToucheSir]

Yes. I'd expect any overhead from type instability to be a non-issue for GaussianSplatting.jl, because each operation is much larger and any AD bookkeeping overhead is relatively small. But for problems more sensitive to AD overhead, it would be good to have some numbers.

[pxl-th]

@ToucheSir do you have such a test in mind?