How to avoid edge color bleeding when rendering on transparent background

[stukowski]

I wonder if there is a way to prevent the background color from bleeding into edge pixels when rendering objects on a fully transparent background (alpha=0). This happens when you calculate more than one sample per pixel for anti-aliasing. The following images show the effect, once for a black but transparent background (background=(0,0,0,0)) and once for a white transparent one (background=(1,1,1,0)).

To demonstrate the issue, I read the rendered RGBA image from the ANARI framebuffer to compose it on top of a checkerboard pattern using "source over" blending.

The following reference picture was created using my own OpenGL-based renderer, employing a simple supersampling technique to achieve anti-aliasing. It shows how the result should ideally look.

I would like to allow users of my application to render transparent graphics that are saved as a PNG file, e.g. to insert them later into a PowerPoint presentation. At the time of rendering, the user may not yet know whether the graphic will later be placed against a dark or a light background.

A possible workaround could be to render only 1 sample per pixel in VisRTX, but with a higher image resolution (true supersampling). The application can then downscale the final image, similar to what I did with the OpenGL image. However, it would be nicer if VisRTX directly supported a rendering mode that produces a clean image with smooth edges.

[stukowski]

I think I can take this question back, because I've found the solution myself. If I render with background=(0,0,0,0) and then interpret the pixel data in the framebuffer as RGBA values with premultiplied alpha, I obtain correct results.

In terms of QImage pixel formats, which I am using in my application, the values in the ANARI framebuffer must be interpreted as Format_RGBA8888_Premultiplied, not Format_RGBA8888. When saving the image to a PNG file, it gets converted to a non-premultiplied alpha format (see https://www.w3.org/TR/PNG-Rationale.html, section "12.8. Non-premultiplied alpha").

Please close this issue if you think this is the end of the story.

[stukowski]

The edge bleeding problem is unfortunately not fully solved yet. When I activate denoising (right image), the background color creeps back in at the edges. Do you think this can somehow be avoided?

The following pictures have been rendered with background=(0,0,0,0) and then painted onto a solid white background, once without and once with active denoising.