Skip to content

Remote VAE encode #11069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vladmandic opened this issue Mar 15, 2025 · 2 comments
Closed

Remote VAE encode #11069

vladmandic opened this issue Mar 15, 2025 · 2 comments
Labels
bug Something isn't working stale Issues that haven't received updates

Comments

@vladmandic
Copy link
Contributor

Describe the bug

Remote VAE encode was added via PR ##11017
I've added remote-vae feature for img2img and inpaint workflows for sd15, sdxl and flux.1

  • sdxl: works for both img2img and inpaint
  • sd15: works for img2img, but fails for inpaint
    • binary mask (typically a pil image type l) cannot be combined with image param if its a tensor
    • using remote vae to also encode mask image does work, but results in nonsense output as its not designed to encode binary masks
  • flux: fails for img2img and inpaint
    • flux pipeline doesn't care if input image param is already encoded, it tries to encode it again
/home/vlado/dev/sdnext/venv/lib/python3.12/site-packages/diffusers/pipelines/flux/pipeline_flux_img2img.py:637 in prepare_latents636 │   │   image = image.to(device=device, dtype=dtype)
│ ❱  637 │   │   image_latents = self._encode_vae_image(image=image, generator=generator)

Reproduction

(see above)

Logs

System Info

diffusers main branch commit 82188c

Who can help?

@hlky

@vladmandic vladmandic added the bug Something isn't working label Mar 15, 2025
@hlky hlky mentioned this issue Mar 17, 2025
2 tasks
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Apr 15, 2025
@vladmandic vladmandic changed the title Remove VAE encode Remote VAE encode Apr 15, 2025
@vladmandic
Copy link
Contributor Author

this issue is resolved, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working stale Issues that haven't received updates
Projects
None yet
Development

No branches or pull requests

1 participant