Google Colab

[scatterp2]

I started work on a google colab anyone got any idea how to finalize this ?
https://colab.research.google.com/drive/1Sv5Z1uVM5ZCGJdJSc67qGzEnyA3mlpeF?authuser=3#scrollTo=lzVAqpGLoESp

[j0seo]

Hi, @scatterp2 Can you try the following setup for Colab? I'm still preparing the Colab demo, but I confirmed this env. setup code works well.

import sys
import torch
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
version_str="".join([
    f"py3{sys.version_info.minor}_cu",
    torch.version.cuda.replace(".",""),
    f"_pyt{pyt_version_str}"
])
!pip install fvcore iopath
!pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
!git clone https://github.com/j0seo/3DFuse_Colab
!mv 3DFuse_Colab/* ./
!pip install -r requirements.txt
!mkdir weights
!wget -P ./weights https://huggingface.co/jyseo/3DFuse_weights/resolve/main/models/3DFuse_sparse_depth_injector.ckpt
!pip install pytorch-lightning==1.8.3.post0
!pip install 'prettytable<1'
!pip install 'setuptools==62.3.2'
!pip install 'jedi>=0.16'

[scatterp2]

Hi

With the following code:
https://colab.research.google.com/drive/1uNgcLy6h8E1dki6AHjEhrGKDa-gGuAlt?usp=sharing
and keeping the settings on default (a bed) after generating the image when trying to create the 3d result
it crashes below is the output: note: I found this "Ya, currently pypi version 0.3.0 is broken for the latest TF 2.6.0:"

Collapsing Lin Lora in q_proj
Collapsing Lin Lora in out_proj
Reshaping encoder.mid.attn_1.q.weight for SD format
Reshaping encoder.mid.attn_1.k.weight for SD format
Reshaping encoder.mid.attn_1.v.weight for SD format
Reshaping encoder.mid.attn_1.proj_out.weight for SD format
Reshaping decoder.mid.attn_1.q.weight for SD format
Reshaping decoder.mid.attn_1.k.weight for SD format
Reshaping decoder.mid.attn_1.v.weight for SD format
Reshaping decoder.mid.attn_1.proj_out.weight for SD format
Added Token: ['<0>']
a comfortable <0>
0% 0/10000 [00:00<?, ?it/s]/usr/local/lib/python3.9/dist-packages/torch/nn/functional.py:4236: UserWarning:

Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.

0% 0/10000 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 337, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1015, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 847, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/dist-packages/anyio/backends/asyncio.py", line 867, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.9/dist-packages/gradio/utils.py", line 410, in async_iteration
return next(iterator)
File "/content/gradio_app.py", line 41, in gen_3d
yield from model.run_gradio(points)
File "/content/run_3DFuse.py", line 147, in run_gradio
yield from fuse_3d(**cfgs, poser=poser,model=model,vox=vox,exp_dir=exp_dir, points=points, is_gradio=True)
File "/content/run_3DFuse.py", line 195, in fuse_3d
y, depth, ws = render_one_view(vox, aabb, H, W, Ks[i], poses[i], return_w=True)
File "/content/run_3DFuse.py", line 325, in render_one_view
rgbs = rearrange(rgbs, "(h w) c -> 1 c h w", h=H, w=W)
File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 424, in rearrange
return reduce(tensor, pattern, reduction='rearrange', **axes_lengths)
File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 368, in reduce
return recipe.apply(tensor)
File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 203, in apply
backend = get_backend(tensor)
File "/usr/local/lib/python3.9/dist-packages/einops/_backends.py", line 49, in get_backend
if backend.is_appropriate_type(tensor):
File "/usr/local/lib/python3.9/dist-packages/einops/_backends.py", line 513, in is_appropriate_type
return self.K.is_tensor(tensor) and self.K.is_keras_tensor(tensor)
AttributeError: module 'keras.backend' has no attribute 'is_tensor'