Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factorized linear supports implementation switch and gradient checkpoint #26

Merged
merged 6 commits into from
Jun 23, 2022
+66 −44
Merged

Factorized linear supports implementation switch and gradient checkpoint #26

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4e5af82
Embedding supports CUDA Tensor
Jun 16, 2022
c147c1e
Embedding supports CUDA Tensor
Jun 16, 2022
aa24134
original version
Jun 16, 2022
3cec478
support implementation switch and gradient checkpoint
Jun 16, 2022
6f8e368
Apply suggestions from code review
JeremieMelo Jun 23, 2022
cb4a507
Update factorized_linear.py
JeremieMelo Jun 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Embedding supports CUDA Tensor
jiaqig committed Jun 16, 2022
commit 4e5af825fb6b00ad5982b2b3d3a04166804295ec
4 changes: 2 additions & 2 deletions tltorch/factorized_layers/factorized_embedding.py
View file
Original file line number Diff line number Diff line change
@@ -90,7 +90,7 @@ def forward(self, input, indices=0):
#to handle case where input is not 1-D
output_shape = (*input.shape, self.embedding_dim)

flatenned_input = input.view(-1)
flatenned_input = input.flatten()

if self.n_layers == 1:
if indices == 0:
@@ -107,7 +107,7 @@ def forward(self, input, indices=0):
elif self.factorization.lower() == 'tucker':
embeddings = embeddings.reshape(input.shape[0], -1)

return embeddings.view(output_shape)
return embeddings.reshape(output_shape)

@classmethod
def from_embedding(cls,
Loading