'CLIPTextTransformer' object has no attribute '_build_causal_attention_mask'

[Aukture]

[xyhanHIT]

Hello, I encountered the same issue, and I resolved it by changing the version of transformers to 4.27.3!

[ankan8145]

option 1: !pip install -q --upgrade transformers==4.25.1 downgrade the transformers will help you

option 2: another fix, without having to downgrade, could be to use what the function _build_causal_attention_mask used to have (from here

def build_causal_attention_mask(bsz, seq_len, dtype):
    # lazily create causal attention mask, with full attention between the vision tokens
    # pytorch uses additive attention mask; fill with -inf
    mask = torch.empty(bsz, seq_len, seq_len, dtype=dtype)
    mask.fill_(torch.tensor(torch.finfo(dtype).min))
    mask.triu_(1)  # zero out the lower diagonal
    mask = mask.unsqueeze(1)  # expand mask
    return mask

Thanks! Hope this helps