hint plz: how can I have the image encoders output

[mohaEs]

Hi.
Thanks for the great work.
I am going to use your foundation model for extracting features from fundus photos.
How can I have/return the output of the image encoder?
What's the variable name
Thanks

[jusiro]

Hello @mohaEs,

I would want to apologize for the late response.

Following the example depicted in the cover page of FLAIR's repository, you can extract features for a specific image as follows:

from PIL import Image
import numpy as np
import torch

# Import FLAIR
from flair import FLAIRModel

# Set model
model = FLAIRModel(from_checkpoint=True)

# Load image
# (if the repo is not cloned, download the image and change the path!)
image = np.array(Image.open("./documents/sample_macular_hole.png"))

# Pre-process image
image_norm = model.preprocess_image(image)

# Forward in the image encoder
with torch.no_grad():
    img_embeds = model.vision_model(image_norm).cpu().numpy()

# Analize shape
print("Feature embedding dimensionality: ")
print(img_embeds.shape)
print("First features: ")
print(img_embeds[0,0:5])

The expected output you should get by running this example is:

Feature embedding dimensionality: 
(1, 512)
First features: 
[-0.00990654 -0.06638613  0.04151397  0.02718035  0.04644661]

Kind regards.