We aim to create an AI function that can find images similar to a given input image or retrieve an image based on the user's input.

We have named this function embed_clip, and its implementation is shown below.

DROP FUNCTION IF EXISTS embed_clip(text);

CREATE or REPLACE FUNCTION embed_clip(text_or_image TEXT)
RETURNS VECTOR AS $$
BEGIN
    RETURN thanosql.embed(
        engine := 'huggingface',
        input := text_or_image,
        model := 'openai/clip-vit-base-patch32',
        model_args := '{"device_map": "cpu"}'
    );
END;
$$ LANGUAGE plpgsql;

You can declare and implement the function in ThanoSQL's Query Manager (QM). By setting the engine to Hugging Face and defining the model as openai/clip-vit-base-patch32, you can immediately start using the CLIP-based SOTA model from Hugging Face.

By pressing the triangular button at the top of the Query Editor, you can execute the function, and the results will appear in the query results window below.

Next, we will create a sample table to use with this function.
We need a table where images are stored. If such a table does not exist, you can create a new one, import an existing one from a database, or upload a CSV file.

For this example, we downloaded ten images from the web and created a table named unsplash_meta_sample. The schema of that table is shown in the image below.

You must embed the data first to classify and search through unstructured data like images. We will embed the sample table using the embed_clip function we created earlier. The picture below shows the query that applies the embed_clip function to the sample table and stores the results in a table named test_vector.

DROP TABLE IF EXISTS test_vector;

CREATE TABLE test_vector
  AS SELECT photo_id
          , photo_image_url
          , photo_submitted_at
          , photo_description
          , embed_clip(photo_image_url) AS embedding
     FROM my_data.unsplash_meta_sample;

SELECT * FROM test_vector

Once the query is successfully executed, you can view the embedded results for each image in the Query Result window, as shown below.

In the Data Viewer, the embedded images will appear as follows.