Fine-tuned Image Model

SDXL / Replicate

September 2023

Representing the fusion of machine learning and creativity — I teamed up with Anne Milan to fine-tune the SDXL image model with a dataset of 21 digital illustrations from her "A Toast a Day" series. You can now enter short prompts and generate images of toast in Anne's distinctive style.


Fine-tuning is the process of taking a pre-trained model (sometimes referred to as a foundation model) and training it with your own data to create a new model better suited to a specific task.

Replicate makes it easy to deploy machine learning models in the cloud using your own code.

SDXL is the latest iteration of Stable Diffusion, an image generation model developed by Stability AI.

This project was born out of curiosity, and Anne provided the perfect original dataset to work with. While we were positively surprised by the results, it left us pondering the implications for art.


setting up the training

the model identifying the content of each image

first output generated for the prompt "bread with pesto rosso"

generated results


A few days later, we fine-tuned another model using 27 digital illustrations from Anne's "Month of the Moon" series. It was featured in Replicate's email newsletter.