HowTo use Dreambooth Training to personalize Stable Diffusion models

Dreambooth - Customize diffusion models for your needs

Dreambooth allows users to train diffusion models for their specific use case. We offer Dreambooth training as a service and there are 2 ways you can train a diffusion model on Tiyaro.

• From the UI
• Dreambooth API for training

The Dreambooth API for training is documented here . This is the goto option for developers who are trying to use training in their applications or workflows. Please follow that document if you are primarily interested in training models using an API. This blog covers using Dreambooth through our UI

Dreambooth training in Tiyaro UI

Start with Model training at ‘Model Sudio’ -> ‘Train’ . Click ‘New Training Project’

In the popup select the ‘Dreambooth’ tile as shown below and Click ‘Next’

The next screen asks you the configuration information for the training. The 3 most important and required inputs for configuration are

1. Upload Dataset. (Format of the dataset zip file explained below)
2. instance prompt
3. class prompt

Rest all the other parameters are 'optional' and you can simply use the defaults.

Dataset is a zip file. The Zip file contents should have the following structure. The instance class images (the images which add the concept to the stable diffusion) must be directly zipped at the root level. These images are required. If you need to add class images to your zip then you should add them to a folder called 'dataset_class'. Class images are for images generated by the stable diffusion model for a dataset prompt and they are optional. For reference you can download the dog toy dataset and examine it. You will see something like the following.

.
└── dataset.zip/
  ├── image1.jpeg
  ├── image2.jpeg
  ├── .
  ├── .
  └── dataset_class/
    ├── image1.jpeg
    └── image2.jpeg

Next add the name of the project and follow the guided ‘Next’ buttons to ‘Submit’ the job.

Once the job is submitted it typically takes 30-45 minutes to complete. You will receive an email once the job is finished.

You can find the trained job again in the ‘Model Studio’ -> ‘Train’ window. See below.

In the above screenshot, ‘dreambooth-training-example’ shows a completed job. You can click on the model in the ‘Models’ column .e.g ‘dreambooth-training-example’ to take you to the model card of your ‘Custom model’

You can also find a link to this trained model by clicking on the training project row in the ‘Results’ sub tab, see below.

Clicking on ‘dreambooth-training-example’ in Results->TrainedModels above will take you to your custom model card.

The custom model card is similar to any other model hosted as API with the API url, sample code, demo etc.

Train using Github Action

Now, you can also train your own custom Dreambooth Stable Diffusion model on Tiyaro using Github Action: https://github.com/tiyaro/dreambooth-training-action

Summary

With a few clicks you can use the Tiyaro Model Studio to personalize stable diffusion models using Dreambooth