Skip to content

Commit 2541d14

Browse files
authored
[advanced flux lora script] minor updates to readme (#9705)
* fix arg naming * fix arg naming * fix arg naming * fix arg naming
1 parent 5704376 commit 2541d14

File tree

1 file changed

+6
-8
lines changed

1 file changed

+6
-8
lines changed

examples/advanced_diffusion_training/README_flux.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ Please keep the following points in mind:
9696
To activate pivotal tuning for both encoders, add the flag `--enable_t5_ti`.
9797
* When not fine-tuning the text encoders, we ALWAYS precompute the text embeddings to save memory.
9898
* **pure textual inversion** - to support the full range from pivotal tuning to textual inversion we introduce `--train_transformer_frac` which controls the amount of epochs the transformer LoRA layers are trained. By default, `--train_transformer_frac==1`, to trigger a textual inversion run set `--train_transformer_frac==0`. Values between 0 and 1 are supported as well, and we welcome the community to experiment w/ different settings and share the results!
99-
* **token initializer** - similar to the original textual inversion work, you can specify a token of your choosing as the starting point for training. By default, when enabling `--train_text_encoder_ti`, the new inserted tokens are initialized randomly. You can specify a token in `--initializer_token` such that the starting point for the trained embeddings will be the embeddings associated with your chosen `--initializer_token`.
99+
* **token initializer** - similar to the original textual inversion work, you can specify a concept of your choosing as the starting point for training. By default, when enabling `--train_text_encoder_ti`, the new inserted tokens are initialized randomly. You can specify a token in `--initializer_concept` such that the starting point for the trained embeddings will be the embeddings associated with your chosen `--initializer_concept`.
100100

101101
## Training examples
102102

@@ -147,7 +147,6 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
147147
--pretrained_model_name_or_path=$MODEL_NAME \
148148
--dataset_name=$DATASET_NAME \
149149
--instance_prompt="3d icon in the style of TOK" \
150-
--validation_prompt="a TOK icon of an astronaut riding a horse, in the style of TOK" \
151150
--output_dir=$OUTPUT_DIR \
152151
--caption_column="prompt" \
153152
--mixed_precision="bf16" \
@@ -165,7 +164,7 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
165164
--lr_scheduler="constant" \
166165
--lr_warmup_steps=0 \
167166
--rank=8 \
168-
--max_train_steps=1000 \
167+
--max_train_steps=700 \
169168
--checkpointing_steps=2000 \
170169
--seed="0" \
171170
--push_to_hub
@@ -190,7 +189,6 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
190189
--pretrained_model_name_or_path=$MODEL_NAME \
191190
--dataset_name=$DATASET_NAME \
192191
--instance_prompt="3d icon in the style of TOK" \
193-
--validation_prompt="a TOK icon of an astronaut riding a horse, in the style of TOK" \
194192
--output_dir=$OUTPUT_DIR \
195193
--caption_column="prompt" \
196194
--mixed_precision="bf16" \
@@ -209,7 +207,7 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
209207
--lr_scheduler="constant" \
210208
--lr_warmup_steps=0 \
211209
--rank=8 \
212-
--max_train_steps=1000 \
210+
--max_train_steps=700 \
213211
--checkpointing_steps=2000 \
214212
--seed="0" \
215213
--push_to_hub
@@ -229,7 +227,6 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
229227
--pretrained_model_name_or_path=$MODEL_NAME \
230228
--dataset_name=$DATASET_NAME \
231229
--instance_prompt="3d icon in the style of TOK" \
232-
--validation_prompt="a TOK icon of an astronaut riding a horse, in the style of TOK" \
233230
--output_dir=$OUTPUT_DIR \
234231
--caption_column="prompt" \
235232
--mixed_precision="bf16" \
@@ -249,7 +246,7 @@ accelerate launch train_dreambooth_lora_flux_advanced.py \
249246
--lr_scheduler="constant" \
250247
--lr_warmup_steps=0 \
251248
--rank=8 \
252-
--max_train_steps=1000 \
249+
--max_train_steps=700 \
253250
--checkpointing_steps=2000 \
254251
--seed="0" \
255252
--push_to_hub
@@ -273,8 +270,9 @@ pipe = AutoPipelineForText2Image.from_pretrained("black-forest-labs/FLUX.1-dev",
273270
pipe.load_lora_weights(repo_id, weight_name="pytorch_lora_weights.safetensors")
274271
```
275272
2. now we load the pivotal tuning embeddings
276-
💡note that if you didn't enable `--enable_t5_ti`, you only load the embeddings to the CLIP encoder
273+
> [!NOTE] #1 if `--enable_t5_ti` wasn't passed, we only load the embeddings to the CLIP encoder.
277274
275+
> [!NOTE] #2 the number of tokens (i.e. <s0>,...,<si>) is either determined by `--num_new_tokens_per_abstraction` or by `--initializer_concept`. Make sure to update inference code accordingly :)
278276
```python
279277
text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
280278
tokenizers = [pipe.tokenizer, pipe.tokenizer_2]

0 commit comments

Comments
 (0)