Closed
Description
What API design would you like to have changed or added to the library? Why?
l guess maybe the param low_cpu_mem_usage need to be added in load_lora_weight method of StableDiffusionXLLoraLoaderMixin.
What use case would this enable or better enable? Can you give us a code example?
In my pro. l come to a mixed case between stabilityai/stable-diffusion-xl-refiner-1.0 and lcm-lora-sdxl, so l choose the pipeline 'StableDiffusionXLImg2ImgPipeline' wth the inhert method 'load_lora_weights' to load the function of img2img.But it failed while
my torch version >= 1.9.0 and low_cpu_mem_usage's default value is True.
When l set low_cpu_mem_usage to be False, it didn't worked.
like this:
pipe = StableDiffusionXLImg2ImgPipeline.from_single_file(
model_id, # the way to stable-diffusion-xl-refiner-1.0
torch_dtype=torch.float16,
variant="fp16"
).to("cuda")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.load_lora_weights(
lcm_lora_id,
low_cpu_mem_usage=False,
ignore_mismatched_sizes=True)
init_image = load_image(img_orgin)
image = pipe(
prompt=prompt_orgin,
image=init_image,
num_inference_steps=4,
guidance_scale=1,
strength=0.6
).images[0]