ControlNet SD 1.5 Soft Edge

ControlNet SD 1.5 Soft Edge, also known as HED (Holistically-Nested Edge Detection), is a specialized model within the ControlNet 1.1 family designed to add conditional control to Stable Diffusion 1.5 image generation. The model focuses on producing smooth lines around objects, making it particularly effective for tasks involving recoloring and stylization, as detailed in the Civitai guide to ControlNet.

The model represents a significant evolution from its predecessor in ControlNet 1.0, incorporating improved robustness through the introduction of "SoftEdge_safe" pre-processing. This technique, documented in the ControlNet v1.1 repository, helps mitigate issues caused by hidden patterns in original images that could potentially distract the model.

Training and Variants

The training data for ControlNet SD 1.5 Soft Edge comprised multiple datasets:

• SoftEdge_PIDI
• SoftEdge_PIDI_safe
• SoftEdge_HED
• SoftEdge_HED_safe

These variants offer different trade-offs between robustness and maximum achievable image quality. The performance hierarchy, as documented in the official repository, is:

Robustness:

SoftEdge_PIDI_safe > SoftEdge_HED_safe >> SoftEdge_PIDI > SoftEdge_HED

Maximum result quality:

SoftEdge_HED > SoftEdge_PIDI > SoftEdge_HED_safe > SoftEdge_PIDI_safe

The default recommendation is to use SoftEdge_PIDI for most applications.

Implementation and Usage

The model is available in multiple formats, including a more efficient Low-Rank Adaptation (LoRA) version developed by Stability AI. The LoRA variants significantly reduce the model size from 4.7GB to approximately 738MB (Rank 256) or 377MB (Rank 128), as detailed in the Stability AI Control-LoRA repository.

For implementation, users can run the model through various interfaces:

• The gradio_softedge.py script from the original repository
• Automatic1111's web UI via the ControlNet extension
• ComfyUI
• StableSwarmUI

When using the sd-webui-controlnet extension (version 1.1.400 and later), the model can be placed in either:

• stable-diffusion-webui\extensions\sd-webui-controlnet\models
• stable-diffusion-webui\models\ControlNet

For optimal performance on systems with limited VRAM:

• 8GB-16GB VRAM: Use the --medvram-sdxl command-line argument
• Less than 8GB: Use the --lowvram argument

References

• Civitai Guide to ControlNet - Comprehensive overview of ControlNet models and their applications
• ControlNet v1.1 Repository - Official repository with technical details and implementation code
• SD-WebUI ControlNet Discussion - Technical discussion about implementation and optimization
• Stability AI Control-LoRA - Information about the compressed LoRA versions of the model
• Automatic1111 ControlNet Extension - Popular implementation interface for the model