Launch a dedicated cloud GPU server running Laboratory OS to download and run HiDream E1 Full using any compatible app or framework.
Direct Download
Must be used with a compatible app, notebook, or codebase. May run slowly, or not work at all, depending on local system resources, particularly GPU(s) and available VRAM.
Forge is a platform built on top of Stable Diffusion WebUI to make development easier, optimize resource management, speed up inference, and study experimental features.
Model Report
HiDream-ai / HiDream E1 Full
HiDream E1 Full is an instruction-based image editing model developed by HiDream-ai that enables natural language-guided modifications of existing images. Built upon the HiDream-I1 foundation model, it utilizes a 17-billion parameter sparse Diffusion Transformer architecture with Mixture-of-Experts components. The model supports various editing tasks including style transfer, object manipulation, and content addition or removal while preserving unmodified image regions through spatially weighted loss functions.
Explore the Future of AI
Your server, your data, under your control
HiDream E1 Full is an instruction-based image editing model developed by HiDream.ai, designed to enable fine-grained modifications of images through natural language commands. Built atop the HiDream-I1 foundation model, it leverages advanced diffusion transformer techniques to facilitate both photorealistic and stylized image manipulation. HiDream-E1 integrates architectural innovations and a large-scale training regime, supporting a variety of interactive editing tasks. This article examines its model architecture, training methodology, performance benchmarks, and core functionalities, with reference to primary technical documentation and repositories including arxiv.org, huggingface.co, and github.com.
Example artwork generated by the HiDream-I1 foundation, illustrating the model’s artistic image synthesis capabilities.
HiDream-E1 extends conventional image generation to interactive, instruction-based editing. Users can issue natural language instructions to perform precise modifications to source images, such as altering visual styles, changing object attributes, manipulating color or background elements, or adding and removing content. The model is designed to balance editing fidelity with preservation of image details not targeted by the instruction. This is achieved through a spatially weighted loss function, which focuses model capacity on regions of greatest intended change while minimizing unintended alterations arxiv.org.
A key innovation in HiDream-E1 is its use of strong in-context visual conditioning, providing rich visual guidance through latent embeddings of the source image. The model allows explicit control over the editing pipeline via the refine_strength parameter, determining the relative emphasis on direct modification versus later refinement stages. Additionally, it incorporates a language model backbone, using Llama 3.1 8B Instruct for instruction encoding, further enhanced by optional instruction-refinement scripts which leverage visual language models huggingface.co.
Inherited features from HiDream I1 Full include photorealistic synthesis, rapid processing, and style versatility, enabling outputs that range from lifelike imagery to painterly or cartoon aesthetics hidream.org.
Photorealistic portrait generated by HiDream-I1, reflecting the quality and detail achievable by the underlying model family.
HiDream-E1 is an extension of the HiDream I1 Full model, which features a 17-billion parameter backbone constructed using a sparse Diffusion Transformer (DiT) architecture arxiv.org. The system operates primarily within a learned latent space, using a variational autoencoder (VAE) for encoding and decoding between pixel and latent domains.
The architecture employs a hybrid text encoding module, integrating representations from multiple sources: Long-Context CLIP (CLIP-L/14 and CLIP-G/14), T5-XXL, and a decoder-only language model (Llama 3.1 8B Instruct), yielding textual embeddings to guide image generation and editing huggingface.co.
The main DiT backbone utilizes both dual-stream and single-stream attention blocks. Dual-stream layers process latent image patches and text tokens in parallel before fusing them, followed by a single-stream phase where these representations are concatenated and updated jointly. To enable scalability and efficiency, the transformer integrates a sparse Mixture-of-Experts (MoE) mechanism, where routing algorithms dynamically assign input tokens to specialized feed-forward expert networks. SwiGLU nonlinearities are used within these experts to enhance expressivity. Other elements include adaptive layer normalization (adaLN) with global conditioning from CLIP-derived features, sinusoidal timestep embeddings for diffusion control, and QK-normalization for training stability.
For editing-specific adaptations, HiDream-E1 applies a spatially weighted loss during fine-tuning, which intensifies learning focus on altered image regions. The model pipeline’s refine_strength parameter regulates the transition from editing steps to optional post-processing with HiDream-I1-Full, improving output consistency and realism github.com.
Demonstration grid of HiDream-E1 output, exhibiting editing capabilities such as style conversion, content alteration, background replacement, and text integration on objects. (For input prompts and instructions, refer to cited HuggingFace and GitHub documentation.)
HiDream-E1 is trained via fine-tuning from the HiDream I1 Full generative model. The editing model uses a dataset composed of five million triplets, each containing a source image, a natural language editing instruction, and a corresponding target (edited) image arxiv.org. This enables the model to learn contextual relationships between instructions and visual modifications.
The foundation HiDream I1 Full was trained over multiple stages with latent-flow-matching objectives. This involved assembling large and diverse web-scale image-text datasets, incorporating data deduplication through SSCD feature clustering and Faiss-based similarity filtering. Extensive pre-processing was applied, including content safety filtering, technical quality assessment, watermark detection, and aesthetic scoring. Images were automatically annotated using the MiniCPM-V vision-language model, with captions generated at varying levels of specificity.
Training was performed progressively across resolutions of 256², 512², and 1024² pixels. Optimization leveraged the AdamW optimizer, fully sharded data parallelism (FSDP), mixed-precision, and gradient checkpointing for efficiency. Latent representations were precomputed for rapid training. Knowledge distillation using adversarial objectives produced faster model variants without major quality loss, supporting rapid inference when needed arxiv.org.
Performance and Evaluation
HiDream-E1 has undergone evaluation on established instruction-based editing benchmarks, particularly EmuEdit and ReasonEdit, which assess the accuracy and restraint of model outputs when executing editing tasks. Performance is rated using GPT-4o on a 0–10 scale, where higher scores denote better instruction fulfillment and avoidance of undesired changes. In comparative results, HiDream-E1 achieved an average score of 6.40 on the EmuEdit benchmark and 7.54 on ReasonEdit, surpassing several other contemporaneous models huggingface.co. Categories where HiDream-E1 demonstrated strong results include global edits, object removal, text manipulation, color and style adaptation, local editing, and object/background alteration.
Typical Applications and Limitations
HiDream-E1 is designed for interactive image editing, enabling users to transform images by specifying desired changes in natural language. Typical applications include artistic and photorealistic style transfer, object and background manipulation, color and lighting adjustments, object addition and removal, and custom text insertions. It also forms a component of the broader HiDream-A1 agent system, which integrates text-to-image generation, image understanding, and conversational editing within a unified AI framework arxiv.org.
The model’s limitations involve requirements for licensing certain components (notably the Llama text encoder), possible dependencies on external vision-language APIs for instruction refinement scripts, and developmental status of both model and code, which are subject to ongoing updates github.com. All components are distributed under the MIT License, except for select encoders and VAE modules which retain their original open source licenses huggingface.co.
Family Comparison and Release Milestones
HiDream-E1 is one member of the HiDream model family based around the HiDream I1 Full foundation:
HiDream I1 Full is the standard text-to-image generator with a full sampling schedule.
