Hierarchical Control Composition for Multi-Scale Semantic Manipulation

Existing control methods like LOCO Edit and Ingredient Control Module operate at single semantic levels, forcing users to choose between granular detail or high-level concepts. This research proposes hierarchical control composition that allows simultaneous manipulation across multiple abstraction levels. The core insight is that control signals naturally form a hierarchy - changing "style" affects "brush strokes" which affects "pixel patterns." We learn this hierarchy automatically by discovering the natural abstraction levels in the latent space using information bottleneck principles. Users could then invoke control at any level (e.g., "make this more impressionistic") and the system would automatically propagate appropriate changes to all subordinate levels. This goes beyond the composability in LOCO Edit by providing principled cross-level interactions rather than independent editing directions. The approach could enable unprecedented creative control in applications ranging from architectural design (modifying both building style and window details simultaneously) to molecular design (controlling both protein structure and amino acid interactions).

References:

1. CookGALIP: Recipe Controllable Generative Adversarial CLIPs With Sequential Ingredient Prompts for Food Image Generation. Mengling Xu, Jie Wang, Ming Tao, Bing-Kun Bao, Changsheng Xu (2025). IEEE transactions on multimedia.