NEWSLETTER
Image text dissemination models (T2I) have shown impressive results in the generation of visually convincing images after user indications. On the basis of this, several methods further adjust the previously trained T2I model for specific tasks. However, this requires separate models architectures, training designs and multiple parameter sets to handle different tasks. In this article, we present UNIVG, a general dissemination model capable of supporting a wide range of image generation tasks with a single set of pesos. UNIVG deals with multimodal inputs as unified conditions to allow several subsequent applications, ranging from the generation of T2I, interphage, instructions based, the generation of identity preservation and the generation guided by design, to the estimation of depth and the reference segmentation. Through comprehensive empirical studies on data mixing and training in several tasks, we provide detailed information on training processes and decisions that inform our final designs. For example, we show that generation T2I and other tasks, such as instructions based edition, can coexist without performance compensation, while auxiliary tasks such as depth estimation and reference segmentation improve image edition. In particular, our model can even overcome some specific models of tasks at their respective reference points, marking a significant step towards a unified image generation model.
