Thomas, Aldrin P., George, Shiju, Raj, N. Anand, Ajaz, S. Mohemmed, and Shaju, Midhun (2026) Decision Flow Tracing and Word Impact Analysis in Hybrid Transformer-Conditioned Diffusion Models for Text-to-Image Generation. In: Lecture Notes in Networks and Systems (1927) pp. 163-174. From: SCIS 2025: International Conference on Sustainable Computing and Intelligent Systems, 7-8 November 2025, Canberra, ACT, Australia.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

Text-to-image diffusion models have become a cornerstone of modern generative AI, offering high-quality synthesis yet remaining constrained by their black-box nature, which limits controllability and interpretability. In this work, we propose a hybrid transformer-conditioned diffusion model that integrates UNet-based denoising with multi-head cross-attention transformer blocks at critical latent stages of the diffusion process. The architecture is trained on a curated set of 50,000 samples from DiffusionDB with a 200-step latent diffusion schedule. Text prompts are encoded using a 16-token BERT encoder and mapped into a 256-dimensional latent feature space. Cross-attention layers with eight heads are interlaced within the UNet bottleneck and decoder, enabling token-to-region correspondence and fine-grained semantic propagation. To ensure interpretability, we design an explainability framework that combines hierarchical token-level attention heat maps, temporal attention rollouts, and perceptual ablation studies based on learned image patch similarity. Analysis reveals that object tokens remain spatially and temporally consistent, while attribute tokens demonstrate sharper temporal volatility. Jensen–Shannon divergence quantifies this redistribution of attention across diffusion steps. Experimental evaluation against a standard UNet diffusion baseline demonstrates clear improvements: Fréchet Inception Distance decreases by 19.6, CLIP alignment score increases by 5.4, and Inception Score improves by 18.6. Moreover, attention coherence improves by 22%, underscoring the gains in explainability. The proposed framework establishes a pathway toward accountable, high-fidelity, and interpretable text-to-image synthesis. Beyond performance, it supports critical tasks such as bias evaluation, fairness auditing, and quality assurance, offering a robust foundation for the next generation of explainable generative AI systems.

Item ID:	92413
Item Type:	Conference Item (Research - E1)
ISBN:	978-3-032-22914-4
ISSN:	23673370
Keywords:	Cross-Attention, Diffusion DB, Hybrid Transformer Diffusion, Interpretable Generative Modeling, Prompt Engineering
Copyright Information:	© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2026
Date Deposited:	23 Jun 2026 23:10
	More Statistics