Strategies for LLM-Driven User Interface Generation in Smart Systems: A Comparative Analysis with Decision Guidance

##plugins.themes.bootstrap3.article.main##

Taras Lypak
Halyna Lypak
Taras Kramar
Oleksij Duda

Анотація

Large language models (LLM) provide opportunities to generate user interfaces directly from natural-language intent, but the emerging generative-UI literature remains focused primarily on web and chat contexts rather than smart systems. Smart-system interfaces are characterized by stricter demands, necessitating grounding in concrete device registries, validation against real-world action consequences, and explainability to non-developer users. This paper identifies and compares six distinct strategies currently used for LLM-driven UI generation (direct structured-output prompting, template-augmented generation, retrieval-augmented generation, model-driven hybrid pipelines, constrained generation, and multi-agent decomposition) using criteria tailored to smart-system contexts: output consistency, safety and accuracy validation, explainability, latency, implementation complexity, generalisability, and failure handling. The comparison is then adapted to three representative domains: smart-home consumer IoT, smart industrial supervisory control, and smart cultural heritage (virtual and AR museums, AI-assisted reconstruction interfaces, and smart exhibit displays). For each domain, a primary strategy is recommended, modifications required to meet specific domain constraints are demonstrated, and an adapted reference architecture is presented. A decision matrix and a selection flowchart are developed to guide the selection of an LLM-based UI generation strategy for specific smart-system deployments. The paper closes by outlining several open challenges, including the verification of accuracy-critical generated interfaces and the integration of static generation with runtime adaptation.

##plugins.themes.bootstrap3.article.details##

Номер

Розділ

Articles

Посилання

[1] Y. Cao, P. Jiang, H. Xia, Generative and malleable user interfaces with generative and evolving task-driven data model, in: Proc. of the 2025 CHI Conference on Human Factors in Computing Systems, ACM, New York, 2025. doi:10.1145/3706598.3713285.

[2] N. Hojo, K. Shinoda, Y. Yamazaki, K. Suzuki, H. Sugiyama, K. Nishida, K. Saito, GenerativeGUI: dynamic GUI generation leveraging LLMs for enhanced user interaction on chat interfaces, in: Extended Abstracts of the 2025 CHI Conference on Human Factors in Computing Systems (CHI EA '25), ACM, New York, 2025. doi:10.1145/3706599.3719743.

[3] A. Carrera-Rivera, F. Larrinaga, G. Lasa, G. Martinez-Arellano, G. Unamuno, AdaptUI: a framework for the development of adaptive user interfaces in smart product-service systems, User Modeling and User-Adapted Interaction 34 (2024) 1929–1980. doi:10.1007/s11257-024-09414-0.

[4] E. King, H. Yu, S. Lee, C. Julien, Sasha: creative goal-oriented reasoning in smart homes with large language models, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8 (2024). doi:10.1145/3643505.

[5] S. Amershi, D. Weld, M. Vorvoreanu, A. Fourney, B. Nushi, P. Collisson, J. Suh, S. Iqbal, P. N. Bennett, K. Inkpen, J. Teevan, R. Kikin-Gil, E. Horvitz, Guidelines for human-AI interaction, in: Proc. of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, New York, 2019, pp. 1–13. doi:10.1145/3290605.3300233.

[6] S. Brdnik, T. Heričko, B. Šumak, Intelligent user interfaces and their evaluation: a systematic mapping study, Sensors 22 (2022) 5830. doi:10.3390/s22155830.

[7] I. Laitaruk, I. Mamchych, Some methodological conclusions about the generative АI model data analyst, Scientific Journal of TNTU (Tern.), vol 120, no 4 (2025), pp. 130–140. doi.org/10.33108/visnyk_tntu2025.04.130.

[8] G. Calvary, J. Coutaz, D. Thevenin, Q. Limbourg, L. Bouillon, J. Vanderdonckt, A unifying reference framework for multi-target user interfaces, Interacting with Computers 15 (2003) 289–308. doi:10.1016/S0953-5438(03)00010-9.

[9] K. Todi, G. Bailly, L. A. Leiva, A. Oulasvirta, Adapting user interfaces with model-based reinforcement learning, in: Proc. of the 2021 CHI Conference on Human Factors in Computing Systems, ACM, New York, 2021. doi:10.1145/3411764.3445497.

[10] M. X. Liu, J. Yang, T. J.-J. Li, K. K. Reza, M. Terry, We need structured output: towards user-centered constraints on large language model output, in: Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (CHI EA '24), ACM, New York, 2024. doi:10.1145/3613905.3650756.

[11] Y. Dong, C. F. Ruan, Y. Cai, R. Lai, Z. Xu, Y. Zhao, T. Chen, XGrammar: flexible and efficient structured generation engine for large language models, in: Proc. of Machine Learning and Systems (MLSys 2025), 2025. https://doi.org/10.48550/arXiv.2411.15100

[12] I. Vasic, H.-G. Fill, R. Quattrini, R. Pierdicca, Knowledge graphs vs. large language models: competitors or partners in supporting virtual museums, ACM Journal on Computing and Cultural Heritage (2025). doi:10.1145/3756016.

[13] M. Fakih, R. Dharmaji, Y. Moghaddas, G. Quiros Araya, O. Ogundare, M. A. Al Faruque, LLM4PLC: harnessing large language models for verifiable programming of PLCs in industrial control systems, in: Proc. of the 46th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP '24), ACM, New York, 2024, pp. 192–203. doi:10.1145/3639477.3639743.

[14] Y. Xia, N. Jazdi, J. Zhang, C. Shah, M. Weyrich, Control Industrial Automation System with Large Language Model Agents, in 2025 IEEE 30th International Conference on Emerging Technologies and Factory Automation (ETFA) (pp. 1-8). IEEE. doi:10.48550/arXiv.2409.18009.

[15] R. Birkmose, N. M. Reece, E. H. Norvin, J. Bjerva, M. Zhang, On-device LLMs for home assistant: dual role in intent detection and response generation, in: Proc. of the Tenth Workshop on Noisy and User-generated Text (W-NUT 2025), ACL, 2025, pp. 57–67. https://doi.org/10.18653/v1/2025.wnut-1.7

[16] S. Gallo, F. Paternò, A. Malizia, A conversational agent for creating automations exploiting large language models, Personal and Ubiquitous Computing (2024). doi: 10.1007/s00779-024-01825-5

[17] M. Mountantonakis, Y. Tzitzikas, Generating SPARQL queries over CIDOC-CRM using a two-stage ontology path patterns method in LLM prompts, ACM Journal on Computing and Cultural Heritage, 18(1), (2025). 1-20. doi:10.1145/3708326

[18] H. Lypak, N. Kunanets, N. Veretennikova, H. Matsiuk, T. Kramar, O. Duda, An Information System Project Using Augmented Reality for a Small Local History Museum, in: 2023 IEEE 18th International Conference on Computer Science and Information Technologies (CSIT), Lviv 2023, pp. 1-4. ISSN 27663655. doi:10.1109/CSIT61576.2023.10324194

[19] H. Lypak, T. Lypak, N. Kunanets, Designing a Machine Learning-Based Information System for Preserving and Classifying Documentary Heritage Artifacts, Herald of Khmelnytskyi National University. Technical Sciences, vol. 339, no. 4 (2024), pp. 176-182. ISSN 2307-5732. Doi:10.31891/2307-5732-2024-339-4-29.

Статті цього автора (цих авторів), які найбільше читають