Automating operations and maintenance manual data extraction for retrofit applications: assessing GPT-4o’s performance and challenges
Author(s)
Type
Conference Paper
Abstract
Operations & Maintenance (O&M) manuals contain data that is essential to understand ahead of building retrofits. However, their unstructured format makes information retrieval time-consuming and error-prone. This study evaluates GPT-4o’s ability to extract information from O&M manuals, assessing accuracy in text extraction, categorization, and numerical data handling. Structured extraction steps and prompt strategies were developed based on industry needs. The extraction performance was compared to human-extracted data based on a dataset of 60 O&M manuals, using quantitative and qualitative evaluation. Results form the study show that text-based extraction is reliable, particularly for component names, types, and manufacturer details, where formatting differences had little impact on accuracy. However, challenges appeared in fields requiring categorization, numerical interpretation, and structured mapping. One issue was category misalignment, where GPT-4o could assign components to broad categories but had difficulty with domain-specific classifications that require specialized knowledge. This inconsistency suggests that rule-based constraints or further adjustments may be needed to improve standardization in retrofit applications. Another challenge was the model’s handling of numerical data. Mistakes occurred in quantity extraction, page numbering, and location references, where the model miscalculated totals, misread measurement units, or mismatched printed and digital page numbers. These errors suggest that GPT-4o lacks a way to process numerical information in context, meaning manual checks are still required for numerical fields. These findings highlight GPT-4o’s potential to automate O&M data extraction, reducing manual effort and speeding up the retrofit process.
Date Issued
2025-10-27
Date Acceptance
2025-05-15
Citation
International Retrofit Conference 2025 Proceedings, 2025, pp.98-103
ISBN
9781917780049
Publisher
The University of Salford
Start Page
98
End Page
103
Journal / Book Title
International Retrofit Conference 2025 Proceedings
Copyright Statement
© 2025 by the authors. Licensee University of Salford, Manchester, United Kingdom. This conference proceedings is distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)
License URL
Source
International Retrofit Conference 2025
Publication Status
Published
Start Date
2025-07-03
Finish Date
2025-07-04
Coverage Spatial
Salford, UK