Automatic evaluation of design alternatives with quantitative argumentation
File(s)Baronietal_TARC_1001791.pdf (1.48 MB)
Accepted version
Author(s)
Baroni, Pietro
Romano, Marco
Toni, Francesca
Aurisicchio, Marco
Bertanza, Giorgio
Type
Journal Article
Abstract
This paper presents a novel argumentation framework to support Issue-Based Information System style debates on design alternatives, by providing an automatic quantitative evaluation of the positions put forward. It also identifies several formal properties of the proposed quantitative argumentation framework and compares it with existing non-numerical abstract argumentation formalisms. Finally, the paper describes the integration of the proposed approach within the design Visual Understanding Environment software tool along with three case studies in engineering design. The case studies show the potential for a competitive advantage of the proposed approach with respect to state-of-the-art engineering design methods.
Engineering design is often described as an information-processing activity based on problem-solving within the constraints of bounded rationality (Simon, 1996; Simon & Newell, 1971). It consists of decomposing an initial problem into a range of sub-problems, proposing and assessing partial solutions, and integrating them as to satisfy the overall problem. This process is collaborative and often involves communication between non-co-located engineers. The development and communication of design solutions require engineers to form and share design rationale, that is, the argumentation in favour or against proposed designs.
These aspects of the engineering design process have led to the development (Kunz & Rittel, 1970) and subsequent investigation (Buckingham Shum & Hammond, 1994; Fischer, Lemke, McCall, & Morch, 1991) of the issue-based information system (IBIS) method, a graph-based formalisation of the decisions made during a design process along with the reasons why they were made. The IBIS method envisions a decision-making process where problems (or issues) are given solutions (or answers) after a thorough debate involving technical, economical, life, environmental and safety considerations. It also provides means to actively develop, communicate and record the reasons (or arguments) in favour or against the options explored during the design process. Initially, IBIS was conceived purely as a conceptual information system and its first implementations were paper-based and totally operated by hand. However, over time several software tools supporting editing and visualisation of IBIS graphs have been developed, for example, Compendium, DRed and design Visual Understanding Environment (designVUE) (e.g. see Aurisicchio & Bracewell, 2013; Buckingham Shum et al., 2006). These IBIS-based tools, including designVUE, which was selected as a starting point for this research, still leave to the users the burden of actually deriving any conclusion from the argumentative process and, eventually, making a decision. This is a task that, depending on the structure of the graph, may not be trivial.
This paper describes the outcome of collaborative research, involving experts of engineering design and argumentation theory, undertaken to overcome the limitations of standard design tools in general, and designVUE in particular. The ultimate goal of this research is to support engineers by providing them with a visual tool to automatically evaluate alternative design solutions and suggest the most promising answers to a design issue, given the underlying graph structure developed during the design process.
Since one of the main features of argumentation theory is evaluating arguments’ acceptability (e.g. as in Cayrol & Lagasquie-Schiex, 2005a; Dung, 1995) or strength (e.g. as in Cayrol & Lagasquie-Schiex, 2005b; Evripidou & Toni, 2012; Leite & Martins, 2011; Matt & Toni, 2008) within debates and dialogues, we have singled it out as a promising companion to engineering design to achieve our research goal. For this application area, conventional notions of ‘binary’ acceptability (e.g. the notions in Dung, 1995), sanctioning arguments as acceptable or not, are better replaced with notions of numerical strength, as the latter are more fine-grained and allow to distinguish different degrees of acceptability.
This paper presents both theoretical and practical results. On the theoretical side, we propose a formal method to assign a numerical score to the nodes of an IBIS graph, starting from a base score provided by users. On the practical side, we describe the implementation of this method within designVUE and its preliminary evaluation in the context of three case studies.
The paper is organised as follows. Section 1 gives the basic notions concerning IBIS and the necessary background on argumentation theory. Section 2 introduces a form of argumentation frameworks abstracting away (a restricted form of) IBIS graphs, and Section 3 defines our approach for the quantitative evaluation of arguments in these frameworks. Section 4 studies some formal properties of our approach, and Section 5 gives formal comparisons with two traditional non-numerical argumentation frameworks, namely abstract (Dung, 1995) and bipolar (Cayrol & Lagasquie-Schiex, 2005a) argumentation frameworks. Section 6 describes an implementation of our approach as an extension of designVUE, and Section 7 illustrates its application in three engineering case studies. Section 8 discusses related work, and Section 9 concludes.
The paper expands the work in Baroni, Romano, Toni, Aurisicchio, and Bertanza (2013) in several ways, notably by studying the properties of our proposed method for the quantitative evaluation of debates (Section 4), by considering the formal relationship with two traditional non-numerical argumentation frameworks (Section 5) and by developing one additional case study (Section 7.3). The latter amounts to revisiting a well-known design problem in the engineering design literature (Ulrich & Eppinger, 2004) and comparing it with standard decision techniques used for this problem, namely decision matrices (Pugh, 1991).
Engineering design is often described as an information-processing activity based on problem-solving within the constraints of bounded rationality (Simon, 1996; Simon & Newell, 1971). It consists of decomposing an initial problem into a range of sub-problems, proposing and assessing partial solutions, and integrating them as to satisfy the overall problem. This process is collaborative and often involves communication between non-co-located engineers. The development and communication of design solutions require engineers to form and share design rationale, that is, the argumentation in favour or against proposed designs.
These aspects of the engineering design process have led to the development (Kunz & Rittel, 1970) and subsequent investigation (Buckingham Shum & Hammond, 1994; Fischer, Lemke, McCall, & Morch, 1991) of the issue-based information system (IBIS) method, a graph-based formalisation of the decisions made during a design process along with the reasons why they were made. The IBIS method envisions a decision-making process where problems (or issues) are given solutions (or answers) after a thorough debate involving technical, economical, life, environmental and safety considerations. It also provides means to actively develop, communicate and record the reasons (or arguments) in favour or against the options explored during the design process. Initially, IBIS was conceived purely as a conceptual information system and its first implementations were paper-based and totally operated by hand. However, over time several software tools supporting editing and visualisation of IBIS graphs have been developed, for example, Compendium, DRed and design Visual Understanding Environment (designVUE) (e.g. see Aurisicchio & Bracewell, 2013; Buckingham Shum et al., 2006). These IBIS-based tools, including designVUE, which was selected as a starting point for this research, still leave to the users the burden of actually deriving any conclusion from the argumentative process and, eventually, making a decision. This is a task that, depending on the structure of the graph, may not be trivial.
This paper describes the outcome of collaborative research, involving experts of engineering design and argumentation theory, undertaken to overcome the limitations of standard design tools in general, and designVUE in particular. The ultimate goal of this research is to support engineers by providing them with a visual tool to automatically evaluate alternative design solutions and suggest the most promising answers to a design issue, given the underlying graph structure developed during the design process.
Since one of the main features of argumentation theory is evaluating arguments’ acceptability (e.g. as in Cayrol & Lagasquie-Schiex, 2005a; Dung, 1995) or strength (e.g. as in Cayrol & Lagasquie-Schiex, 2005b; Evripidou & Toni, 2012; Leite & Martins, 2011; Matt & Toni, 2008) within debates and dialogues, we have singled it out as a promising companion to engineering design to achieve our research goal. For this application area, conventional notions of ‘binary’ acceptability (e.g. the notions in Dung, 1995), sanctioning arguments as acceptable or not, are better replaced with notions of numerical strength, as the latter are more fine-grained and allow to distinguish different degrees of acceptability.
This paper presents both theoretical and practical results. On the theoretical side, we propose a formal method to assign a numerical score to the nodes of an IBIS graph, starting from a base score provided by users. On the practical side, we describe the implementation of this method within designVUE and its preliminary evaluation in the context of three case studies.
The paper is organised as follows. Section 1 gives the basic notions concerning IBIS and the necessary background on argumentation theory. Section 2 introduces a form of argumentation frameworks abstracting away (a restricted form of) IBIS graphs, and Section 3 defines our approach for the quantitative evaluation of arguments in these frameworks. Section 4 studies some formal properties of our approach, and Section 5 gives formal comparisons with two traditional non-numerical argumentation frameworks, namely abstract (Dung, 1995) and bipolar (Cayrol & Lagasquie-Schiex, 2005a) argumentation frameworks. Section 6 describes an implementation of our approach as an extension of designVUE, and Section 7 illustrates its application in three engineering case studies. Section 8 discusses related work, and Section 9 concludes.
The paper expands the work in Baroni, Romano, Toni, Aurisicchio, and Bertanza (2013) in several ways, notably by studying the properties of our proposed method for the quantitative evaluation of debates (Section 4), by considering the formal relationship with two traditional non-numerical argumentation frameworks (Section 5) and by developing one additional case study (Section 7.3). The latter amounts to revisiting a well-known design problem in the engineering design literature (Ulrich & Eppinger, 2004) and comparing it with standard decision techniques used for this problem, namely decision matrices (Pugh, 1991).
Date Issued
2015-01-02
Date Acceptance
2014-10-21
Citation
Argument and Computation, 2015, 6 (1), pp.24-49
ISSN
1946-2166
Publisher
Taylor and Francis
Start Page
24
End Page
49
Journal / Book Title
Argument and Computation
Volume
6
Issue
1
Copyright Statement
© 2015 Taylor & Francis
Downloaded by. This is an Accepted Manuscript of an article published by Taylor & Francis Group in Argument & Computation on 09 Feb 2015, available online at: http://www.tandfonline.com/10.1080/19462166.2014.1001791
Downloaded by. This is an Accepted Manuscript of an article published by Taylor & Francis Group in Argument & Computation on 09 Feb 2015, available online at: http://www.tandfonline.com/10.1080/19462166.2014.1001791
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000364161400004&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Subjects
Science & Technology
Technology
Computer Science, Artificial Intelligence
Computer Science
argumentation
debate
design rationale
Issue-Based Information System (IBIS)
decision support
ACCEPTABILITY
BIPOLARITY
Publication Status
Published