Lost in Translation: Examining the Complex Relationship Between Prototyping and Communication

Prototyping is critical to engineering design education [1,2] and the act of prototyping has been shown to improve concept generation [3], accelerate the development of tacit knowledge [4], and increase final product quality [5]. In design, inaccurate mental models can often be catastrophic [6]. Prototypes have the ability to improve the accuracy of mental models of design concepts and engineering systems held by novice designers [7,8]. Recent work has sought to understand the manner in which prototypes affect students' conceptions, with particular focus on solution ideation [9,10] and fixation [6,11] during the early stages of the engineering design process. Yet, reflective practice, or the act of evaluating one's actions and decisions, forms a central part of design practice and can be an essential learning tool for novice designers [12,13]. While past work has highlighted the efficacy of physical artifacts in stimulating reflection in designers [14], this work extends prior research by investigating how the construction of a physical artifact, such as a prototype, shapes novice designers' recollections of, and reflections on their design decisions.

Prototypes can also mediate communication with the ability to facilitate and improve user feedback [15] and build stakeholder buy-in, particularly for risky concepts [16]. Novice designers, however, often struggle to use prototypes as effective communication tools [17,18]. Lauff et al. [17,19] demonstrated that practitioners leverage prototypes as communication tools to facilitate both internal (within the design team) and external communication (with stakeholders/users). This finding was in sharp contrast to the practices of novice designers, who often fail to see the ability of prototypes as being effective communication tools [17]. One of the main objectives of engineering design education is to provide students with authentic experiences that prepare students for engineering practice [20], yet typical engineering communication curricula focuses on the development of technical writing skills via laboratory and project reports. The engineering design process, however, comprises multiple modes of formal (reports, memos), and informal (team meetings, stakeholder interviews) communication. To encourage the development of informal communication skills, we must first understand how novice designers use prototypes as communication tools.

To address the concomitant gaps in the literature and build foundational knowledge that serves engineering education, this work aims to investigate the complex relationship between prototyping and communication. Specifically, we seek to gain insight into the communication and prototyping practices of novice designers, and the rhetorical patterns that emerge when novice designers rationalize their design decisions in the context of prototyping tasks.

The remainder of this paper is structured as follows. First, literature from the areas of communication, prototyping, and design cognition are reviewed, and motivating gaps in research and driving research questions are identified. Following this, details of the controlled study, data collection methods, and analysis are described. Lastly, key findings from these analyses and the relevance of this work within the engineering design and engineering design education fields is presented.

Design is by nature highly collaborative [21,22] and multi-disciplinary [23], such that effective communication is necessary to ensure cohesion among teams and long-term project success [24–26]. Effective communication, paired with open communication channels between teams and across disciplines, can additionally aid designers in diagnosing problems or flaws early in the design process [27]. Further, not only is communication within design teams critical to project success, but effective communication and engagement with end users allow designers to create successful products in the market [28]. Given the exceedingly heterogenous nature of the design process, various researchers have proposed engineering design existing as a socio-technical system—a complex system where social, human, and organizational factors co-exist and interact with technical elements [29]. In such complex socio-technical environments, effective communication can assist designers in tracing design decisions and sharing knowledge with one another [30].

Prior research has studied how the exchange of information in the design process allows designers to navigate the problem space more effectively [31] and reflect on design decisions [32]. When exchanging design information, designers often rely on physical media, such as drawings and sketches, to communicate mental models of design concepts [33]. While these media communicate the mental model of the designer held at a particular instant in the design process, or “what” the design is, sketches and drawings on their own do not communicate the reasoning behind the creation of the designs [34]. Designers often pair sketches and drawings with textual or verbal descriptors to rationalize and explain design decisions or concepts [35].

An understanding of rationale behind design decisions is critical to the design process and provides context for understanding the possibilities in the design space, the trade-offs evaluated, and the alternatives that led to the design outcome [36]. During the design process, requirements are translated to product specifications, and an understanding of design intent provides understanding for how this evolution takes place [37]. The effective documentation of design rationale has been shown to aid the use of information from previous design work [38], facilitate learning [36], and improve an understanding of the design space [39].

Consolidating the cited literature, we note that while the communication of design information is pertinent to the success of the design process, it is crucial that designers effectively communicate the rationale behind design decisions, or “why” an object was designed the way it was. Prior research has largely focused on the development of systems to aid designers in capturing rationale during the design process [40–42]. However, limited work has been done in exploring how, or the manner in which designers communicate the rationale behind their design decisions and solutions [43].

In his seminal work “The Reflective Practitioner,” Schön [12] argues that design is a highly reflective activity. When solving ill-defined problems in engineering design, designers iteratively frame and reframe the design problem, move toward a solution, and upon the formation of a design outcome, evaluate the moves that led to the design outcome [12]. The metacognitive process of reflection is a key element of the design process, as it encourages the designer to assess the rationale behind design decisions [44], evaluate assumptions made [45], and mitigate cognitive biases [46]. Christiaans and Venselaar [47] showed that through reflection, designers are able to gain greater knowledge about design actions and control over the design process, subsequently increasing the creativity of design outcomes. Reflection also plays a critical role in communication, as it is designers' perceptions of the design activities performed and decisions taken during the design process that shape their narratives of the design process [48,49].

Reflective practice plays a key role in the learning trajectory of designers. Experts and novices are known to fundamentally use reflective practice in different ways [12]. While experts use reflection regularly in practice to inform their design decisions and actions [12], novices tend to perceive reflection as more effortful, as it comes less automatically [50]. Schön [51], however, argues that reflecting on design actions is what leads to greater competencies and expertise among novice designers. By evaluating their past design actions, designers may able to identify successful and unsuccessful strategies, enabling them to effectively plan for future design activities [52]. Reflection on design actions allows novice designers to build useful heuristics and strategies they may use to develop expertise in design [47,53]. Highlighting this critical role of reflection in design practice and its contribution to the learning process, prior research has dedicated efforts into the development of strategies to stimulate reflection in novice designers [54]. However, little is known about how novice designers identify useful heuristics and build design knowledge through the process of reflection. This knowledge formation process is important to understand, as it may help us discern how designers transition between different levels of expertise.

Reflection in design may be stimulated by the externalization of thought during the design process [55,56]. Schön [57] describes design as a “reflective conversation with the materials of the situation” and notes how designers use external representations to form and evaluate their own subjective judgments of the design outcomes created. As physical manifestations of mental models held by designers, prototypes enable the designer to discover new knowledge [58,59], identify unknowns in the design process [60], and aid in the generation of new concepts [3]. Lim et al. [14] argue that prototypes, while stimulating reflection, assist the designer in reframing the problem and identifying new alternatives in the design space. While prior research has studied the prototyping practices of novice designers in relation to design fixation [6,11], risk-taking behaviors [61], and the intentions behind prototyping [18], there is a dearth of literature investigating how novice designers reflect on their prototyping practices. Hence, in this paper, we seek to extend Schön's work by understanding how the act of creating a physical artifact, such as a prototype, influences a novice designer's reflection on and recollection of design decisions and actions.

The process of reflecting on and evaluating a design solution is not always reflexive, and researchers have proposed a myriad of tools and strategies that would better foster reflection among designers. Specifically, researchers have emphasized the utility of an argumentation-based approach in fostering reflection among designers [62], and how it can be leveraged to gain insight into the design process used by designers [63]. Scholars have stated that a parallel can be drawn between what Schön calls “action” and “reflection” in reflective practice, and “construction” and “argumentation” in design, where designers evaluate the reasoning behind their problem-solving process following the construction of an artifact [32,62]. Engaging in the act of argumentation would require the designer to reason about the design problem and its solution, hence requiring them to reflect on why the design outcome is the ideal solution to the problem, and how the design outcome was materialized. Andiliou and Murphy [64] found that when novice designers were tasked with evaluating their design outcomes through argumentation, they tended to evaluate the effectiveness of their design outcomes in a more reflective and critical manner. Hence, in this work, we seek to use an argumentation-based approach to stimulate reflection among novice designers following a prototyping task.

Boundary objects are objects with continuous identities that can be easily adapted to suit a number of heterogenous perspectives [65], and the use of boundary objects by designers has been shown to support communication across disciplinary boundaries [66]. Physical and digital artifacts, such as prototypes, act as boundary objects within design and have been linked to successful design outcomes due to improved inter-team communication [67–69]. As manifestations of design concepts and ideas, design artifacts have the capability to prevent misunderstanding and enable communication during the design process, particularly in situations where words cannot effectively convey necessary information [70]. In addition to communicating information about the speculated functioning of a design, artifacts also convey details about the processes and methods used in the development of the design [71].

As boundary objects, prototypes act as a medium between internal and external actors within the design process and allow designers to communicate essential product features. The ability of prototypes, both digital and physical, to be used as communication tools has been explored by a number of researchers [72–74]. Lim et al. [14] studied how prototypes can be leveraged as effective tools to communicate rationale for design decisions within design teams and to various stakeholders. In an ethnographic study of professional design teams, Gerber and Carroll [75] found that the construction of low-fidelity prototypes allowed team members to communicate ideas more effectively, helped them generate new concepts, and increased overall group efficacy. With the advent of crowdfunding platforms such as Kickstarter, researchers have also explored how designers leverage such platforms to communicate to a large base of stakeholders [76,77]. Researchers have stressed the need for prototypes during these crowdsourcing campaigns to support the exchange of information between external audiences and design teams [78], as communicating with prototypes may prove beneficial for product developers during these fundraising opportunities [79].

Prior research suggests that novice designers often struggle to perceive prototypes as effective communication tools [17,18]. Deininger et al. [18] interviewed 16 engineering students following the completion of a project-based capstone design course. In these interviews, only three students perceived prototypes as being effective communication tools. In a qualitative study of professional and student engineers, Lauff et al. [17] found that 51% of the professionals interviewed saw prototypes as tools to aid communication, while only 8% of students echoed this sentiment. These studies not only highlight the inability of novice designers to view prototypes as effective communication tools but also emphasize the well-known dissociation between prototyping practices of novice and professional designers [80].

Synthesizing the above research, there exists a complex relationship between prototypes and communication [43]. While prototypes have the ability to stimulate reflection and help designers communicate critical information, such as the rationale behind design decisions, novice designers struggle to see the utility of prototypes in being effective communication tools [17,18]. Yet, little research has been done in understanding how novice designers currently use prototypes to communicate and rationalize design decisions. In response to these concomitant gaps in the literature, this work investigates how novice designers communicate and reason their design outcomes and design decisions at the conclusion of a prototyping task. Specifically, we seek to answer the following research questions:

1. How does the construction of a prototype influence novice designers' reflection on their design decisions and perception of design outcomes?

2. What communication patterns do novice designers use to justify design decisions during a prototyping task?

To answer these research questions, a study was conducted with 45 undergraduate students from a large, public, mid-Atlantic, Research 1 University. The study was conducted in accordance with the procedures dictated by the Institutional Review Board (IRB). This section details the methods used.

The participants in the study were 45 undergraduate students (21 female, 24 male) enrolled in a project-based, introductory, first-year engineering design course. Students were recruited from Engineering Design 100; all engineering students at the Pennsylvania State University are required to enroll in this first-year engineering design course that teaches the fundamentals of engineering design theory and methods and involves the completion of multiple design projects. Students were informed that their participation and performance in the design task was voluntary and would not in any way contribute to or detract from their grades in the class. Students participated in the design task individually, and there was no collaboration between students.

In this study, we chose to adapt and apply the experimental design outlined in the work by Neeley et al. [81], who studied the effects of rapid prototyping on the design outcomes of novice designers, and examined students' perceptions of their design outcomes and processes. In their work, Neeley et al. [81] found that when novice designers were tasked with building multiple prototypes under a time constraint, participants expressed dissatisfaction with both their design outcomes and the time constraint of the design tasks. However, their final design quality was found to be superior than their counterparts who constructed a single prototype within the same time frame. This key finding by Neeley et al. [81] points to the possibility that there may exist a discrepancy between how novice designers perceive their performance in prototyping tasks and their actual design performance. As our work seeks to gain a deeper understanding of the factors affecting novice designers' perceptions of design outcomes and decisions following a prototyping task, the experimental outline from Neeley et al. was adapted and applied to better suit our research goals.

The study was conducted in five phases, as seen in Fig. 1. Prior to the start of the study, a general overview of the study was shared with students and any question were answered. Following this, under the IRB guidelines, consent was secured from the students. Students were then given 10 min to complete a pre-survey containing questions pertaining to general demographic information, as well as the Engineering Design Self-Efficacy Scale [82], and the Tinkering Self-Efficacy Scale [83]. While not the focus of this work, the results of the analyses of the data obtained using these scales can be found in the prior work by Menold et al. [43].

Once students completed the survey, they were introduced to the design problem, provided below:

Students were then given 10 minutes to brainstorm and sketch as many solutions as possible on ideation sheets. At the conclusion of this stage, students were informed that they would prototype one design in the coming stage and were told to select a final design idea. Once the ideation phase was complete, students transitioned to a university makerspace. All the students were physically present in the same makerspace during the prototyping task. The makerspace was equipped with standard shop equipment, including a bandsaw, table saw, power tools, and hand tools; all students were trained in the shop prior to entering the workspace in accordance with university safety requirements. Students were given 45 minutes to prototype their selected tower designs. Students were given the choice of building their towers with either one 2′ × 2′ sheet of plywood, or two 2′ × 2′ sheets of cardboard. Both materials were in adequate supply; 23 students selected cardboard and 22 selected wood. Apart from this, they also had access to a variety of other building materials and tools, including but not limited to, hacksaws, hot glue guns, tape, utility knives, rulers, nails, hammers, and a number of commonly found hand and power tools.

At the end of the 45-minute prototyping phase, students were asked to write their student ID on their towers and store them. Pictures of each tower were taken by the research team and were stored corresponding to the students' university ID numbers. In the final phase of the experiment, students participated in an online post-task survey. The text boxes in the electronic survey did not have any character limits. The survey asked students to reflect on their prototyping experiences via four open-ended questions:

1. Why did you choose the materials and tools that you used?

2. How did your design change throughout the prototyping process? Please list any experiences, thoughts, or behaviors that influenced your design along the way.

3. Please argue why your tower has the best combination of aesthetics, height, and strength.

4. Is there anything else that you would like to tell us about your design experience that helped you succeed?

These questions were deliberately phrased to elicit students' justifications in describing their problem-solving process and towers' features and to encourage students argue for why their tower had the best design, hence prompting them to reflect on their design solution and the decisions that led to its creation. Once the students completed the survey, the research team measured and recorded the heights and load bearing capacities of the final towers constructed by the students. Weights were placed incrementally on the towers from 0.25 lbs to 20 lbs at increments of 0.25 lbs until the tower collapsed or a load of 20 lbs was reached.

In pursuit of our research questions, a comprehensive codebook was developed to analyze students' responses to the open-ended questions listed earlier. The codebook comprised three sections. The first two sections contained coding schemes that were used to answer the first research question: to understand how students perceived and evaluated their design decisions and design outcomes. The third section of the codebook contained a coding scheme that was used to answer our second research question: to understand the communicative patterns present in students' justifications. It should be noted that two unique sets of coders analyzed the data for each research question, as the analyses for the two research questions were performed at different time points of the research project.

First, we will introduce and describe the first two sections of the codebook and the associated analysis, before discussing the third: The first, shown in Table 1, identifies the subject of students' justifications (i.e., “what” objective of the design problem the students were justifying). The second section of the codebook, shown in Table 2, identifies the manner in which, or “how” the students framed these justifications in the context of the objectives. These two sections of the codebook were used to analyze students' arguments for why their towers were the best combination of height, strength, and aesthetics.

The first section of the codebook, as shown in Table 1, contains themes related to the objectives of the problem statement (height, strength, and aesthetics), along with example statements for each code. The first section of the codebook was used to determine if students had addressed each of the three objectives in their arguments.

Next, the second section of the codebook, as shown in Table 2, was used to understand the valence, or the affect with which students evaluated their design outcome and how it met each objective of the problem statement. We were interested in characterizing the language with which the participants justified their designs, especially as they pertained to the final design.

The valence of each statement with relation to each of the design objectives was coded using the codebook shown in Table 2, where each statement was categorized as either “Positive,” “Neutral,” “Negative,” or “Not Addressed,” depending on how a specific design objective was communicated in students' arguments. Two coders coded approximately 25% of the responses to the question “Please argue why your tower has the best combination of aesthetics, height, and strength.” using the first two sections of the codebook (Tables 1 and 2) to coherence. Inter-rater reliability was calculated using Cohen's κ [84], and was found to be ≥ 0.74. Prior literature suggests a κ value of 0.7 or greater as being indicative of substantial agreement between raters [85]. Figure 2 provides two examples for how the coding process using the first two sections of the codebook was performed.

For our second research question, we used qualitative methods via the constant comparative method [86] using an abductive paradigm [87] to analyze the open-ended responses from students. An abductive approach allows researchers to incorporate prior literature and theory as the basis for a preliminary codebook (unlike pure grounded theory [88]), while also being receptive to data that does not fit into previously determined codes. Specifically, the prior literature that was referenced related to the areas of rhetoric and argumentation. Argumentation, or the act of reasoning to support an idea or claim, is typically fraught with a number of rhetoric devices that the communicator uses to shape their arguments and persuade an audience [89]. Prior research has sought to identify how these rhetorical devices, or indicators, are utilized in a number of different contexts, such as academic writing [90], advertising [91], and political discourse [92]. The domain closest in similarity to our work is that of scientific writing—a form of “persuasive writing” where the author convinces the reader to recognize and accept the warrants and claims stated by them. Hyland [93] analyzed 28 research articles from four academic disciplines to identify rhetorical devices used by authors when communicating scientific findings to their disciplinary communities. As communication in design is also often argumentative in nature [40], it is likely that such rhetorical indicators often shape designers' arguments when communicating design outcomes and rationale behind design decisions. As such, the codebook was informed by the works of Hyland [90,94] and Brown [95], as explained in greater detail later.

The final section of the codebook, shown in Table 3, identifies the rhetorical patterns present in students' justifications of their design decisions. This section of the codebook was used to analyze students' responses to the open-ended questions through thematic analysis. The communication patterns identified included hedging, boosting, and satisficing. Hedging, as explained by Hyland [90], is a linguistic practice often used in academic writing, when communicators intentionally use vague language to evade making affirming statements. Boosting, on the other hand, is the use of superlative language to amplify one's success in a particular task [94]. Satisficing, a term coined by Brown [95], involves a combination of sufficing and satisfaction in their decision-making process—a “good-enough” approach where a satisfactory solution is selected rather than an optimal one.

Coding of responses using the final codebook was done by five trained raters, and coding was performed to agreement; raters independently coded a randomly selected subsample of responses, coded, and discussed any disagreements as a team prior to coding responses independently. This process occurred twice to ensure raters were coding in similar manners. As coding was performed to agreement, no inter-rater reliability was calculated.

The design task presented to the students involved them designing a tower that met three distinct objectives: to be as tall as possible, to have weight bearing capacity at its highest point, and to be aesthetically pleasing. They were told that these objectives can be visualized as the customer needs, and over the course of the design task, no explicit hierarchy or relative importance of these objectives (or customer needs) was conveyed to students.

Our first research question aims to understand how the construction of a prototype influenced novice designers' reflection on their design decisions and perception of their design outcomes. To do so, we analyzed the arguments of the students using the first two sections of the codebooks (Tables 1 and 2), to identify how students reflected on their design outcomes and decisions that led to the creation of the artifact. Our second research question aims to understand the linguistic themes emergent in students' justifications. The identification of these patterns was done using the third section of the codebook (Table 3), and uncovers the underlying tone present in students' responses. This section reviews the results from our qualitative and quantitative analyses in detail. All quantitative analyses were performed on R Cran 4.0.1, and all p-values < 0.05 were considered to be significant.

RQ1: How does the construction of a prototype influence novice designers' reflection on their design decisions and perception of design outcomes?

To understand how the students' evaluated their design decisions and perceived their design outcome, we analyzed and coded the students' responses to the question “Please argue why your tower has the best combination of aesthetics, height, and strength.” The question was deliberately phrased in this manner to elicit students' argumentation strategies when describing their problem-solving process and towers' features. For the purposes of constructing their arguments, students would have to evaluate their prototypes and explain the rationale behind their design decisions that led to its creation, hence stimulating them to reflect on their actions during the design task.

We first analyzed students' arguments using the codebook shown in Table 1. This was done to analyze how students reflected on and evaluated their design outcomes, which would subsequently shape their arguments for if/why their designs were optimal. Ideally, if students had approached the problem space in an unbiased manner and put all the objectives at equal importance, all three objectives should have been addressed in their justifications. However, only 24 out of the 45 students justified their design decisions in the context of all three objectives. Of the 45 participants, three failed to address any objective in their justifications, seven students addressed only one objective, and 11 students addressed two objectives. Because nearly half of the participants (21 students) failed to address all three objectives in their arguments, we sought to understand whether students perceived certain objectives to be more important and to use their language patterns to understand these perceptions.

First, the frequencies of each code were captured. Our analysis revealed that 91% of students communicated the strength of the towers, and 71% addressed the height of the towers in their responses. In comparison, only 62% of students communicated features contributing to the aesthetics of their towers.

Among the students who addressed strength in their arguments, only two of the students felt that they had not successfully met the goal of building a strong tower. Hence, not only did 91% of students address strength, but most students addressed strength in a positive manner. We also observed that out of the seven students that addressed only one objective, none of the justifications pertained to either height or aesthetics—all seven students chose to justify only the strength of their towers. Among the 18 students who addressed only one or two objectives, 15 failed to communicate about the aesthetics of their towers in their justifications. It is unclear as to why students focused their reflections on the strengths of their towers and omitted other design objectives, as noted by the quote “I tried to design something where its center of gravity could hold the most weight which is where the weight will actually be placed.” We hypothesize that following the design task, students may have chosen to only communicate objectives that they had successfully accomplished during the prototyping task. This hypothesis will be contextualized in the Discussion section.

We note that out of that six students who justified the height of their towers in a negative manner, all had positive perceptions about the strength of their towers. An examination of the responses from students indicates that students may have placed the objective of strength at a higher precedence, potentially at the cost of tower height:

A similar trend was found among students who had negative perceptions about the aesthetics of their towers. Students stated that the aesthetics of their towers were “sacrificed” to successfully meet other objectives, and how they valued the objectives of height and strength over aesthetics. Consider the quotes below:

From these observations, we infer that not only was the strength of their towers often the focal point of students' self-evaluations, but corresponding rationale tended to be expressed with a positive affect. We also observed that when students evaluated the heights or aesthetics of their towers negatively, their associated rationale tended to be related to the importance of meeting the objective of strength. We hypothesize that this may be related to how students framed the design problem during the design task (as noted by one student who stated “[I] kept the main qualities [of] stability and strength in mind at all times”). This hypothesis will be later contextualized in the Discussion section.

In pursuit of our first research question investigating how students' reflections on their design task, stimulated by their prototypes, related to their actual performance in the design task, we analyzed the relationship between the valence of students' reflections in the context of the three objectives of the design task (i.e., height, strength, and aesthetics), and the actual heights and weight bearing capacities of their tower prototypes. We wanted to discern if students' conceptions accurately translated into their performances in the design task, and subsequently their arguments for their towers.

A series of Kruskal–Wallis H-tests [96] were run to determine if students who communicated performance in relation each of the design objectives differently (positive, negative, neutral, or not addressed), also had towers of different heights and weight bearing capacities. The Kruskal–Wallis H-test was selected since the data violated the assumptions of the parametric one-way anova. The Kruskal–Wallis H-tests revealed no significant difference between the heights of the towers of students who expressed different sentiments (or those who did not communicate) about the heights of their towers (χ²(3) = 2.958, p = 0.39). Similarly, we found no significant difference between the strength of the towers of students who expressed different sentiments (or those who did not communicate) about the strengths of their towers (χ²(3) = 4.34, p = 0.23). These results imply that there was no significant difference in the heights or strengths of towers built by students who had positive, negative, or neutral sentiments about (or those who did not communicate) height and strength. Interestingly, we found a significant difference between the heights of the towers of students who expressed different sentiments (or those who did not communicate) about the aesthetics of their towers (χ²(3) = 11.45, p < 0.01). Post hoc Wilcoxon signed-ranked tests [97] revealed that students who expressed positive sentiments about the aesthetics of their towers had significantly taller towers than those who did not address aesthetics at all (p = 0.014).

RQ2: What communication patterns do novice engineers employ to justify design decisions during a prototyping task?

Students' open-ended responses collected from the post-survey were analyzed to identify the rhetorical trends within participants' justifications of their design decisions. Students often used hedges (intentionally vague language to soften bold claims) and boosters (elaborate statements to boost success) in their design rationale. While aesthetics were addressed least often by students, when they were addressed, the argument often took the form of a lofty statement. When asked why their tower was the best combination of the objectives of the design challenge, one student wrote:

Some students also made far-reaching analogies to argue for their success in building in aesthetically pleasing tower. One student said that their tower had the potential to be aesthetically pleasing because of its “visible columns that create a Romanesque appearance,” while another stated that they had one of the tallest towers, and “it strongly resembled a stick. Sticks are a part of nature therefore making it quite aesthetically pleasing.” Students also used hedging statements to avoid making definitive statements about either the success or failure of their towers. While justifying why their tower was the best, one student said that their tower had “almost maximum height for the time and materials provided.”

The qualitative analysis also uncovered elements of satisfice (expressing trade-offs between objectives and accepting “good-enough” solutions) present in a number of students' justifications. Considering that this was a timed design challenge, a number of students saw success in the context of one or more objectives possible only when compromising on another. For example, one student said that while they thought their tower looked good, “some of the aesthetics were sacrificed for height and strength.” Some students also communicated the existence of a “good-enough” solution to the design problem, particularly when dealing with the objectives of height and strength. While discussing how they had succeeded in building a tower that met all the objectives, one student said that “While reaching a height of slightly over 2 feet and having a simple, nice look, it is still able to hold weight.” The response here indicates the student being satisfied with what from their perspective was a “good-enough” solution to the design problem.

Further, some students acknowledged that they did not succeed in meeting the design objectives, but typically these admissions were followed by citing success in a different objective; compensating for their failure in one aspect by claiming success in another. For example, one student argued that their tower was the best design because “While my tower was of medium height relative to may [sic] classmates, the sturdy base, strong supports, and unique shape make it both visually pleasing and strong.” Here, the student tried to compensate for the lack of height by citing features that contributed to the strength and aesthetics of their tower. This pattern of compensation was emergent in a number of students' responses, particularly with students who addressed all three objectives of the problem statement. Of the 25 students who provided rationale for all three objectives, 12 used elements of satisfice, compensation, or both in their justifications.

Our abductive coding schema also identified newer, emergent qualitative themes present in students' justifications. The themes that emerged were those of “Self-imposed design criteria,” “Affective Decisions,” and “External Limitations.” Many of these self-imposed criteria imposed by students pertained to the aesthetics of their towers, and these self-imposed criteria were often used to strengthen their arguments for why their design was the optimal solution. For example, one student said “It is simple, but does have an aesthetic aspect to it. I did not want to go too detailed because of the risk of the tower looking too “messy” because of the time constraint.” While aesthetics was an objective of the design challenge, a “simple” or “more detailed” design was not part of the design problem or communicated as explicit evaluation criteria. The tendency of using self-imposed design criteria to strengthen their arguments for the aesthetics of their towers is not surprising. Aesthetics is a highly subjective measure, and students may have had their own unique perceptions as to what constitutes an aesthetically pleasing tower.

What is interesting to note, however, is students' use of self-imposed design criteria to justify their selection and use of materials and tools during the prototyping task. For example, one student felt they had accomplished in building a successful tower because “the minimalist design would make use of the material best, allowing much left over for additional ‘renovations.’” Having extra material for renovations was not a part of the original design criteria. As another example, when asked why they chose the materials they did to build their tower, one student said “I wanted to use as little glue as possible. So I cut my pieces so that I could fit them together instead of glue them together.” The use of glue was not a constraint in the design task, and students were allowed to use as much glue as they needed.

The theme of affective decisions was an emergent communicative pattern when students were asked to provide rationale as to why they selected the specific tools and materials they built their tower with. In their justifications, students often cited their skills and familiarity with using certain tools and materials as being the primary driving factors in their design decisions. For example, one student said that they chose cardboard because it was “quick and easy and did not require me learning a new tool and worrying about safety.”

When justifying their lack of success in meeting one or more of the design objectives, students often cited factors outside of the technical limitations of the design task. An external factor that came up frequently in students' justifications was the lack of time. At the start of the design task, students were informed about and were aware of the duration of the design task. When asked to justify why they selected the materials and tools they chose to work with, some students clearly articulated in their rationale that they “understood choosing wood would mean waiting in line for a saw and more difficult construction,” and that they “knew we would be short on time.” The same students, however, when asked how their prototype had evolved over the course of the design challenge, attributed their lack of success to not having enough time to finish their prototypes. One student said they were “forced to improvise” due to the lack of time, while another said that they had to remove certain design features from their tower “because there wasn't enough time to feasibly add those parts.” Additionally, some students tended to use passive voice when citing these external limitations. For example, one student said “Some of the measurements weren't exact and the lines weren't straight so I ended up having to adjust and re-glue things and in the end scrap the design all together because I didn't have enough access to a hot glue gun.” While the external limitations cited here were measurements and glue gun access, the student failed to take responsibility for the inaccurate measurements and lines, and instead used passive voice to distance themself from their own actions and decisions.

This study aimed to answer the following research questions:

1. How does the construction of a prototype influence novice designers' reflection on their design decisions and perception of design outcomes?

2. What communication patterns do novice designers use to justify design decisions during a prototyping task?

The findings for our first research question indicate that when reflecting on their design outcomes, students' evaluations were focused on the strengths of their towers, and students largely felt that their towers successfully met this objective. Aesthetics, however, was the objective least addressed by students. Further, when students provided their rationale as to why their towers did not meet the objectives of height and aesthetics, students often stated that meeting the goal of a strong tower was their primary objective, and that they held the objective of strength at a greater value. It appears as though when evaluating their design outcomes, novice designers may have skewed the design objectives based on their own assumptions and biases to strengthen their arguments for why their towers were the optimal design solution. Considering that it is these evaluations of design outcomes and decisions that subsequently shape designers' narratives [48,49], it is possible that novice designers might emphasize certain product attributes more closely related to the design objectives prioritized by that designer, while neglecting other attributes. Such a practice may be problematic when these designers engage with users and stakeholders, as these users and stakeholders may have their own perceptions of which design objectives and needs are important [98].

While our results reveal that students disproportionately focused on communicating certain design objectives, what remains unclear is the stage at which novice designers shifted their focus onto certain design objectives rather than all three of them. We propose two possibilities. Our first hypothesis proposes that this shift may have occurred during the prototyping task. Our qualitative results revealed that some students saw strength as being the most important objective to meet, as seen in the quotes “[I] kept the main qualities [of] stability and strength in mind at all times” and “Finally it was of medium height, but I chose a sturdy build over pure height.” From these observations, we hypothesize that during the design task, given the multiple objectives in the design problem and constraints, students may have framed the problem in a way that would allow them to meet design objectives that they assumed to be of greater importance. Prior research studying the problem-solving practices of novice designers would support this hypothesis. Both Ullman et al. [99] and Ball et al. [100] found that novice designers tend to use a “satisficing” strategy when solving design problems, where suboptimal, satisfactory solutions are selected rather than solutions that optimize all the requirements of the design problem. Hence, it is plausible that participants in this study used a similar satisficing strategy, in that students may have seen a tower with a high weight bearing capacity as being an optimal solution given the multiple objectives and constraints of the design task. This might further explain why students' reflections were focused on the strengths of their towers. We acknowledge that the post task assessment of their prototyping task is not sufficient to conclusively state that some participants used this satisficing practice and reframed the design problem to focus specifically on the strength of their towers during the prototyping task. However, we believe our results warrant future work investigating the problem-solving strategies of novice designers when engaged in prototyping tasks for design problems with multiple, potentially competing design objectives.

Conversely, this shift in focus on certain design objectives may have also occurred post design task, when students were reflecting on their design outcomes, and arguing for why their tower had the best combination of height, strength, and aesthetics. We speculate that students may have been susceptible to the IKEA effect—a cognitive bias commonly identified in behavioral economics which states that individuals perceive physical objects constructed by them at a higher value than identical artifacts not constructed by them [101]. Prior research has highlighted how the IKEA effect leads to novices overvaluing their creations [101], and how this effect can bolster feelings of competence in the creator of the artifact [102]. Hence, applying the IKEA effect to our work, it is possible that the prototyping task may have led novice designers to attach greater perceived value to their design. This cognitive bias may have led them to communicate only the aspects of their design they were able to succeed in, in an attempt to indicate competence to both themselves and others [102].

Interestingly, when comparing the perceptions of students' performance to their actual performance in meeting the design objectives, we observed a relationship between valence when addressing aesthetics and the final height of student towers. We found that students with taller towers tended to justify aesthetics in a more positive manner. It is possible that students were affected by some social biases due to commonly held notions of a tower, (i.e., towers should be tall). For example, while justifying why their tower was aesthetically pleasing, one student said “The tall eloquent (sic) figure resembles the Eiffel Tower; therefore, it is by far the most beautiful of designs.” It is also possible that students may have been affected by the representativeness heuristic, where judgements about an outcome are made using limited evidence or criteria unrelated to the outcome [103,104]. Hence, while reflecting on and evaluating the aesthetics of their tower, students may have perceived the heights of their towers to be representative of aesthetics (i.e., tall designs are aesthetically pleasing and short designs are not). We do however, acknowledge that since the responses were collected at the conclusion of the experiment, it remains unclear as to whether students intentionally built a high tower with the assumption that it would also be aesthetically pleasing, or if their rationale was formed upon completion of the design task. Further, students also tended to make grandiose statements about the aesthetics of their towers when justifying their design decisions. We observed students making far-reaching analogies, some seemingly unrelated to aesthetics, to justify their success in building an attractive tower. These observed themes related to aesthetics could be due to the abstract nature of aesthetics itself. Unlike height and strength, which are based on quantitative measures, perceptions of aesthetics could vary from person to person. This makes it is quite possible for designers to base their judgments of aesthetics of products on hunches and guesses, rather than definitive measures [105].

The findings for our second research question revealed a number of linguistic themes present in novice designers' justifications of their design decisions. Our analyses find that students' rationales were often accompanied by elements of hedging, boosting, satisfice, and compensation. Pairing the findings from our two research questions weaves an interesting story about the nature of students' design rationale. In these arguments, students often alleviated the extent of their failures in meeting one objective by citing success in another, seemingly unrelated objective, and often relied on elements of satisfice and/or compensation to do so. Another troubling pattern that surfaced was the tendency of students to attribute their failures to external factors, rather than owning up to their design practices that led to these failures. Rather than taking ownership, students tended to distance themselves away from these failures. Students often reverted to using passive voice in their justifications, creating an impression that the decisions they made were imposed on them, and that they were forced into making the design decisions they took. Comparing our findings to those of Neeley et al. [81], the attribution of failure to the lack of time is a noteworthy observation. In Neeley et al. [81], students who developed multiple prototypes during the experiment expressed dissatisfaction with their design outcomes, and stated the time constraint as being a limiting factor during the design task. The attribution of their perceived failure to time, an external factor, is not unfounded, as it might be challenging for novice designers to build multiple prototypes within a short time period. However, in our work, we see that even when tasked with producing one prototype, students expressed time, or lack thereof, as being a key factor that affected their design outcomes. Noting this observation, perhaps the attribution to external factors has little to do with the nature of the design task and constraints. Hence, we posit that the attribution to external factors may have something to do with unsatisfactory design outcomes rather than the nature of the design task itself. We suspect that novice designers may have been affected by the self-serving bias, where positive outcomes are attributed to one's own efforts, and negative outcomes are attributed to external factors regardless of whether these factors truly played a role in the design process [100,106]. Future work should examine the role the self-serving bias plays in novice designers' design processes in more depth.

We believe the use of these communicative strategies may not be perceived well when students leave an educational setting, particularly since in industry, communication often takes place with the utilization of a physical design artifact [17]. This then calls into question the effectiveness of current teaching methods in engineering education that focus on communication skills, especially since it is these educational settings that prepare novice designers for authentic, real-world design problems. As highlighted earlier, much of communication curriculum in engineering education focuses on developing technical written communication skills through laboratory and project reports. However, students should also be equipped with the skills needed to informally communicate design knowledge with design team members, stakeholders, and users. Hence, we highlight the need for novice designers to be better equipped with the skills needed to effectively leverage prototypes as effective communication tools, as prototypes are key facilitators of communication in design [19], and are used by design practitioners as communication tools [17,107].

This study aimed at uncovering the complex relationship between prototyping and communication, and gaining insight into novice designers' reflections on design outcomes and decisions after the construction of physical artifacts. The results from our qualitative analyses indicate (1) the potential skewing of design objectives when novice designers reflect on their design outcomes and decisions post a prototyping task and (2) the presence of various linguistic patterns that novice engineers rely on when communicating design intent and justifying design decisions.

As with any study, there are a few limitations that should be noted and addressed in future work. First, the students in this study were predominately first-year engineering students, who, at the time of the study, had limited experience with the engineering design process. Future work should involve the analysis of communicative patterns of senior engineering students to gain insight into how these students, who have more training and are about to transition into industry, communicate design intent and outcomes. Another limitation of this study is the nature of the design challenge given to students. While this study involved students designing and constructing a prototype of a tower from either cardboard or wood, future work could explore the communication strategies used by novice engineers when working on longer, more resource-intensive, and more authentic real-world design problems. Further, the use of short answer format for questions is another limitation of the paper. The short answer format was selected to encourage students to succinctly communicate their justifications and clearly communicate which factors were more or less influential in their design processes. We acknowledge, however, that students may not have justified or detailed all design decisions due to the nature of the short answer format. Future work should use methods such as think-aloud protocols to gain richer insights into novice designer behaviors. Additionally, when asking students to argue why their tower was the best combination of height, strength, and aesthetics, it is possible that the framing of the question may have biased answers. In theory, the framing of the question may have caused some students who felt they were not successful in the design task, to, nevertheless, argue that their solution was optimal. We note, however, that several students clearly communicated the failure of their design to meet objectives and do not feel that this significantly impacted findings. A manipulation check should be conducted in future work to ensure this is the case.

Overall, this study highlights some potentially problematic practices of novice design engineers when constructing prototypes and using them as communication tools. It is imperative that engineering education better prepares novice designers for real-world design settings; settings where designers often communicate design narratives and justify their decisions using design artifacts such as prototypes. While prior work has established the role of prototypes in facilitating communication between stakeholders in real-world design settings [19], this is understudied in the context of engineering education. Prior research has highlighted the efficacy of structured interventions in improving the prototyping practices of novice designers [108]. Through this work, we highlight the need for the development of further such interventions and educational tools that better equip students with the skills necessary to excel in real-world design settings.

• This work is supported by funding provided by the National Science Foundation (NSF) (Grant No. 1916386; Funder ID: 10.13039/501100008982).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.