a method to generate and evaluate a creative … · · 2009-09-18students were required to create...

1 Copyright © 2009 by ASME

Proceedings of the ASME 2009 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference

IDETC/CIE 2009 August 30-September 2, 2009, San Diego, California, USA

DETC2009-87193

A METHOD TO GENERATE AND EVALUATE A CREATIVE DESIGN IDEA BY FOCUSING ON ASSOCIATIVE PROCESS

Eiko Yamamoto Department of Mechanical Engineering, Kobe University, 1-1 Rokkodai, Nada,

Kobe 657-8501, Japan

Futoshi Mukai Nagoya City Water works & Sewerage Bureau, 1-1-12 Tiyoda, Naka, Nagoya 460-0012, Japan

Nor Fasiha Mohd Yusof Department of Mechanical Engineering, Kobe University, 1-1 Rokkodai, Nada,


Toshiharu Taura Department of Mechanical Engineering, Kobe University, 1-1 Rokkodai, Nada,


Yukari Nagai School of Knowledge Science,

Japan Advanced Institute of Science and Technology, 1-1 Asahidai, Nomi 923-1292, Japan

ABSTRACT

In order to support creative design, we propose an approach composed of two methods for generating creative design ideas. The first method involves generating a candidate for a highly creative design idea by focusing on the notion of “polysemy.” The second involves evaluating the generated candidate by using a virtual network. In this study, we focus on concept generation, specifically, the process of synthesizing two concepts. Furthermore, we use design idea features for representations of the design idea. We attempt to create a set of design idea features that allows the realization of highly creative ideas. For verifying the validity of the first method, we confirmed that the estimated scores of originality of all the generated candidates extrapolated the score of the base design idea.

1 INTRODUCTION

Many studies have been conducted on concept generation, and the development of computer-aided concept generation support systems is being actively pursued. Many studies have focused on the notion of analogy as a method for generating design ideas, and many design-by-analogy methods have been proposed. For example, a method using mutation and analogy

was proposed by Gero and Maher in 1991 [1]. Other researchers have investigated a knowledge support system [2], a model-based design-by-analogy method [3], an expert system using analogical reasoning [4], a quantitative similarity metric for design-by-analogy [5], and a unique tool involving analogies from nature that can provide inspiration to designers [6].

Studies were also conducted on linguistic database-aided methods and analysis using design-by-analogy; Linsey et al (2008) focused on creating multiple linguistic representations of a design problem [7], and Chiu and Shu (2007) proposed a methodology that involved enhancing concept generation by using verbs as stimuli; that is, verbs would support concept generation [8]. In addition, Kolb et al. (2008) developed a tool for generating compelling metaphors for design [9]. The use of this tool enabled interaction among the designers during the metaphor generation process and helped obtain better metaphors.

Most of the supporting methods or tools evaluate the created design ideas by comparing them with past design ideas [10]. On the other hand, a methodology for generating creative design ideas by the meaning-structure modeling [11], which was recently proposed by Georgiev et al. (2008), uses an


evaluator to evaluate the effectiveness of designs generated by computer, by yielding an evaluation rating.

In this study, we develop a methodology for generating a candidate creative design idea and evaluating it. Generally, the evaluation of a design idea is performed after the idea is completed, by analyzing its features by using methods such as the semantic differential method. Creativity estimation at the early stage of generation of an idea would enable designers to create better creative design ideas.

For proposing creative design ideas, we focus on the notion of association. Concept association is a key notion in creative design and is performed during concept generation. Many of the concepts associated with a realized design idea can enhance the users’ impressions regarding it; in other words, this association corresponds to the evaluation of a design idea. Concept association in concept generation can enhance the thought process of the designer as well. Some research groups have mentioned and focused on this association [11, 12, 13].

2 PURPOSE AND APPROACH We propose an approach composed of two methods for

generating creative design ideas. The first method involves generating a candidate by focusing on the notion of polysemy. The second method involves evaluating the generated candidate using a virtual network.

In this study, we focus on concept generation, particularly, the process of synthesizing two concepts (hereafter referred to as base concepts). Furthermore, we use design idea features (explained later) as the representations of the design idea, and attempt to create a set of design idea features that may indicate a high level of creativity.

We first reanalyze the creativity of design ideas obtained in a design experiment [14]. Then, we develop a method for generating creative design ideas on the basis of the experimental results. Finally, we verify the validity of the method. Although we have already analyzed the design ideas by using virtual modeling to understand the nature of design creativity [20], we reconsider them as a method for evaluating design ideas in this study.

In our previous design experiment [14], 22 Japanese students were required to create a new design concept from two base concepts; in other words, the students were asked to carry out concept synthesis. Using the concepts of SHIP-GUITAR and DESK-ELEVATOR as the base concepts, they were required to sketch, explain, and list design idea features that could explain their design ideas. They came up with 20 design ideas for SHIP-GUITAR and 19 design ideas for DESK-

ELEVATOR. Eleven evaluators rated the creativity of their design ideas on a scale of 1 to 5, on the basis of their practicality and originality [15]. In this study, we analyze the design ideas for SHIP-GUITAR (Table 1) as an example of a design idea.

Although this concept-synthesizing process is an abstract process, we believe that this is important in engineering design. The concept-synthesizing process is found in an actual field. Empirically, the invention of the art knife―the first snap-off blade cutter―is an appropriate example of the use the concept-synthesizing process. The inspiration for this original idea stems from the synthesis of two concepts, namely, chocolate segments that can be broken off and the sharp edges of broken glass [16]. Thus, such abstract factors would be important in creating a new product.

3 DESIGN GOAL

3.1. Concept Generation In this study, we adopt the process of synthesizing two

concepts as the basis for generating creative design ideas. Concept-synthesizing process is a typical concept generation process wherein the design idea is newly created from given base concepts [17, 18].

3.2. Design Idea Features We use design idea features that are expressed by a single

word as a substitute for the design idea. Design idea features are concepts that can explain the design idea. Instead of the actual design process, namely, the concept-synthesizing process, we consider the process between the base concepts and design idea features as the design process (see Figure 1).

Figure 1. Design process in this study.

Table 1. Example of design idea.


4 METHOD 1: GENERATING A CANDIDATE OF A SET OF DESIGN IDEA FEATURES

In this section, we propose a method for generating a candidate of a creative design idea on the basis of the notion of polysemy.

4.1. Polysemy We have previously found that the number of concepts

associated with a realized design idea corresponds to its creativity [13]. However, the property of the design idea features that is related to the association is unknown. From a linguistic viewpoint, we assume that the polysemy of a design idea feature is related to the creativity of the design idea; this is because a design idea with highly polysemic features is expected to fulfill a rich meaning structure that enhances a user’s impression of it.

Polysemy is the capacity of signs such as words and phrases to have multiple meanings. This is a pivotal concept in social sciences such as media studies and linguistics. For example, the word milk is a noun, as in, “I drink a cup of milk every morning.” Milk is also a verb, as in “I tried to milk a cow.” Another example is the word wood, which may refer to the material that forms the trunks and branches of trees and also to a forest area.

4.2. Correlation between Creativity and Polysemy In this study, we examined the correlation between

creativity and polysemy of the design idea features. We estimated the number of meanings of all the words expressing the design idea features by using WordNet [19, 20], and calculated the average of the polysemy (the number of meanings) for each design idea. There were 20 design ideas. The averages were compared with the score of originality obtained in our previously described experiment. Then, we performed a correlation analysis and obtained a significant model (R2 = 0.51), which was decided by its F-value. Therefore, this result shows that the polysemy of a design idea feature has a significant correlation with the originality (see Figure 2).

Figure 2. Correlation between originality and polysemy of

design idea features.

4.3. Design Idea Feature Replacement On the basis of the abovementioned findings, we propose a

method based on polysemy for generating a candidate of a set of design idea features. In this method, some of the features in a given base design idea set are replaced with an equivalent number of features from another design idea set. The features of a base design idea with low polysemy are replaced with those of a design idea with high polysemy. Figure 3 illustrates this feature replacement. The circle represents a word expressing a design idea feature; the number represents the polysemy.

Figure 3. Illustration of design idea feature replacement.

5 METHOD 2: EVALUATING THE GENERATED CANDIDATE

As mentioned above, association is a way to enhance the thought process of a designer. In this section, a method for evaluating the candidate generated by method 1, as shown in Section 4, is proposed. In this method, the originality of the generated candidate is estimated by using a regression based on the correlation between the score of originality and the structure of the virtual network, which virtually represents concept generation process by focusing on their association.

5.1. Virtual Modeling of Concept Generation Process

This method is based on the notion of the virtual modeling of the thinking process, which focuses on the notions of “structure” and “latent sensitivities,” which underlie the notion of association. Below is an outline of the method, which has been previously developed by us [21] below. This method is based on the following hypothesis.

Factors that affect the evaluated score of creativity of design idea are closely related to the structure of the thinking space.

On the basis of this hypothesis, a thinking space is represented by a network, with the concepts evoked during concept generation as nodes and the relationship among the nodes as links. Several research groups have used the notion of “network” for analyses in various domains [22, 23, 24, 25]. In this study, we constructed a network representing a thinking


space as a model of concept generation process by employing the notion of network. This structure of the thinking space, which is used to find thinking patterns, is analyzed. Figure 4 shows the image of the thinking space and its structure.

Figure 4. Image of the thinking space and its structure.

Some studies have discussed the notion of “latency.” Many

studies in the field of design have focused on latent sensitivities such as latent function [26, 27] and latent semantic approach [28]. We believe that a designer cannot explicitly express all his/her thoughts; however, we can consider the thinking space to be composed of chain processes of concepts that are explicitly represented (for example, protocol analysis using linkography [29]). The results of the previous research indicate that latent sensitivities can be used to extract functions or relations that cannot be expressed explicitly. We study the latent concepts involved in the thinking space by modeling the virtual chain processes of concepts evoked in concept generation process; in other words, we assume the concepts that appear in the virtual model to involve latent concepts.

Accordingly, we propose a method for modeling a virtual thinking space by using a semantic network and quantitatively analyze its pattern using network theory.

5.2. Construction of a Virtual Thinking Network Recently, several researchers in the design domain have

used semantic networks [7, 8, 11]. Semantic networks have structures composed of semantic relations between words, such as the hypernym-hyponym relation and associated relations. Semantic networks are practically useful for searching links between words. In addition, since semantic networks do not

depend on the designer’s background, they are suitable for the virtual modeling of concept generation process, and the virtual thinking space constructed on the basis of these networks can be reproduced on a computer. A semantic network is used for identifying virtual chain processes of concepts during the concept generation process. The virtual chain processes comprise paths that lead from base concepts to design idea features, and each path represents a chain of association from a concept to another concept. There exist nodes except base concepts and design idea features in each of the paths. These nodes are virtual concepts. Virtual concepts are assumed to involve latent concepts in concept generation process. We construct a network that involves the paths composing a virtual chain process as a representation of a virtual thinking space (hereafter referred to as a virtual thinking network). Thus, virtual thinking networks consist of two types of nodes: (1) the explicit concepts that are assumed to be imaged explicitly in the design thinking process (2) latent concepts.

Figure 5 shows the process of constructing the virtual thinking network. This process involves the following four steps:

Step 1: Searching for paths between a base concept and a design idea feature (first diagram in Figure 5). Here, a concept is expressed as a word, and a path is a set of direct links joining two words. Since words have multiple meanings in the semantic network, multiple paths between the base concept and the design idea feature are extracted.

Step 2: Extracting virtual concepts that appear in the paths between the base concepts and the design idea features in the form of latent concepts (second diagram in Figure 5).

Step 3: Constructing a network with the virtual concepts as nodes, and the links as the extracted paths (third diagram in Figure 5).

Step 4: Extracting the structure of a virtual thinking network as the structure of the thinking space (fourth diagram in Figure 5).

Figure 5. Process of constructing a virtual thinking network.


5.3. Correlation between Creativity and Structure By using network theory, we analyzed the structure of the

thinking space extracted from the virtual thinking network. Using the network, a graph consisting of a set of nodes and a set of links joining the nodes was drawn. Although a wide range of statistical criteria exists in network theory, we used a few important criteria to characterize our thinking network. We chose the same network statistical criteria used by Steyvers and Tenenbaum [30]. These authors used these statistical criteria to examine whether or not the structure of semantic networks is different from that of other complex natural networks.

We analyzed the design ideas described in Section 3 and found that some criteria, particularly the number of nodes in a virtual thinking network, had a significant correlation with the originality scores of the design ideas. We performed stepwise regression for automatic selection of significant variables on the basis of the F-value significance (Figure 6); then, we obtained the regression equation with the number of nodes as the significant variable.

In this study, the originality of a newly generated design idea was estimated using this regression equation.

Figure 6. Correlation between the originality and the number of noses in a virtual thinking network for a

design idea.

6 METHODOLOGY On the basis of Methods 1 and 2, we propose the following methodology for supporting design idea generation:

Step1: Select two design ideas from the existing design ideas: the base design idea and the design idea to be combined with this base design idea.

Step2: Using Method 1, combine the design idea features of the design ideas selected in Step1.

Step3: Using Method 2, evaluate the set of design idea features generated in Step2.

7 EXPERIMENT We used WordNet [19, 20] as the semantic network and the network analysis tool Pajek [31] to visualize and analyze networks in terms of network theory. Using Pajek 1.23, we visualized the constructed networks and calculated the values of the statistical criteria of the network. WordNet was run in a Linux environment, while Pajek was run on Windows XP system.

7.1. Selecting design ideas We selected design ideas with the four highest scores for estimated originality by using the regression equation shown in Figure 7. B6 was selected as the base design idea, while A2, B5, and B9 were selected as the design ideas to be combined with the base design idea. Table 2 shows the following for each design idea: the originality score, which is based on the actual design ideas, given by the evaluators; the score estimated by regression equation; the number of nodes in the virtual thinking network; and the number of design idea features.

7.2. Combining Design Idea Features We show the design idea features and their polysemy in Table 3. We combined the design idea features by using Method 1. In order to verify the effectiveness of generating a set of design idea features based on Method 1, we also combined design idea features by random replacement. For this purpose, we replaced a few design idea features (extracted randomly from the set of design idea features of the selected base design idea) with an equivalent number of features (extracted randomly from the set of design idea features from another design idea set).

The number of design ideas (the set of design idea features) generated by using Method 1, and those generated randomly, are shown in Table 4.

Table 2. Selected design ideas.

Originality Score Estimated Originality Score Number of Nodes Number of Design Idea Features

B6 3.5 3.9 168 18 A2 2.7 3.4 114 12 B5 2.8 3.3 107 10 B9 3.4 3.3 104 8


Table 3. Design idea features and their polysemy for each design idea.

Table 4. Number of sets of design idea features (design idea candidates) generated.

Generated Method Number of generated design idea candidates

Method 1 (polysemy-based replacement) 12 Random replacement 84

Figure 7. Examples pf virtual thinking networks: for a base design idea (left) and for a candidate generated by Method 1 (right). ▲: base concept node, and ■: design idea feature node.

B6 A2 B5 B9 Design idea

features Polysemy Design idea features Polysemy Design idea

features Polysemy Design idea features Polysemy

field 17 picture 10 string 10 tone 10 land 11 wood 8 shape 8 sound 8 first 6 scrap 4 performance 5 shape 8

water 6 room 4 music 5 function 7 toy 5 plant 4 material 5 feature 6

practice 5 fashion 4 thickness 4 beauty 3 peace 5 deck 4 electricity 3 originality 2

pleasure 5 planter 3 ship 1 ship 1 fish 4 recycling 1 parts 1

novelty 4 recycling 1 brightness 3

flower arrangement 1

imagination 3 decor 1 experience 3 corkboard 1 enjoyment 3

pleasantness 2 symbiosis 1

humankind 1 egalitarianism 1


Figure 8. The estimated originality score for each

candidate generated by Method 1 (Polysemy-based replacement).

Figure 9. The estimated originality score for each candidate generated by random replacement.

7.3. Results Figure 7 shows the virtual thinking network for a base

design idea (left) and that for a candidate generated by Method 1 (right). Figures 8 and 9 show the estimated originality scores for each candidate generated by Method 1, and those by random replacement. Circles indicate the estimated originality of the base design ideas.

We tried to design a new concept from the set of features, which was estimated to have the highest score of originality, from a set of features.

This set of features was obtained by replacing “pleasantness,” “symbiosis,” “humankind,” and “egalitarianism,” with “string,” “sharp,” “performance,” and “material” from the design idea features of B5. Figure 10 shows the design idea sketched by us from the obtained features. This design idea corresponds to a vehicle on water; this is driven by a duck, a dolphin, or a fish-shaped robot, which is manipulated by the sound of a string plucked by a person in the vehicle. This design idea is expected to be creative.

Figure 10. New design idea generated from the set of features, which is estimated to have the highest score of originality

in the candidates generated by Method 1.


7.4 Consideration The network shown on the right in Figure 7 is more

complex than that shown on the left. We have previously reported that the complexity of a virtual thinking network is related to the originality of the design ideas [21]. Therefore, network complexity indicates whether or not a candidate might be highly creativity.

From Figure 8, we can see that all the candidates generated by Method 1 have the estimated score of originality that extrapolate the one of the base design idea. Although the difference between the base idea and the generated candidates seems not to be significant, the increase of the estimated score is 0.08-0.25 and this improvement is expected to be significant for the designers. This suggests that Method 1 (method for generating a candidate of a design idea) is consistent with Method 2 (method for evaluating the generated candidate), and vice versa. Although these methods are based on the same notion of association, they have different points of view: Method 1 refers to the user’s impression, while Method 2 refers to the designer’s thinking space. Therefore, our result shows that these are complementary to each other. As a result, the proposed methodology, which involves method 1 and method 2 is expected to be useful for the generation of highly creative design ideas.

8 CONCLUSION In this study, we proposed two methods for generating creative design ideas: Method 1 and Method 2. Method 1 involves generating a candidate for a more creative design idea by focusing on the notion of “polysemy.” Method 2 involves evaluating the generated candidate by using a virtual network. In this study, we focused on the concept generation, particularly, the process of synthesizing two concepts. Furthermore, we used design idea features as the representations of the design idea and attempted to create a set of design idea features that would result in high creativity. The candidates generated by Method 1 were evaluated by Method 2, and we found that their candidates extrapolated the estimated score of originality of the base design idea. Our results show that the generated candidates are capable of manifesting a high level of creativity.

REFERENCES [1] Gero, J. S. and Maher, M. L., 1991, “Mutation and Analogyto Support Creativity in Computer-aided Design,” Proc. Computer Aided Architectural Design Futures ’91, ETH, Zurich, pp. 241–249. [2] Edmond, E. A., Candy, L. and Murray, B. S., 1993, “Knowledge Support Systems for Designers,” Proc. Workshop on Artificial Intelligence in Design: The 13th International Joint Conference on Artificial Intelligence, pp. 7–12, Chambery, France. [3] Goel, A. K., 1997, “Design, Analogy and Creativity,” IEEE Expert Intelligent Systems and Their Applications, 12(3), pp. 62–70.

[4] Issa, G., Shen, S., and Chew, M. S., 1994, “Using Analogical Reasoning for Mechanism Design,” IEEE Expert: Intelligent Systems and Their Applications, 9(3), pp. 60–69. [5] McAdams, D. A., and Wood, K. L., 2002, “A Quantitative Similarity Metric for Design-by-Analogy,” Journal of Mechanical Design, Transactions of the ASME, 124, pp. 173–182. [6] Chakrabarti, A., Sarkar, P., Leelavathamma, B., and Nataraju, B.S., 2005, “A Functional Representation for Aiding Artificial Inspiration of New Ideas,” Artificial Intelligence in Engineering Design and Manufacturing, 19(2), 113–132. [7] Linsey, J. S., Wood, K. L., and Markman, A. B., 2008, “Increasing Innovation: Presentation and Evaluation of the WordTree Design-by-Analogy Method,” Proc. ASME Design Theory and Methodology Conference, DETC2008-49317, New York City, NY. [8] Chiu, I., and Shu, L. H., 2007, “Using Language as Related Stimuli for Concept Generation,” AI Edam: Artificial Intelligence for Engineering Design Analysis and Manufacturing, 21(2), pp. 103–121. [9] Kolb, E. M., Hey, J., Sebastian, H. J., and Agogino, A. M., 2008, “Meta4acle: Generating Compelling Metaphors for Design,” Proc. ASME Design Theory and Methodology Conference, DETC2008-49331, New York City, NY. [10] Eckert, C., Stacey, M. and Earl, C., 2005, “References to Past Designs”, Studying Designers ’05, J. S. Gero and N. Bonnardel (Eds.), 2005 Key Centra of Design Computing and Cognition, University of Sydney, pp. 3–21. [11] Georgiev, G. V., Taura, T., Chakrabarti, A., and Nagai, Y., 2008, “Method of Design through Structuring of Meanings,” Proc. ASME Computers and Information in Engineering Conference, DETC2008-49500, , New York City, NY. [12] Lai, I. C., and Chang, T. W., 2006, “A Distributed Linking System for Supporting Idea Association during the Conceptual Design Stage,” Design Studies, 27, pp. 685–710. [13] Nagai, Y. and Taura, T., 2006, “Role of Action Concepts in Creative Design Process,” Kim, Y. S. (Ed.), Proc. International Design Research Symposium 2006, pp. 257–267, Seoul, Korea. [14] Nagai, Y., Mukai, F., and Taura, T., 2008, “Concept Blending and Dissimilarity: Factors for Creative Design Process – A Comparison between the Linguistic Interpretation Process and Design Process,” Proc. Design Research Society Conference 2008, CD-ROM, Sheffield, UK. [15] Finke, R. A., Ward, T. B., and Smith, S. M., 1992, Creative Cognition: Theory, Research, and Applications, MIT Press, Cambridge, MA. [16] Taura, T., Nagai, Y., and Tanaka, S., 2005, “Design Space Blending,” Proc. the 15th International Conference of Engineering Design, on CD-ROM, Maastricht, the Netherlands. [17] Lubart, T., 1994, “Creativity,” in R. J. Stenberg (Ed.), Thinking and Problem Solving, Academic Press, pp. 289–332. [18] Rothenberg, A., 1979, The Emerging Goodness: The Creative Process in Art, Science, and Other Fields, University of Chicago Press, Chicago.


[19] Miller, A., Beckwith, R., Fellbaum, C., Gros, D., Millier, K., and Tengi, R., 1990, “Five Papers on WordNet,” Technical Report CSL Report 43, Cognitive Science Laboratory, Princeton University, New Jersey. [20] Fellbaum, C., 1998, WordNet: An Electronic Lexical Database, MIT Press, Cambridge, MA. [21] Yamamoto, E., Goka, M., Yusof, N. F. M., Taura, T.,and Nagai, Y., 2009, “Virtual Modeling of Concept Generation Process for Understanding and Enhancing the Nature of Design Creativity,” The 17th International Conference of Engineering Design, Stanford University, San Francisco, CA. (accepted) [22] Ferrer, I., Cancho, R., and Solé, R. V., 2001, “The Small World of Human Language,” The Royal Society of London Series B, 268, pp. 2261–2265. [23] Schilling, M. A., 2005, “A ‘Small-World’ Network Model of Cognitive Insight,” Creativity Research Journal, 17(2), pp. 131–154. [24] Matsuo, Y., Osawa, Y., and Ishizuka, M., 2001, “KeyWorld: Extracting Keywords from a Document as a Small World,” Proc. the 4th International Conference on Discovery Science (Lecture Notes in Computer Science), 2226, pp. 271–281, Springer Verlag, Berlin/Heidelberg.

[25] Sato, S., Fukuda, K., Kazama, K., and Murukami, K., 2005, “Preliminary Results on Describing and Interpreting Context by Network Structure,” Proc. the 1st International Workshop on Agent Network Dynamics and Intelligence, pp. 39–44, Kitakyushu, Japan. [26] Yoshikawa, H., 1998, “General Design Theory and a CAD System,” Man-Machine Communication in CAD/CAM, pp. 35–58. [27] Minami, K., Nagai, Y., and Taura, T., 2005, “Study on Latent Function for Sustainable Design,” Proc. the 1st International Design Congress IASDR, CD-ROM, Taiwan. [28] Dong, A., 2005, “The Latent Semantic Approach to Studying Design Team Communication,” Design Studies, 26 (5), pp. 445–461. [29] Goldschmidt, G., 1990, “Linkography: Assessing Design Productivity,” in R. Trappl (Ed.), Cybernetics and System ’90 World Scientific, Singapore, pp. 291–298. [30] Steyvers, M., and Tenenbaum, J. B., 2005, “The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth,” Cognitive Science, 29, pp. 41–78. [31] Pajek, Program for Large Network Analysis, available at http://vlado.fmf.uni-lj.si/pub/networks/pajek/.

a method to generate and evaluate a creative … · · 2009-09-18students were required to create...

Documents