基于_维量_思想的人工情感模型英文

第34卷第1期 中 　国 　科 　学 　技 　术 　大 　学 　学 　报 Vol. 34 ,No. 1 2 0 0 4 年 2 月 JOURNAL OF UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA Feb. 2 0 0 4 Article ID :025322778 (2004) 0120083209 An Artif icial Emotion Model Based on the Dimension Idea Ξ WAN G Shang2fei , WAN G Xu2fa ( A rtif icial Intelligence L ab. Department of Com puter Science and Technology of US TC , Hef ei , A nhui , P. R . China , 230027) Abstract : Inspired from“dimension”idea from psychology , an artificial emotion model is presented in this paper. The object of the model is adjectives such as“beautiful”, which are often used to express emotions. To build this model , first , a psychological se2 mantic quantificational experiment is done , and factor analysis is condueted to analyze the experimental result , thus an orthogonal emotion space is constructed. Then , the measurement of similarity in the emotion space is analyzed. After that , sensitive fea2 tures are extracted from images to construct the feature space , and support vector ma2 chines are used to map each image from the feature space to the emotion space. Thus an emotional image retrieval system is obtained. Last an interesting experimental result is presented. Key words :affective computing ; dimension idea ; factor analysis ; emotional image re2 t rieval ; support vector machines CLC number :TP391 　　　Document code :A 0 　Introduction The main purpose of Artificial Intelligence (AI) is to mimic human intelligence , in all its aspects. So , since the beginning of AI , numerous researches have aimed at modeling the cogni2 tive processes identified in the human mind and artificial systems that learn , plan , make deci2 sions and reason , have been designed. Despite Minsky′s famous remark in his Society of Mind dating back to 1985“the question is not whether intelligent machines can have any emotions , but whether machines can be intelligent without any emotions”, less effort has been devoted toΞ Received date :2003204214 　Foundation item :Supported by National 973 Project of P. R. China ( G1998030500) and USTC Youth Project . 　Biography :WAN G Shangfei , female , born in 1974 , PhD , research on computation intelligence , affective computing and time series prediction. E2mail :sfwang @mail. ustc. edu. cn © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. modeling emotional behaviors than purely rational intelligent behaviors. In particular , classical types of reasoning in AI like deduction , induction , classification or case2based reasoning , for example , have been intensively studied , while attempts to endow machines with emotions , i. e. to conceive computational models taking emotions into account , are rather recent [1～4 ] . In this paper , an artificial emotion model based on the“dimension”idea from psychology is proposed. First , psychological semantic quantificational experiments are used to collect user’ s emotion information and build the emotion database. Second , factor analysis is applied to con2 st ruct an orthogonal emotion space. After that , the measurement of similarity in the emotion space is analyzed. Then this model is used in emotional image retrieval. Illumed from painting and fashion design , sensitive features , such as color and shape features of scenery images , style and material of fashion , have been extracted from images to construct the feature space. Sup2 port vector machines are applied to annotate images emotionally , to define the map function from the low level feature space to the high level emotion space , to memorize users’emotions , and to automatically annotate unevaluated images based on users’emotion. In this way images can be indexed in the emotion space. The paper is organized as follows : Section 1 describes our artificial emotion model ; Section 2 shows the emotional image retrieval f rame. Section 3 represents performance by computer ex2 periments ; Section 4 gives the conclusion. 1 　Artif icial Emotion Model 1. 1 　Dimension Idea In the study of face expression [5 ] , psychologists have obtained two great achievements : one is classification , and the other is dimension. We get inspiration form the latter and provide the artificial emotion model here. The dimension idea came from H. Schlosberg. He proposed three dimensions , happiness2 unhappiness , attention2rejection and relaxation2tensity. Some researches proved the existence of happy2unhappy and relax2tense dimension , but ignored attention2rejection. After that , Os2 good C. E. and Frijda N. H. proposed three and six dimensions respective in their research. In 1960 , R. Plutchik provided a complex model of emotions. He said that mixed emotion in daily life were defined by various combinations of eight pure emotions , i. e. joy2sadness , anger2fear , expectation2surprise , and hate2acceptance , which were regarded as basic att ributes not to be further classified. He listed these eight pure emotions on an inverse cone. The length from the top to the bottom indexed intensity. The eight pure emotions on the cross section were ar2 ranged according to similarity and polarity. Thus there were three dimensions in Plutchik’s complex emotion model , intensity , similarity and polarity. From the psychological point of view , the dimension analysis is very important in the study of emotions because it can explain the amazing similarity between them , i. e. happy and contempt , and interpret multifarious nuance among face expression. Furthermore , it is very simple. But the shortcoming is the difficulty to choose the number of dimensions and to give the 48 中国科学技术大学学报 　　　　　　　　　　　　　　　　　第 34 卷 © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. meaning of each dimension. In our opinion , the concision is very suitable for engineering appli2 cation and the name of each dimension is not important since dimension is a kind of mathemati2 cal and philosophical abstraction. It gives us inspiration that we can build an emotion space by proper method and regard each kind of emotion as a vector in this space. 1. 2 　How to Build Emotion Model Usually people use adjectives , such as beautiful , romantic or elegant to express emotions. So the objects of the emotion model are those adjectives. To build emotion model , we should quantitate and analyze those adjectives. In this section we examine the constructional process of the emotion model in detail. The construction procedure includes the careful selection of adjec2 tive pairs , psychological semantic quantificational experiment and factor analysis. We first carefully consider 18 ( N = 18) pairs of adjectives about scenery images and 15 ( N = 15) pairs of adjectives about fashion images , as separately listed in table 1 , then 180 ( K = 180) graduate students in their 20 s (120 male and 60 female) are asked to evaluate 206 ( M = 206) sample scenery images and 200 ( M = 200) sample fashion images using these adjectives in 5 scale values. For example , for the adjective pair beautiful and ugly , a student would have five scales as shown in Fig. 1 and assign each given image a scale value based on his/ her impression. Tab. 1 　Adjective words To Scenery Images To Fashion Images 1. likeable —dislikable 2. beautiful —ugly 3. harmonious —absonant 4. romantic —unromantic 5. comfortable —uncomfort2 able 6. enthusiastic —ice2cold 7. gentle —cool 8. bright —dull 9. soft —unsoft 10. orderly —unorderly 11. clear —dim 12. quiet —unquiet 13. impressive —fade 14. carefree —depressive 15. changeable —monotone 16. vital —desert 17. largo —limited 18. warm color —cold color 1. implicit —ebullient 2. formalistic —vivacious 3. unvarnished —showy 4. sober —fashionable 5. lightsome —massive 6. multifarious —compact 7. mellow —spell able 8. intellectual —tameless 9. innervation —t ranquil 10. mysterious —canty 11. mature —immature 12. conversant —noblest 13. tedious —comely 14. gaudy —elegant 15. cool —warm Fig. 1 　Five2degree scoreSecond , the evaluation values given by thesestudents are averaged according to Eq. 1. Then M×N data matrix X is obtained by centering andstandardizing matrix Y according to Eq. 2. After that factor analysis [ 6 ] is applied to this matrix X , and Principle Component Analysis ( PCA) is used to evaluate the factor loadings A in Eq. 3. ym n = 1 K ∑ K k = 1 z m nk (1) Where , z m nk is the evaluation value of user k to the sample image m using the adjective n . x m n = ym n - yn sn (2) Where , 58第 1 期 　　　　　　　　　　An Artificial Emotion Model Based on the Dimension Idea © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. y n = 1 M ∑ M m = 1 ym n , 　sn = ∑ M m = 1 ( ym n - …y n) 2 X = FA′+ UD (3) A = a11 ⋯ a1 L … ω … aN1 ⋯ aNL , 　F = f 11 ⋯ f 1 L … ω … f M1 ⋯ f ML 　　In Eq. 3 : F denotes the L underlying factors called common factor matrix , A is loading matrix , U is unique matrix , and D is weight of unique factor. By PCA , the original N2dimen2 sion space is reduced to orthogonal L2dimension space called emotion space. The m2th row of F ∶f m = ( f m1 , f m2 , ⋯, f mL ) indicates the coordinate of the sample image m in the emotion space. And the n2th row of A ∶an = ( an1 , an2 , ⋯, anL ) corresponds to the adjective n in the same space. 2 　Emotional Image Retrieval Fig. 2 shows the overview of our Emotional Image Retrieval ( EIR) framework. It consists of four components : image processing kernel , an orthogonal emotion space and emotional anno2 tation and emotional image retrieval. Construction of the orthogonal emotion space is described in section 1 , In this section , image processing kernel , emotional annotation and image retrieval will be explained in detail. Fig. 2 　Overview of Emotional Image Retrieval 2. 1 　Image Processing Kernel The goal of image processing ker2 nel is to extract features from images to construct feature space. Here , the basic principle is to extract sensitive features , which can easily stimulate users’emotions. Painters [ 6 ] con2 cluded that color , shape , position , light or shadow , false or t rue , and density are used to express artistic e2 motions. So , domain color , shape , color and gray2level dist ribution are extracted from the scenery images as the features. The latter two features include some information on position , light or shadow , false or t rue and density. The details of how to extract these features from scenery images can be found in [5 ] . Thus a 1962D feature space ( J = 196) has been construct2 ed , including 11 domain color features , 7 shape features , 114 color features and 64 gray2level dist ribution features. As for fashion images , we discussed with fashion designers and coded 68 中国科学技术大学学报 　　　　　　　　　　　　　　　　　第 34 卷 © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. 1692D features such as color , style , length , cloth etc. 2. 2 　Emotional Annotations In section 1 , by factor analysis , each adjective or sample image can been seen as a vector in emotion space. The image database consists of 1300 scenery images and 1486 fashion images. So there are 1094 scenery images and 1286 fashion images in the database besides the samples , which are needed to be mapped to the emotion space. Then all images can be indexed in the e2 motion space. This is the basic idea of emotional annotation. The function of annotation is to construct the mapping function from the feature space to the common emotion space , to learn and memorize users’emotions and to automatically annotate each unevaluated image based on users’emotions. This is a small sample modeling problem because the number of sample images cannot be too large so as to avoid user fatigue. A new learning method named support vector machines ( SVM) can solve small samples modeling problem properly [ 9 ] . Here , SVM for regression are used to annotate images. The features of a sample image are used as the inputs of the SVM and the corresponding values in the L2dimensional orthogonal emotion space are the outputs. After t raining , the parameters of the SVM represent the relationship between features space and emotion space. Then when fea2 tures of a new image are input , SVM can output its corresponding values in the emotion space. In other words , Fig. 3 　Flowchart of the emotional image retrieval the image is automatically annotated by the SVM. Thus we can retrieval an image just by indexing it with adjective words in the emotion space [5 ] . 2. 3 　Image Retrieval After annotation , images can be retrieved just by index2 ing them with adjectives. Fig. 3 is the flowchart of emotional image retrieval. The user chooses an adjective stored in the adjective database , and then the retrieval system indexes the adjectives of each image and displays top the 12 images with the closest similarity to the input adjectives. In this section , the definition of similarity is given as following : dm n = an ·f m an f m (4) dm n represents the similarity between image m to adjective n . 3 　Experiment 3. 1 　Emotion Space Experiment Before doing psychological experiment , every user must register his/ her personal informa2 tion such as sex , age , nationality and interest to the system. The information will be used to classify users. Here , in our experiment , the users are 180 graduate students in their twenties , of Han 78第 1 期 　　　　　　　　　　An Artificial Emotion Model Based on the Dimension Idea © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. Tab. 2 　Factor loading (scenery images) Series Number Factor loading Λ1 Λ2 Λ3 Λ4 1 0. 894 4 - 0. 362 4 0. 106 8 - 0. 046 2 2 0. 932 4 - 0. 252 9 0. 097 3 - 0. 041 2 3 0. 907 8 - 0. 287 3 0. 005 0 - 0. 034 5 4 0. 884 2 0. 157 3 0. 010 6 0. 299 8 5 0. 962 9 - 0. 045 - 0. 035 9 0. 144 1 6 0. 140 6 0. 716 8 0. 583 4 - 0. 125 1 7 0. 870 8 0. 310 2 - 0. 097 1 0. 313 0 8 0. 706 5 0. 327 7 - 0. 083 2 - 0. 511 2 9 0. 910 4 0. 049 9 - 0. 142 0 0. 253 0 10 0. 798 5 0. 154 0 - 0. 337 8 - 0. 130 3 11 0. 697 6 0. 142 9 - 0. 243 6 - 0. 565 5 12 0. 758 1 - 0. 312 2 - 0. 370 5 0. 093 6 13 0. 854 4 - 0. 295 3 0. 169 0 - 0. 102 5 14 0. 924 4 0. 226 5 - 0. 028 0 0. 067 2 15 0. 421 5 - 0. 395 9 0. 738 9 0. 018 9 16 0. 748 9 0. 370 1 0. 215 6 0. 124 2 17 0. 330 3 - 0. 640 2 0. 311 8 - 0. 190 7 18 0. 285 7 0. 826 0 0. 178 7 - 0. 007 4 nationality. So we just classifies the users into two classes , female users and male users. After registeration , users begin to e2 valuate the sample image. The evaluation data and the users’personal information are all stored in the emotion information database. Table 2 and Table 3 give the fac2 tor loadings. For scenery images , the addi2 tive contributing rate of the first four com2 ponents amounts to 87. 78 % , while for fashion images , the additive contributing rate of the first seven components amounts to 87. 19 %. Thus the original 18 pairs ( or 15 pairs) of adjectives can be seen as vectors in the 42dimension (72dimension) common e2 motion space , and the 206 sample scenery images (200 sample fashion images) can also be seen as vectors in the same space. Tab. 3 　Factor loading (fashion images) Serial Number Factor loading Λ1 Λ2 Λ3 Λ4 Λ5 Λ6 Λ7 1 0. 056 4 0. 013 4 - 0. 240 0 - 0. 103 0 - 0. 158 1 - 0. 152 9 0. 890 2 2 - 0. 146 2 0. 802 6 - 0. 047 7 - 0. 161 1 - 0. 216 9 - 0. 266 8 - 0. 255 9 3 - 0. 221 8 - 0. 151 9 - 0. 082 0 0. 882 2 - 0. 119 9 0. 192 5 0. 506 7 4 0. 396 7 - 0. 449 7 0. 509 9 0. 068 3 - 0. 474 0 - 0. 131 5 - 0. 030 5 5 - 0. 214 0 - 0. 106 0 - 0. 244 6 0. 198 9 - 0. 019 9 - 0. 019 2 0. 002 6 6 - 0. 222 1 0. 048 1 - 0. 802 1 - 0. 182 6 0. 135 1 - 0. 849 1 0. 058 4 7 - 0. 283 5 0. 045 7 0. 135 0 - 0. 086 9 0. 089 2 - 0. 031 8 - 0. 021 5 8 - 0. 340 2 0. 008 5 - 0. 715 4 - 0. 284 0 0. 147 6 - 0. 090 6 0. 260 9 9 - 0. 080 5 - 0. 453 9 0. 116 8 - 0. 067 7 0. 038 4 0. 090 4 - 0. 052 1 10 0. 056 4 0. 013 4 - 0. 240 0 - 0. 103 0 - 0. 158 1 - 0. 152 9 0. 890 2 11 - 0. 146 2 0. 802 6 - 0. 047 7 - 0. 161 1 - 0. 216 9 - 0. 266 8 - 0. 255 9 12 - 0. 221 8 - 0. 151 9 - 0. 082 0 0. 882 2 - 0. 119 9 0. 192 5 0. 506 7 13 0. 396 7 - 0. 449 7 0. 509 9 0. 068 3 - 0. 474 0 - 0. 131 5 - 0. 030 5 14 - 0. 214 0 - 0. 106 0 - 0. 244 6 0. 198 9 - 0. 019 9 - 0. 019 2 0. 002 6 15 - 0. 222 1 0. 048 1 - 0. 802 1 - 0. 182 6 0. 135 1 - 0. 849 1 0. 058 4 3. 2 　Similarity Measurement Experiments To test whether the similarity is suitable , the similarity of each adjective to each sample image is calculated , then the similarity and the original evaluation data are compared , as shown 88 中国科学技术大学学报 　　　　　　　　　　　　　　　　　第 34 卷 © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. in fig. 4 , the abscissa repre sents the serial numbers of sample images. The real line is the origi2 nal evaluation data , the dotted line presents the similarity of each sample image to the adjective (likeable) . From fig. 4 , we can see that these two lines match well , proveing that the similari2 ty defined by Eq. 4 is proper. Fig. 4 　Comparison of similarity and original evaluate data (adjective : likable) 3. 3 　Emotional Image Retrieval Experiment Fig. 5 gives the retrieval result of“dull scenery images”,“romantic scenery images”,“ele2 gant fashion images”and“massive fashion images”. Most users are satisfied with the retrieval result . To further evaluate the performance of this system , we requested about 20 graduate stu2 dents in their twenties to do emotional image retrieval experiment . The test result is shown in Table 4. Here we define two parameters : Precision rate = the number of images which users are satisfied withthe number of all ret rieved images Succeed rate = the number of users who find the images he or she wanted the user number In this test , about 80 percent of the subjects can retrieve the images he/ she wanted. 4 　Conclusion How to deal with human emotions is an important issue in designing multimedia database system. Inspired by psychological dimensional idea , this paper t ries to build an artificial emo2 tion model through psychological experiments and factor analysis. The objects of the emotion model are Tab. 4 　Test result of emotional image retrieval Parameters Scenery images Fashion images Precision rate 40 % 50 % Succeed rate 78 % 80 % adjectives such as romantic or beautiful expressed by people who are retrieving images , music or other media. Then an emotional image retrieval f rame is proposed , which consists of four parts , an image processing kernel extracting sensitive features from images , an orthogonal emotion space based on the artificial emotion model , emo2 98第 1 期 　　　　　　　　　　An Artificial Emotion Model Based on the Dimension Idea © 1995-2006 Tsinghua Tongfang Optical Disc Co., Ltd. All rights reserved. tion annotation using support vector machine that maps each image from the feature space to the emotion space , and image retrieval in the emotion space using COS similarity. Last , an in2 teresting experiment result is presented , demonstrating the effectiveness of our approach. “dull”　　　　—scenery image retrieval —　　　　“romantic” “elegant”　　　　—fashion image retrieval —　　　　“massive” Fig. 5 　Emotional image retrieval result (female) This emotion model and the retrieval f rame can also be used in emotional ret rieval of music or other media. However , since the retrieval result is very subjective , efficient evaluation is very difficult . We will do some further research on this issue. Furthermore , human emotions include two aspects [10 ] : common emotions and individual emotions. A comm on emotion is an “average”emotion of a certain number of people , while an individual emotion reflects the dif2 ference between each other. This paper only deals with the common emotions. How to deal with the individual emotions is another issue [8 ] . In our opinion , interaction and adaptive lean2 ing mechanism are the key issues. 09 中国科学技术大学学报 　　　　　　　　　　　　　　　　　第 34 卷 © 1995-2006 Tsi