we feel fine and searching the emotional web

We Feel Fine and Searching the Emotional Web Sepandar D. Kamvar Stanford University sdkamvar@stanford.edu Jonathan Harris Number 27 jjh@number27.org ABSTRACT We present We Feel Fine, an emotional search engine and web-based artwork whose mission is to collect the world’s emotions to help people better understand themselves and others. We Feel Fine continuously crawls blogs, microblogs, and social networking sites, extracting sentences that in- clude the words “I feel” or “I am feeling”, as well as the gender, age, and location of the people authoring those sen- tences. The We Feel Fine search interface allows users to search or browse over the resulting sentence-level index, ask- ing questions such as “How did young people in Ohio feel when Obama was elected?” While most research in senti- ment analysis focuses on algorithms for extraction and clas- sification of sentiment about given topics, we focus instead on building an interface that provides an engaging means of qualitative exploration of emotional data, and a flexible data collection and serving architecture that enables an ecosys- tem of data analysis applications. We use our observations on the usage of We Feel Fine to suggest a class of visual- izations called Experiential Data Visualization, which focus on immersive item-level interaction with data. We also dis- cuss the implications of such visualizations for crowdsourc- ing qualitative research in the social sciences. Categories and Subject Descriptors: H.5.2 Information Interfaces and Presentation: User Interfaces, H.4.m Infor- mation Systems: Miscellaneous General Terms: Design, Human Factors Keywords: Sentiment Analysis, Social Media, Search, Com- putational Social Science 1. INTRODUCTION The growth of the social web and the corresponding rise in available emotional text over the past several years has led to an increased interest in sentiment analysis. Research in sentiment analysis has concerned itself primarily with algo- rithms for the extraction of text spans that contain a view- point and the classification of those text spans [20]. Typi- cal motivating applications have been technologies that help consumers make purchase decisions, for example, classify- ing a movie review as “thumbs up” or “thumbs down” [19]. However, the large-scale availability of emotional text – es- pecially when coupled with the availability of demographic information from social media profiles – gives us the ability Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WSDM’11, February 9–12, 2011, Hong Kong, China. Copyright 2011 ACM 978-1-4503-0493-1/11/02 ...$10.00 to do much more than make better decisions about products or politicians. It gives us the ability to better understand emotions themselves. In this paper, we present We Feel Fine, a project that aims to collect the world’s emotions to help people better un- derstand themselves and others. Since August 2005, We Feel Fine has been harvesting human feelings from a large number of weblogs. Every few minutes, the system searches the world’s newly posted blog entries for occurrences of the phrases “I feel” and “I am feeling”. When it finds such a phrase, it records the full sentence and identifies the “feel- ing” expressed in that sentence (e.g. sad, happy, depressed, etc.). Because blogs are structured in largely standard ways, the age, gender, and location of the author can often be ex- tracted and saved along with the sentence. The result is a database of several million feelings, increasing by 10,000 - 15,000 new feelings per day. Using a series of playful in- terfaces, the feelings can be searched and sorted across a number of demographic slices, offering responses to specific questions like: do Europeans feel sad more than Americans? Does rainy weather affect how we feel? And so on. We developed We Feel Fine in 2005-2006 and launched it in May 2006. From launch until time of submission, We Feel Fine has collected over 14 million expressions of emo- tion from 2.5 million people, has been used by 8.5 mil- lion visitors, and has been exhibited in several museums, including the Victoria and Albert Museum (London), the National Museum of Contemporary Art (Athens), and the Museum of Fine Art (Houston). We Feel Fine is available at www.wefeelfine.org. Figure 1: We Feel Fine Search Panel 2. DESIGN CONSIDERATIONS Our goal with with We Feel Fine was to enable both a quali- tative and statistical exploration of other people’s emotions. Our belief was that statistical analyses would support a cog- nitive understanding of emotional patterns, while immersive interaction with individual stories would support a more vis- ceral understanding of the nature of emotions. This lat- ter focus on intuitive understanding through qualitative ex- ploration is a markedly different approach from most work in sentiment analysis, which largely focuses on algorithmic analysis of sentiment text around a given topic. We found that creating an immersive means of interacting with individual data items directly affected the end user in line with our mission of helping people to understand them- selves and others. Further, the qualitative and statistical exploration of data that is personal and universal appealed to a broad set of users, enabling crowdsourced data anal- ysis. We discuss these effects in Section 6; in this section we discuss the principles that engendered these effects. We considered the following principles in our design: 1. Sentence-level analysis. The basic atom of the in- formational web is the web page, so it is natural for in- formational search engines to analyze and return pages. On the social web, the basic unit is often much shorter – the post or tweet or short social network message. Further, people often express emotions at the sentence- level; rarely is an entire document about a single emo- tion. For these reasons, we use sentences rather than web pages as the canonical “documents” in the search engine. 2. Indexing context. Through public profiles and times- tamps, there is much useful context to an emotion out- side of the words in the expression itself, for example, the time of the emotion, or the location, gender, and age of the person expressing the emotion. Appropriate index and interface design enable users to ask ques- tions like: “how do women feel right now?” or “how did people in the U.S. feel on September 11th?” 3. Sentiment as the primary organizing principle. Most sentiment analysis search engines aim to under- stand more about certain topics by exploring the pre- vailing emotions around those topics. With We Feel Fine, the primary aim is to understand more about emotions themselves. As such, we focus on sentiment as the primary organizing principle. This dictates a wide range of design decisions. For example, we use a faceted search interface that focuses on emotions and people rather than a free-form search box, and we in- dex a wide range of emotions rather than simply clas- sifying emotions as positive or negative. 4. De-emphasizing ranking. While factual informa- tion can be ranked by validity or authority of the source, it is much more difficult to rank sentiment. In keeping with the old adage “feelings are never wrong”, thou- sands of different expressions can be equally reason- able responses for a query like “how do you feel about going to college?” For this reason we don’t introduce the notion of ranking in We Feel Fine. 5. Emphasizing browsing and summarization. Rather than ranking expressions of emotion, we prefer to to summarize them— allowing the user to quickly get the gestalt of how a population feels. Further, since users can gain intuition through qualitative exploration of a population, we aim to provide easy browsing as well. 6. Enabling the user to easily shift between macro and micro. In the last bullet, we emphasized the ability to see data at the macro-level (summarization) and the micro-level (browsing). It’s also important to allow the user to easily shift between the two. In shift- ing from micro to macro, the user can better under- stand trends exposed through summarization. And in shifting from micro to macro, the user can generalize intuitions gained from browsing. Where possible, we wish to create graphs from individual clickable atoms of data (macro to micro transition). And we wish to label items of data with search terms, allowing the user to easily summarize or see more data of that type (mi- cro to macro transition). 7. Visualizations that reflect the data. The interface should be functional, but not cold — after all, this is a search engine about emotions. An ideal UI should reflect the subject matter, and in this case, we crafted the elements of the visualization to have human qual- ities to reflect the people they represent. 8. Direct Access to the Data. We recognize the value of We Feel Fine’s underlying data for social scientists, particularly those interested in large-scale studies of emotion. We also recognize the difficulty in designing an interface that is intended to be both an artwork and a scientific tool. It is with this in mind that we built a data API for direct data access. 3. ARCHITECTURE The architecture of the We Feel Fine system is composed of five main components, the Crawler, the Indexer, the Data Store, the Web Server, and the Client Applications. We give a brief overview in this section. We describe the user interface in more detail in Section 4 and data analysis com- ponents at greater depth in Section 5. Crawler. A URLServer (1) collects the URLs of pages to be crawled. These pages contain blog posts, microblog feeds, and pages with public social network messages. URL ex- traction and collection is site-specific; for example, LiveJour- nal has a“recently posted”API, while for MySpace, we crawl the friend graph. The URLServer sends the list of urls to the Crawler (2), which feches the pages from the web. At the moment, the Crawler is a single dedicated machine, but has been designed so that we can easily add more crawling machines if desired. Indexer. The Crawler then sends the fetched pages to the Feeling Indexer (3), which extracts the feeling sentence or sentences, the time and date of the post, and any de- mographic information (such as gender, age, and location). Currently, the Feeling Indexer does extraction by matching hand-crafted regular expressions, although certainly there are more sophisticated techniques that may be used. The Feeling Indexer sends the sentence to the Emotional Lex- icon, which determines whether there is a feeling word in the sentence (like “happy,”“sad,” etc.), and if so, sends the Figure 2: We Feel Fine Component Diagram feeling word(s) back to the Feeling Indexer. If there are im- ages in the post, the Feeling Indexer determines the largest image in the post (by file size) and sends it to the Image Repository (7). For those posts where the location is speci- fied, the Feeling Indexer sends the location, time, and date of the post to the Weather Server (5), which determines the weather using several public weather databases. Data Store. When the Feeling Indexer has finished pro- cessing the URL, it send the feeling sentences and metadata to the We Feel Fine Database (6), a MySQL database that stores the emotional data, including sentence, feeling, date, time, URL, weather, and author gender, age, and location. At the moment, the We Feel Fine Database is a replicated database server that has been designed to be easily sharded by date range if desired. Web Server. Several components communicate with the We Feel Fine Database. The API Server (10) defines a RESTful API, translating specified URLs into SQL queries, and then returning the SQL results to the browser as XML. To decrease query latencies, we cache the most common queries and store the results in the Query Cache (8). The Sentiment Mining Server (9) consists of a set of functions that post-process the results of an API query, computing statistics on the data set on-demand. The Montage Server (11) allows the generation of image/sentence composites, called montages. Given a sentence id, the Montage Server will return the feeling sentence overlaid on the largest image from the same blog post. If there are no images on the same post, it will simply return the sentence. Client Applications. The We Feel Fine Frontend (13) is a Java applet written using Processing [7]. It translates user actions into API queries and sends them to the API Server, and translates the returned XML-formatted results into interactive visualizations. The We Feel Fine Frontend is described in depth in Section 4. Users may choose to save a montage from the applet to the Montage Gallery (14), a gallery of user-saved montages. And finally, since the API is public, there are dozens of Third-Party Applications (12) that use the API. 4. USER INTERFACE 4.1 Design The We Feel Fine Frontend is a Java applet intended to encourage item-level data exploration, as well as some shal- low statistical exploration. It is composed of six movements — called Madness, Montage, Murmurs, Mobs, Metrics, and Mounds — each giving a different view of a sample popula- tion selected by a Search Panel. Search Panel. The Search Panel (Figure 1) allows the viewer to choose the sample population on screen. The red bar at the top of the screen presents a summary of the cur- rent sample population. Clicking the bar causes the panel to open, and viewers can select the population along any combi- nation of the following axes: Feeling, Age, Gender, Weather, Location, Date. The faceted interface of the Search Panel reflects Principles 2 and 3 in Section 2. The focus is on searching for emotions or populations, and the various facets are made possible by indexing context. Madness. Madness, the first movement, is a playful inter- face to interact with individual data items. It opens with a swarming mass of 1500 particles, emanating from the cen- ter of the screen and then careening outwards, bouncing off walls and reacting to the behavior of the mouse. Each par- ticle represents a single feeling. The color of each particle corresponds to the tone of the feeling inside — happy feelings are bright yellow, angry feelings are red, and so on. Any particle can be clicked at any time, revealing the sentence inside (along with the photo if there as one) and information about the sentence’s author. As the particles move around the screen, they lose speed and eventually freeze as they ap- proach the mouse cursor, allowing them to be clicked. As the particles approach the bottom left corner of the screen, they become attracted to it and swarm around it, drawing the eye to a menu that gives access to the other five movements of We Feel Fine. The design of the Madness movement was motivated by Principle 7 in Section 2. The tiny colorful particles rep- resent a bird’s eye view of humanity — like standing atop a skyscraper and looking down at the street. People bus- tle to and fro, darting in and out of shops, hailing taxis, falling in love, laughing, handling personal crises. From the skyscraper, the notion of individuality is hard to recognize. However, once a particle is clicked, it explodes into its con- stituent letters, which form its sentence, and that particle becomes the center of attention. At this moment, the viewer sees the open sentence as the only one that matters. Figure 3: Madness Murmurs. Murmurs presents a structured environment in which to view feelings. As this movement begins, every par- ticle on screen gently floats upwards, eventually bouncing off the ceiling several times before settling. Then, one by one, the particles drop from the ceiling to join a simple scrolling list of feelings, organized in reverse chronological order. The sentences appear letter by letter, as if being typed by their author, and fade to black as new sentences appear. The strict formal constraints of Murmurs help to emphasize the polarities in the types of feelings present in the world. Some are mundane, some are funny, some are poignant. As they march along, the feelings begin to strike a common chord. Montage. Montage presents the feelings from a given pop- ulation that contain photographs, and displays these pho- tographs in a simple grid of variable size, depending on the number of photographs available. Any photograph in the grid can be clicked, causing it to zoom in to the size of the screen. When zoomed, a photograph’s associated sentence Figure 4: Montage Figure 5: Individual Montage is revealed. Any user may save a montage to the Mon- tage Gallery, allowing anonymous viewers of We Feel Fine to collaboratively curate an exhibit of beautiful, surprising, or otherwise interesting images. Mobs. Mobs consists of five smaller movements, each of which uses the particle system to configure its shape, color, distribution and physics to best express the different zeit- geists of: feeling, gender, age, weather, and location. For example, Mobs (Feeling) displays the most common feelings in the sample population on a histogram. In each of the other Mobs movements (Gender, Age, Weather, and Loca- tion), the particles similarly self-organize to form summary graphs. In Mobs (Location) (Figure 7), for example, they arrange themselves on a map. This movement aims to sum- marize the data set as per Principles 4 and 5 in Section 2. Consistent with Principle 6 we enable macro to micro transi- tions by allowing the user to click on any particle in a graph to see the underlying feeling sentence. Metrics. Metrics, the fifth movement, consists of five smaller movements. While Mobs expresses the features that are most frequently expressed in the sample population, Metrics expresses the features that are most differentially expressed Figure 6: Mobs (Age) Figure 7: Mobs (Location) from the global average. For example, Mobs shows that the feeling ‘better’ is almost always the most commonly ex- pressed feeling in any sample population, because it is such a common feeling. In Metrics, however, ‘better’ will only be listed if it’s much more expressed in the sample population than it is in a randomly sampled population. Mounds. Mounds, the final movement, is independent of the sample population, always displaying every feeling in our database, scaled and sorted in order of frequency. Each feeling is portrayed as a large bulbous mound, colored to correspond to the feeling it represents. The mounds jig- gle slightly when undisturbed, and bend away as the mouse cursor approaches their perimeter. A small scrollbar below the mounds allows the viewer to jump to a specific point in the feelings list. Above each mound is listed its feeling, along with its and the total number of occurrences in the database. 4.2 Usage Observation To evaluate our design, we observed usage in an informal lab- oratory setting. Additionally, as We Feel Fine has been dis- cussed extensively in the blogosphere by its users, we found it useful to examine the the blog posts and blog comments Figure 8: Metrics (Weather) discussing We Feel Fine. These posts and comments are clearly not a scientific sample, as only a small fraction of the most highly enthusiastic or web-savvy users will blog or comment about an application that they’ve used. Nonethe- less, we found that this examination was a useful comple- ment to the lab study, both in reinforcing some findings from the lab study, and suggesting further hypotheses. In fact, we were surprised at the extent to which the comments from participants in the lab study were similar to comments