CommSense: Identify Social Relationship withPhone Contacts via Mining Communications

Xuan Bao, Jun Yang, Zhixian Yan, Lu Luo, Yifei Jiang, Emmanuel Munguia Tapia, Evan WelbourneSamsung Research America - Silicon Valley, San Jose, USA

E-mail: {x.bao, j3.yang, zhixian.yan, annie.luo, yifei.jiang, e.welbourne}@samsung.com

Abstract—People around the world are more connected todaythan ever before. By making phone calls, sending text messagesand participating in online chats, mobile users are frequentlyinteracting with their social connections through multiple com-munication channels. This trend is expected to continue with theemergence of immensely popular communication apps on mobiledevices. Intuitively, these interactions on users’ mobile phones canreveal valuable information regarding their social relationshipwith their phone contacts. Understanding such relationship canhelp provide new services and improve users’ mobile experience.In this paper, we explore the opportunity to deeply understandthese social relationship through mining mobile communicationdata. By building an on-device mining framework called Comm-Sense, we show that automatically learning and understandingsuch relationship can efficiently support useful applications suchas categorizing mobile contacts, identifying their relative impor-tance, and automatically managing mobile contacts with verylittle human interference.

I. INTRODUCTION

The industry has witnessed the phenomenal growth ofsocial communication applications in recent years. WhatsApp[1] has attracted more than 420 million active users, especiallyamong younger generations. WeChat [2], a popular IM applica-tion with voice messaging functions, has reported a record-highachievement of 272 million active monthly users around theglobe. These applications, along with existing communicationchannels such as phone calls and text messages, have enabledmobile users to interact with people in their social circles morefrequently than ever before.

A closer look at these communications reveals a picture ofa unique real-life social network. Intuitively, the interactionsthrough mobile devices play an important role in defining andkeeping social connections between individuals. Professionalsmake phone calls during working hours with their businessconnections; friends and families often communicate in theevening to share experiences and feelings; acquaintances ex-change emails during holidays to broadcast updates. Unlikethe friendship on online social networks, the people we com-municate with through phone interactions tend to be strongersocial ties in our real life.

Deriving relationship for user’s contacts can consequentlylead to competitive advantages in designing mobile systemsand user interfaces. This is because social relationship betweena mobile user and her contacts have a fundamental impact onhow she would interact with these people in the future. Designswith such insights can minimize user effort and improve userexperience.

Being aware of this opportunity, we propose a socialcomputing method without relying on conventional socialnetwork analysis. Instead of looking into the data owned byonline social network (OSN) providers, which is often difficultto obtain, our system identifies users’ social relationship byanalyzing their communication interactions on mobile devices.Our intuition is that most of the mobile social interactions leavefootprints on the device in the form of call logs, SMS, emails,as well as chat logs (e.g., WeChat). Therefore, analyzing how

such interactions are structured can provide unique insightsregarding the social connections between a mobile user andher potentially hundreds of phone contacts. Moreover, sincemobile communication logs are universally available on ev-ery communication device, this method can potentially workacross multiple platforms without requiring users’ registrations(and their friends’ registrations) with specific social networks.

In this paper, we introduce CommSense, an intelligenton-device framework that automatically partitions mobile so-cial connections into meaningful categories such as Fam-ily, Friends, Colleagues and Insignificant via mining mobilecommunication data. The concept is illustrated in Figure 1.CommSense can enable a series of compelling technologies tooptimize user experience related to contact management, smartcontent sharing, and configuration customization. The rest ofthis section elaborates on some of the use cases and presentsthe demo applications we have developed.

Fig. 1: CommSense: Reveal social relationship via miningmobile communication interactions

A. Use Cases

CommSense enables many novel features to create inter-esting applications on mobile devices. We discuss some of theexample use cases in this section.

1) Contact Management. Contact lists on ones’ mobiledevice grow over time and can eventually become messy.Organizing ones’ contacts on mobile device can be painful,especially for people who may have several hundreds, evenmore than one thousand contacts. Manually grouping andlabeling each contact, even with simple group names such as“business”, “family”, can take a significant amount of time.CommSense can help solve this problem by automaticallycategorizing contacts into social groups according to the iden-tified attributes of each contact. For example, a contact whomthe user frequently interacts with during working hours, andalways promptly responses to could be an important co-workeror manager; a contact whom the user always meets at dinnertime (such information is detectable via Bluetooth) may be afamily member or a good friend. CommSense can effectivelyidentify such patterns thus automatically categorize contactsinto different groups with little user effort or involvement. Fig-ure 2-(1) shows a demo application, Smart Contact Book, built

to showcase the automatic contact management functionality.The long list of mobile contacts is arranged as a tile-basedlayout. Categories are prioritized based on real-time context- for example, colleagues are shown in big top tiles at workwhile friends may be prioritized during weekend evenings.

Smart Contact Book Smart Sharing

Fig. 2: CommSense Demo apps in Tizen, a Linux-based mobileoperating system [3]: (1) Categorize contacts into semanticgroups. (2) Share contents with individual groups.

2) Information Sharing within Social Groups. Selectivelysharing information among a certain group of people is anatural communication demand. A vacation photo that isperfect to share among family members and close friends, maynot be appropriate to be seen by colleagues at the work place;a work document, on the other hand, should only circulateamong contacts within the same organization. CommSense canintelligently help decide the range of these sharing activities.Figure 2-(2) shows another demo application that leveragesthe social relationship learnt from CommSense for contentsharing. The application allows the user to share messagesor content within certain groups in a more convenient fashion.Moreover, based on the analysis of mobile communication, theapp also identifies the best communication channels (e.g., chatapp, email or Bluetooth) for the sharing operation.

3) Distributed Online Social Networks. Privacy concernsremain to be a hurdle for today’s online social networks.Users usually have no choice but to trust their social networkproviders to handle all their personal contents. CommSensecan change this situation. On CommSense enabled mobiledevices, the users’ social groups are entirely derived from theirdaily communication and interaction patterns. The partitionof the groups is completely private to the user and remainsunknown to the outside world. Because of this, attributebased encryption can be applied to protect the communicationswithin each group’s chats, posts, feeds, etc. In fact, most of thesocial network functionality can be supported in such a fullydistributed fashion. In other words, CommSense provides away of forming distributed social networks by generating thelocal view for each user without relying on cloud servers.

II. PROBLEM STATEMENT

The key research problem for CommSense is to accuratelyidentify social relationship via mining multi-modal mobilecommunication data (calls and messages). We define social re-lationship categories as shown in Figure 3. All contacts are firstcategorized into important contacts and insignificant contacts.The important ones include Significant Others, Family, Friends

and Colleagues. Notice that Significant Others also belong toFamily. Insignificant contacts include Business, Acquaintance,unsaved phone numbers, and spams. In our experiment, we askusers to manually assign these labels to the important contactsin their phone book and use these labels as the ground truth.For insignificant contacts, the user only need to assign the“insignificant” label but can choose to assign additional labelsif they are willing to.

Fig. 3: Social relationship category definition in CommSense

A. Data Set Overview

For data collection, we deployed an Android application(called “EasyTrack”) through Elance and oDesk crowdsourc-ing platforms. EasyTrack is able to collect communication logs(calls, messages) along with the associated sensing data - time,location and network. In total, we have collected data from9899 contacts of 106 mobile users and obtained tags of contactrelationship manually labeled by the users as the ground truth.Figure 4 shows the distribution of users’ home locations acrossmultiple states within U.S. and some of them are located inMexico.

Fig. 4: User location distribution across US and Mexico

Table I shows a detailed overview of the distribution ofusers’ contacts among all categories. On average, each usercontributed 107 days of data. In total, 68529 phone calls and20000 messages during recent months are analyzed. Amongall the 9989 contacts, 1920 of them are labeled as importantcontacts (as defined before) by the user. Among the labeledcontacts, there are 60 Significant Others 1, 638 Families, 961Friends and 261 Colleagues. The remaining 7979 contacts areinsignificant contacts (e.g., Acquaintance, Business, others).

1Some users may have labeled multiple significant others – for both cellphone and home number.

TABLE I: Data Set Overview

Attributes StatisticsUser 106

Contacts 9899Labeled Important 1920Significant Other 60

Family 638Friends 961

Colleagues 261Insignificant 7979

B. Challenges

Translating the idea of CommSense to reality and enablingthe use case applications entail many research and implemen-tation challenges. In this section, we briefly explain the fourmajor challenges.

1) Skewed distributions across categories. As shown inTable I, among the potentially hundreds of contacts of oneuser, usually only one contact is the significant other. A smallportion of them are families and some others are friends andcolleagues. In other words, the “important contact” categoryonly accounts for a small part (20%) of the distribution, whilecertain sub-categories (e.g., Significant Others) only accountfor less than 1% of all the data. A naive classification approachmay classify all the instances as negative (e.g., not significantothers) and miss all the important information. A system needsto accurately pinpoint these rare but important contacts amongall others to actually make a usable system.

2) Heterogeneous behaviors. Users behave drastically dif-ferent from one another. Some heavy business users may havehundreds of contacts while many other users may only havetwenty to thirty contacts; the number of phone calls can rangefrom a few calls per month to hundreds of calls per week.These behavioral differences make users’ communication pat-terns towards contacts differ drastically even within the samecategory. For example, a business professional may call a“friend” a few times per week but a house wife may verywell make tens of regular phone calls to a close friend everyday. Therefore, establishing a generic model that works acrossall users posts a great challenge for this system.

To provide some concrete examples, Figure 5 shows thenumber of contacts each user has. Some of the heavy usershave hundreds of contacts while others may have as few asonly 15 contacts. Respectively, the patterns for these usersto make phone calls and messages also show great variation.Figure 6 shows the average daily communication distributionacross all users. Clearly, some users exhibit much more activecommunication patterns than others. On average, each userspends 17.8 minutes (with standard deviation of 19.3) in phonecall and sends 18.2 messages (standard deviation 5.2), showingthe heterogeneity across users.

0 20 40 60 80 1000

100

200

300

400

User ID

Num

ber

of C

onta

cts

Fig. 5: Some users may have far more/fewer contacts.

0

50

100

150

Dai

ly C

all D

urat

ion

(M

inut

es)

0 20 40 60 80 1000

100

200

User ID

Dai

ly M

essa

ges

Fig. 6: Avg. daily communication varies across users.

3) Data sparsity. Users’ communication patterns are ingeneral very sparse - for many contacts, often there are onlya few communications across the span of several months.Even for their frequent contacts, the total duration of phonecommunication is also only a small portion of a user’s dailylife. Therefore, a simple frequency based approach will notwork directly, and sophisticated features are required to revealthe social information hidden behind the limited amount ofcommunications.

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

Per

cent

age

of U

sers

Percentage of Contacts

70% Communication80% Communication

Fig. 7: CDF for contacts that account for more than 70%/80%of communications. Most users only frequently interact witha few contacts.

Figure 7 shows the CDF of users’ interactions with their topcontacts. Among all 106 users, many of them only frequentlyinteract with a small portion of their contacts. More precisely,for most users, 20% of their contacts often accounts for morethan 70% of their total phone calls. This behavior posts anotherchallenge for CommSense since there may not be sufficientinteractions between the user and some of her contacts forfurther classification. Moreover, phone communications ingeneral only account for a small portion of users’ daily life.As mentioned before, on average only 17.8 minutes of callsand 18.2 messages are observed per day, scattering all overthe entire 24 hours, meaning the derived spatial features needto be applicable to the sparse data.

4) System performance. Like any services running onmobile device, CommSense demands efficient usage of systemresources. Consider users’ communication log may grow intime; it is an interesting challenge to run CommSense with anaffordable resource budget.

C. Contributions

The main contribution of CommSense can be summarizedinto the following: 1) We present CommSense, a frameworkwith high performance on identifying social relationship ofphone contacts using heterogeneous mobile communication

data. We also provide a ranking method to further understandusers’ relationship with contacts in each category. Our coremethod uses statistical communication patterns which are plat-form independent, and works regardless of what language themobile users speak; 2) In CommSense, we design a discrimi-native set of interaction features that can better represent socialinformation. These features capture high-level spatio-temporalcorrelations and semantics embedded in users’ daily communi-cation. Using this feature set can significantly boost the miningperformance compared with the frequency-based approach;3) In CommSense, we propose a multi-stage classificationalgorithm to detect contact types incrementally. Our three-stagedetection can achieve significant performance increase com-pared to one-step multi-class classification and can prioritizeidentification of important contacts; 4) We conduct extensiveevaluations of CommSense using crowdsourcing data collectedfrom more than 100 users over six months. Furthermore, weimplement CommSense as a part of Samsungs Tizen platformwith high system performance

III. SYSTEM DESIGN AND APPROACHES

Figure 8 illustrates the system overview of CommSense,integrated with Samsung’s Tizen platform. Users’ mobile in-teractions are stored in the form of sqlite3 databases includingphone book, call log, message log, network, and locationinformation. CommSense retrieves this information through adaemon process that periodically processes the data in batches.For each contact in the log, a series of discriminative features,such as spatial-temporal features, statistics and interactionpatterns, are computed to characterize the communicationbehavior between the user and this contact. Following featureextraction, multi-stage classification is applied to identify thesocial relationship through supervised learning. Then, for eachidentified category, CommSense attempts to further understandthe “closeness” of each contact (e.g., is this a close friend ora regular friend) through a regression-based ranking process.Notice that CommSense on device uses only data locally avail-able from the user herself without relying on cloud assistance,which greatly preserves users’ privacy.

Fig. 8: CommSense System Implemented on Tizen platformA scheduler is used to duty-cycle CommSense in order

to minimize system resource usage. Overall, CommSense isintegrated into the Tizen software framework as a socialrelationship miner and exposes its social context APIs tosupport upper layer applications. The remaining sections willelaborate on design details with respect to the challengesdescribed before.

A. Patterns and Features in Communication

Engineering discriminative features is one of the mostimportant steps for performance. In CommSense, we havedesigned 61 features to represent users’ communications fromdifferent aspects. The key intuition is that the social relation-ship may affect the frequency, time and location of communi-cations, revealing additional information from the sparse data.Table II explains the features aggregated into 10 groups.

TABLE II: Feature Set OverviewFeature Group Name # of Features

1. Reaction to Incoming Calls 32. Weekly Pattern Analysis 9

3. Working Hour/Inconvenient Time 114. Burst of Communications 45. Continuity and Gap Days 26. Adaptive Time Windows 16

7. Location of Communication 98. Statistics of Longer Calls 49. Total Duration/Frequency 210. Saved in Contact Book 1

1) Reaction to Incoming Calls: These features measurehow users usually react to incoming phone calls from a par-ticular contact. Intuitively, people are more responsive towardsincoming calls from important contacts - it may not be a goodidea to skip an urgent phone call from the direct supervisorwhile a casual call from a friend can wait. Therefore, wemeasure how many incoming calls from each contact arepicked up or missed, how many missed calls are returned onthe same day and the ratio between incoming and outgoingcalls. These ratios are used to categorize users’ reactionstowards calls from a contact.

2) Weekly Pattern Analysis: Certain types of communica-tions exhibit a particular week-by-week pattern. For example, auser may regularly contact remote family members each week,even though the specific time may change due to schedule.Many conference calls or updates may also be arranged ona weekly basis. This group of features analyzes the weeklyrecurring pattern for communications, capturing communica-tions per week and its variations throughout the data collectionperiod.

3) Working Hour/Inconvenient Times: We define workinghours as 9 to 5 on weekdays and inconvenient times as earlymorning and late night hours. Regular communications duringthese time windows reveal information about whether a partic-ular relationship is profession related (e.g., colleagues or co-workers) or not. Usually, co-workers may communicate moreduring working hours and less so during those inconvenienttimes.

4) Burst of Communications: Sometimes, users may calla contact several times in a row, for example, to coordinategoing somewhere or meeting at some place. Similarly, usersmay message each other back and forth during a lengthyconversation. We define this phenomenon as a “burst” ofcommunications. In this feature group, we try to characterizethe burst phenomenon for each contact. Currently we definecalls that are within 20 minutes after the finishing time of theprevious call as belong to the same burst. The features capturethe total number of bursts as well as the typical burst length.

5) Continuity/Gap in Daily Communication: For certaincontacts such as significant others, we make phone calls almostevery day. Such continuity in communication may provide ahint about the importance of a particular contact. We define aday without any communication towards a contact as a “gapday” for that contact. These features measure how many “gapdays” are there for each contact and how long the longest gapis in order to characterize the continuity of communications.

6) Adaptive Time Window: One limitation for fixed timewindow is to handle communications around the boundaries.For example, if the time window boundary is set at 5PMbut one user happens to communicate around that time, thesecommunications will fall into two windows arbitrarily. Toavoid this phenomenon, we divide weekdays and weekendsinto adaptive time windows. These windows are defined topartition each user’s communication into separate bins thus tominimize the effect of communications near the boundaries.The communication frequency during each time window isthen calculated as temporal features.

7) Location of Communications: For some of the commu-nications, we have access to the associated location informa-tion - whether it takes place at home, at work, or somewhereelse. These features characterize the communication locationdistribution with a particular contact. The most dominantlocation among the distribution is also computed and used asa feature.

8) Statistics of Meaningful Calls: The entire call historysometimes contains many calls with short durations, presum-ably during which the users may pick up the phone, explainthat they are not available and immediately hang up. Frequentoccurrence of such behavior may distort the statistics ofaverage durations. This feature group filters out these shortcalls and compute statistics about longer, more meaningfulcalls (more than 2 minutes) only.

9) Total Duration and Counts: Total duration and counts ofcalls and messages may be a robust yet rough indicator of thestrength of social relationship. These features are expected torobustly distinguish obviously close contacts from insignificantones.

10) Saved in Contact Book: For classifying significantcontacts vs. insignificant contacts, we also use the feature“saved in contact book”. This feature is set to 1 for savedcontacts and -1 for the unsaved ones.

Together, all these features reveal the intrinsic patternsembedded in the sparse data. In general, we find that these richfeatures can significantly boost the performance compared tonaive frequency based approaches.

B. User-Adaptive Feature Normalization

After computation of the rich features, normalization isadopted to account for heterogeneity in user behavior. Forexample, suppose we naively train a model based on heavyusers that always make hundreds of phone calls per day, theresulted model may conclude that only contacts with heavydaily communications are friends. This model may not workfor typical users who may only receive a few friends’ callseveryday. If we simply include both heavy and light users ina training set, the discrepancy between their behaviors mayconfuse the classifier.

Our intuition is that the features eventually should reflectthe relative importance of a contact within the context of agiven user. In other words, the model needs to decide howmuch a contact weighs in relation to the communications withall of the contacts of a given user. Therefore, for each featuredimension in the feature vector (except “dominant location”and “saved in contact book”), all values are normalized by themaximum absolute value observed in the same dimension ofthe same user. This is different from traditional normalizationtechniques where values are normalized with the maximumobserved across the entire training set. In this way, the relativeranking of each contact within each user’s record along alldimensions are preserved.

C. Multi-Stage Classification Procedures

Using discriminative features and per-user normalizationhelp alleviate the problems of data sparsity and behaviorheterogeneity. However, the skewed distribution among classesstill remains to be a hurdle. Among over 9, 000 contacts,there are only 60 Significant Others and several hundredFamily and Friends, accounting for only a small portion ofthe population. Therefore, directly using a multi-class classifiermay not perform well and may miss some of the important butrare data points. Naively adding weight will result in manyfalse positives.

As shown in Figure 9, CommSense employs a three-stage classification approach with four decision tree classifiersand a rule-based module. The philosophy is to guarantee theperformance of the most important classes first and assignthe unavoidable errors to the non-essential classes. Also, thisdesign allows the potential to accommodate multiple labelsfor the same contact – for example, a contact could be both aSignificant Other and a Family member. In the beginning, thefirst step in the pipeline is to distinguish important contactsvs. insignificant contacts. This step filters out the massivenumber of business, marketing and other insignificant contacts.Then, during the second stage, within the important contacts,CommSense identifies Significant Others and Family membersversus Friends and Colleagues. We use two binary classifiersin parallel in this step to minimize the propagation error forthese two classes. For the remaining important contacts, thesystem attempts to separate friends from colleagues using thefourth classifier. In the end, the system makes estimates forinsignificant contacts using rule-based logic learned from userstudies. Details regarding the rules will be discussed in theevaluation section.

D. Contact Closeness Ranking

Besides categorizing contacts, CommSense also attempts torank contacts according to their “closeness” score, measuringthe strength of each social connection, with the user. Since it’sdifficult for users to manually give a closeness score to everycontact, we ask them to give scores to each category. Then,we design a regression-based ranking algorithm is designedto identify the appropriate weights for each feature dimensionin order to track the per-category closeness score. In the end,since contacts often have unique feature vectors, the learnedparameters together can be applied to derive the closeness foreach individual contact. The feature can help applications tofurther understand the social connections among each category.For example, among all the friends, now the closeness scorescan vary due to the difference in the strength of connections,thus distinguishing close friends from regular ones.

IV. EVALUATION AND INSIGHTS

In this section, we provide evaluation results regarding eachcomponent in CommSense. Considering the skewness of thedistribution, we define three metrics - accuracy, precision andrecall to more accurately quantify our performance for eachcategory of relationship C = {C1, · · · , Cn}. These metrics aredefined as:

Accuracy =|{Correctly Classified Samples}|

|{All Samples}|

Precision =|{Contacts Belong to Ci ∩ Contacts Classified as Ci}|

|{Contacts Classified as Ci}|

Recall =|{Contacts Belong to Ci ∩ Contacts Classified as Ci}|

|{Contacts Belong to Ci}|

Fig. 9: Multi-stage classification procedures.

For performance comparison, we derive the Fmeasure scoreas the harmonic mean of precision and recall: Fmeasure =2∗Precision∗RecallPrecision+Recall We also measure the error propagation

property from the confusion matrix, the mining performanceduring cold start, and the system resource consumption for afull evaluation.

A. Social Relationship Identification

Figure 10 shows the performance for classifiers in identify-ing relationship for each category. Notice here each classifier isevaluated against its inputs. For example, the friends/colleagueclassifier takes non-family members from previous steps asinput. Therefore, the performance for this classifier is mea-sured for its capability of distinguishing between friends andcolleagues among non-family members.

Overall, CommSense achieves over 80% precision amongall categories and near 90% recall for important contacts,significant others, and friends. The recall is relatively lowerfor families, close to 70%. Considering the extremely skeweddistribution (60 significant others and 640 families among 9899contacts), it is challenging to identify the few families amongall the contacts.

0

20

40

60

80

100

Acc

urac

y/P

reci

sion

/Rec

all

AccuracyPrecisionRecall

ImportantContacts

SignificantOthers

Family Friends/Colleague

Fig. 10: Performance per category (friends and colleagues aresplit by the same classifier).

B. Comparison with Alternative Approaches

We compare our performance against two alternative ap-proaches. One approach is to use a single-step multi-class clas-sifier instead of the multi-stage classifier. In our experiment,while keeping the feature set the same, the single-step classifieronly achieves a precision of 47.64% and 68.28% for family andfriends respectively, with recall lower than 50%. It performspoorly for other important categories (e.g., significant others,colleagues), showing that the skewed distribution makes thisproblem unsuitable for the single-step approach.

Another alternative is the frequency-based approach, whereonly the communication frequencies are used as featuresand the classification pipeline is kept the same. Figure 11

illustrates this comparison in Fmeasure. Clearly, CommSenseachieves much higher performance for important categoriesof significant others and families while exhibits comparableperformance for identifying important contacts and distinguishfriends versus colleagues. This shows the effectiveness of ourrich feature set for distinguishing various social relationshipwith phone contacts.

0

0.2

0.4

0.6

0.8

1

F−

Mea

sure

CommSenseFrequency−Based

FamilySignificantOthers

ImportantContacts

Friends/Colleague

Fig. 11: Comparison with frequency-based approach.

C. Effectiveness of Features

Table III illustrates the effectiveness of our feature setin distinguishing different contact categories. It lists the topfive features that contribute the most at each step. Noticethat, after pruning, the friend/colleague classification uses onlythree features. Through all these features, we can also learninteresting insights regarding how people may perceive theirsocial relationship and behave accordingly.

Interestingly, the total communication frequency is in factnot the best feature for any category. Certain features such asratios between weekday and weekend phone calls help capturethe differences between professional and family relationship.Other temporal features like communications during certainperiods also help boost the accuracy. These factors togethermake our method more effective than the frequency-basedapproach.

TABLE III: List of Effective FeaturesImportant Contacts Significant OthersTotal gap day - days without any com-munication

Total communication frequency

Ratio between incoming and outgoingcalls

Call frequency during noon to 6PM onweekdays

Total communication frequency Median of call durationsMedian of call durations Messages on weekday morningsWeekday message between 6PM andmidnight

Average call duration on Weekends

Family Friends/ColleaguesTotal call duration on Weekends Ratio between weekend phone call and

total call durationBursts of calls Total communication frequencyRatio between weekend phone call du-ration and total call duration

Calls on weekday mornings

Median of call durationsRatio of phone calls being returned onthe same day

D. Confusion Matrix

Besides evaluating the performance of individual classifiersat each stage, it is also important to analyze the overall finalperformance. This is because classification errors in earlierstages (e.g., identifying significant others) may be carried overto later stages (e.g., identifying friends), causing errors ineven high-performance classifiers. Table IV shows the finalconfusion matrix for CommSense, including all the propagatederrors. The rows denote the ground truth while the columnsshow the predictions. For example, the value of family rowcross friend column indicates the number of family contactsthat are falsely identified as friends.

The numbers along the diagonals show that most of therelationship are correctly identified. The most errors came fromthe cases that identifying insignificant contacts as friends (195among 8, 000), identifying friends as families (119) and identi-fying colleagues as friends (59). One reason for these errors isthat the social boundaries between certain categories are notas distinct. During our user study, some users acknowledgethat sometimes colleagues are also friends and some friendsare treated like families. Similarly, people may rarely contactcertain remote family members or friends. In fact, some of thefriends or family members were called only once in severalmonths, making it difficult to identify them.

TABLE IV: Confusion MatrixTrue/Predict Insignificant Spouse Family Friends ColleagueInsignificant 7701 3 75 195 5

Spouse 1 53 5 1 0Family 74 4 399 158 3Friends 116 2 119 719 5

Colleague 67 0 16 59 119

E. Cold Start for New Adopters

To be fully functional, CommSense requires user’s com-munication history. In reality, users may expect CommSenseto work soon after they purchase a new device. Figure 12analyzes how CommSense performs after users start to usea device. The two curves at the top show that as quicklyas one week after start using the phone, CommSense canachieve reasonably good performance of identifying importantcontacts as well as identifying user’s significant others. Theperformance for family and friends may fluctuate over the firstseveral weeks and eventually stabilizes after 7 weeks. Thefluctuation is caused by the fact that the user starts callingmore and more friends/families during this period. Therefore,each week, CommSense may observe contacts that are newto the system and it is difficult to identify them right aftertheir appearance. In the future, we expect the models learnedfrom old devices to be saved in the cloud. Then whenever theuser purchases a new device, this model can be immediatelyexported to bootstrap CommSense from the very beginning.

0 2 4 6 8 10 12 140.7

0.75

0.8

0.85

0.9

0.95

1

1.05

Weeks of Usage

Acc

urac

y

Important Contacts Significant Others Family Friend/Colleague

Fig. 12: Accuracy trend vs. weeks of data

F. Closeness Ranking

After categorizing contacts, CommSense ranks contacts ineach category in terms of their “closeness” to the user to find,for example, if a friend is a close friend or just a casualone. The closeness score is computed based on regressionmodels that approximate user assigned scores using the featureset as observations. Figure 13 shows the comparison betweenuser defined scores and CommSense scores. If the closenessscore is in the range of [0, 100], the estimation error is 16.6for in total 2341 labeled contacts collected from 106 users’data. CommSense is able to capture the general trend of thecloseness but still assigns close scores to some contacts withincertain regions.

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

Con

tact

s10 20 30 40 50 60 700

0.5

1

Con

tact

sCloseness Score

Fig. 13: Top: User defined scores. Bottom: CommSense scoresbased on regression.

G. Rules for Categorizing Insignificant Contacts

The last component in CommSense is the rule-based logicthat aims to make the best effort to tag insignificant contacts.These tags include shops/business, spam, wrong number, etc.Since the privacy sensitive information (actual phone number,contact names, etc.) are not collected, the rules are verifiedthrough crowdsourcing surveys. In the survey, the users areasked to evaluate the quality of each rule by observing theirown call log – for example, how likely calls from 800 numbersare business/spam or how likely a short incoming call thathappens only once is a call due to dialing a wrong number.In the end, the user report how accurate these rules are basedon their observations. The results of the survey is recorded intable V based on 11 users’ feedbacks.

TABLE V: Rules for Other ContactsTargets Category Accuracy

Start with 800, 801 Business, Spam 70%Incoming/short/once Wrong number 20%Web query available Shops, business 60%With same area code Local business 10%Foreign country code Remote families 50%Short/rarely returned Spam 50%

H. System Performance

As an on-device data mining framework, CommSensedemands not only high classification accuracy but also efficientsystem performance. Table VI lists the average performancemetrics of CommSense in terms of execution time, CPU usage,memory load, and power consumption and compare theseperformance metrics with other common tasks running onSamsung Tizen devices with five months of data. The averagememory load of CommSense is 2.36MB and the average CPUusage is around 6.7%. Since the total execution time is around

2.35 seconds, the overhead of CPU and memory usage is verysmall. The average power consumption is 227.2mW, and itconsumes a negligible amount of energy considering the fastexecution time. Note that in reality, CommSense only needs tobe executed once per day or even once a week on one user’sdata to keep the contact categorization information up to date,therefore the impact on battery life is rather insignificant.

TABLE VI: Average System Performance MeasurementsCommSense Comparison Tasks

CPU Usage 6.7% Play MP3: ∼ 12%Execution Time 2.35 secondsMemory Load 2.36 MB Play MP3: 5 − 7MBPower Consumption 227.2 mW Radio: ∼ 450mW

V. RELATED WORK

Analyzing the social connections between individuals hasinfluenced research areas such as social network analytics,crowdsourcing and phone sensing [4], [5], [6], [7], [8], [9].CommSense builds on work that falls roughly in two areas:social computing and group interaction analysis.

A. Social Computing

The social computing community has put a significantamount of efforts in identifying group structure in socialnetworks [10], [11] and measuring social network evolutions[12]. The research effort can be traced back to communitydetection in network science and social media, where the focusis on identify communities [13], [14]. All aforementionedsocial grouping approaches, however, are performed with aglobal view of the interaction data among all users and canapply graph related algorithms to the entire social network.CommSense adopts a user-centric approach that only uses datalocally available on users’ own devices without sharing withthe cloud. This helps avoid privacy breaches and does notrequire massive user registrations to a specific OSN.

B. Group Interaction Analysis

Certain interactions naturally involves a group of people– group emails, co-location, conference calls, etc. Recently,people have explored using such information for group detec-tion. Co-location patterns obtained from Bluetooth proximity[15], [16], [17] and mobility patterns [18] can be used toinfer friendships. Social media [10], caller-callee relationship[19], and emails [20] are explored to establish communicationgraphs among users. Instead of relying on occasional groupinteractions, CommSense investigates rich communication fea-tures on different modalities and works for every contactwithout limiting to group conversation participants.

VI. CONCLUSION

Mobile communications are shaping the way people so-cialize with each other. Hundreds of millions active users arecalling, messaging and chatting with their friends and familieseveryday through different channels. This paper explores theopportunity to understand a user’s social relationship viamining the fingerprints naturally left by communications. Thecore idea is to leverage the multi-modal information sourcessuch as calls, messages, time, location and network and learnhow these communication patterns can be translated to assessuser’s social relationship towards a contact. In the end, thelearned insights can be used to support services includingcontact management and information sharing across multipleapplications. Implemented as part of the Tizen framework,CommSense shows promising results for reliably identifyingimportant contacts with affordable system resource usage.

At this stage, CommSense certainly has many limitations.On the technical side, the current category definition is stilllimited and the performance also has room for improve-ment. We intend to explore additional opportunities beyondpattern analysis, potentially along the directions of real-timeweb queries and text analysis. On the social side, thoughCommSense is implemented completely on user’s device andshares no information with the cloud, it may still raise privacyconcerns especially when users falsely perceive the resultsas information leakage. Though we do not have a perfectsolution yet, some measures need to be taken to improveusers’ perception about privacy. With these limitations, westill believe there is value in building an on-device contactmining system. With users’ contact list growing forever, it isimportant to keep these information well organized withoutrelying solely on users’ effort. Functional with any mobilecommunication device and with no constraints in languages,We believe CommSense shows one potential direction towardssolving this problem via mining communication history data.

REFERENCES

[1] “Whatsapp,” http://www.whatsapp.com/.[2] “Wechat,” http://www.wechat.com/en/.[3] “Tizen,” https://www.tizen.org/.[4] A. Misra and R. K. Balan, “Livelabs: initial reflections on building a

large-scale mobile behavioral experimentation testbed,” ACM MC2R,2013.

[5] N. D. Lane, Y. Xu, H. Lu, A. T. Campbell, T. Choudhury, and S. B.Eisenman, “Exploiting social networks for large-scale human behaviormodeling,” IEEE Pervasive Computing, 2011.

[6] J.-K. Min, J. Wiese, J. I. Hong, and J. Zimmerman, “Mining smartphonedata to classify life-facets of social relationships,” in ACM CSCW, 2013.

[7] J. Wiese, J. I. Hong, and J. Zimmerman, “Challenges and opportuni-ties in data mining contact lists for inferring relationships,” in ACMUbiComp, 2014.

[8] N. Aharony, W. Pan, C. Ip, I. Khayal, and A. Pentland, “Socialfmri: Investigating and shaping social mechanisms in the real world,”Pervasive and Mobile Computing, 2011.

[9] Z. Yu, X. Zhou, D. Zhang, G. Schiele, and C. Becker, “Understandingsocial relationship evolution by using real-world sensing data,” WWW,2013.

[10] L. Tang and H. Liu, “Community detection and mining in social media,”Synthesis Lectures on Data Mining and Knowledge Discovery, vol. 2,no. 1, pp. 1–137, 2010.

[11] N. Barbieri, F. Bonchi, and G. Manco, “Cascade-based communitydetection,” in ACM international conference on Web search and datamining, 2013.

[12] P. Doreian and F. Stokman, Evolution of social networks. Routledge,2013.

[13] M. Saravanan, G. Prasad, K. Karishma, and D. Suganthi, “Analyzingand labeling telecom communities using structural properties,” SocialNetw. Analys. Mining, 2011.

[14] S. Fortunato, “Community detection in graphs,” Physics Reports, vol.486, no. 3, pp. 75–174, 2010.

[15] T. M. Do and D. Gatica-Perez, “Groupus: Smartphone proximity dataand human interaction type mining,” in ISWC, 2011, pp. 21–28.

[16] J. Zheng and L. M. Ni, “An unsupervised learning approach to socialcircles detection in ego bluetooth proximity network,” in UbiComp,2013, pp. 721–724.

[17] Y. Lee, C. Min, C. Hwang, J. Lee, I. Hwang, Y. Ju, C. Yoo, M. Moon,U. Lee, and J. Song, “Sociophone: everyday face-to-face interactionmonitoring platform using multi-phone sensor fusion,” in ACM Mo-biSys, 2013.

[18] E. Cho, S. A. Myers, and J. Leskovec, “Friendship and mobility: usermovement in location-based social networks,” in SIGKDD, 2011, pp.1082–1090.

[19] M. Seshadri, S. Machiraju, A. Sridharan, J. Bolot, C. Faloutsos, andJ. Leskove, “Mobile call graphs: beyond power-law and lognormaldistributions,” in SIGKDD, 2008, pp. 596–604.

[20] D. MacLean, S. Hangal, S. K. Teh, M. S. Lam, and J. Heer, “Groupswithout tears: mining social topologies from email,” in IUI, 2011, pp.83–92.