Multiview Social Behavior Analysisin Work Environments

Chih-Wei ChenDepartment of Electrical Engineering

Stanford UniversityStanford, California 94305

Email: louistw@stanford.edu

Hamid AghajanDepartment of Electrical Engineering

Stanford UniversityStanford, California 94305

Email: aghajan@stanford.edu

Abstract—In this paper, we propose an approach that fusesinformation from a network of visual sensors for the analysisof human social behavior. A discriminative interaction classifieris trained based on the relative head orientation and distancebetween a pair of people. Specifically, we explore human inter-action detection at different levels of feature fusion and decisionfusion. While feature fusion mitigates local errors and improvesfeature accuracy, decision fusion at higher levels significantlyreduces the amount of information to be shared among cameras.Experiment results show that our proposed method achievespromising performance on a challenging dataset. By distributingthe computation over multiple smart cameras, our approach isnot only robust but also scalable.

I. INTRODUCTION

Camera networks have been widely deployed for many ap-plications, including security surveillance and traffic monitor-ing. Meanwhile, progress in computer vision has enabled theautomation of content analysis on the data captured by thesesensors. As the size of the network grows and the complexityof the information to be extracted increases, a centralized pro-cessing paradigm seems incompatible. Distributed algorithmsemerge as the more suitable solution to the problem, whereraw data are processed locally. This mitigates the problem oflimited network bandwidth and the computing capacity of asingle processor.

In this paper, we develop algorithms for social behavioranalysis of people in workplace environments. The proposedmethods are designed for distributed smart cameras. In partic-ular, we design a tracking algorithm that tracks multiple peoplein a real, unconstrained environment. Human head posesare then estimated, and a discriminative interaction detectordiscovers the dyadic social interactions in that environment.The processing is done locally at each camera. Informationextracted is then exchanged across the camera network toobtain more accurate, reliable decisions.

The study of proxemics [1] has revealed the close corre-lations between physical distance and social distance. In thatsense, locating people not only assists the understanding ofhow people utilize a given space, but also brings to light howinteractions are conducted. However, tracking multiple peopleis a challenging task, especially in crowded scenes wherepeople are often occluded and the background is cluttered. Wefollow a tracking-by-detection paradigm [2, 3] that has been

widely used. An efficient Chamfer matching [4] comparesan Ω-shape head-shoulder template [5] against the edge mapof the incoming video frame. A low-level tracking algorithmbridges the gaps between detections. Results from each cam-era’s tracker are combined, using the geometric homographiesamong cameras as constraints. The most probable locations ofthe head, i.e. the location most cameras agree on, can thenbe determined. The use of multiple cameras helps resolve theambiguity caused by occlusion.

To understand human behavior, the focus of attention pro-vides important information and evidence. It reveals not onlyhow people interact with each other, but also how peopleinteract with the ambient environment. The gaze direction isarguably the most important indicator of a person’s attention.Head orientation contributes much to gaze direction, and focusof attention estimation based on head orientation has beenshown to have high accuracy [6]. Therefore, we propose toextract the head poses for the understanding of human socialbehavior, as the head orientation indicates the intended targetof interaction.

For surveillance cameras, the faces are usually at a distanceand therefore small in the acquired videos. As a result, accurateextraction of facial features is difficult, if not impossible. Weadopt an image patch based method, where the head poseis estimated from normalized head images via learning. Inthis work we consider only the yaw rotation of the headpose. The estimation problem is treated as a classificationproblem, where the head orientation space is discretized todifferent classes. Image segmentation divides the image intoskin, non-skin foreground, and background pixels. Random-ized ferns [7], a special type of randomized decision treealgorithm, are trained to classify the image patch to one ofthe head poses. We extend the method to the multi-viewframework [8], and show the improvement over monocularresults.

The area we would like to address is human interaction.Automatic extraction of such information not only facilitatessemantic video retrieval but also allows us to answer questionssuch as “how frequently do they talk to each other?” or “wheredo the group meetings take place?” The type of interaction wefocus on in this paper is direct, primitive interaction betweentwo people. Such interaction has two prominent features:

proximity and eye contact. We train a discriminative classifieron dyadic interactions using the distance and relative headpose as the feature.

The contributions of this paper are as follows. First, wepresent a distributed framework for human localization, headpose estimation and social interaction detection. Second, foreach task we propose algorithms tailored for distributed cam-era systems where each camera processes the video locally andfusion is done in a higher abstraction. Finally, we compare theperformance of our proposed approach to single-view results,and evaluate the performance of the distributed algorithmsat different levels of information fusion. The evaluations arecarried out on a real, challenging dataset where people interactnaturally in a real office environment.

II. RELATED WORK

Extensive research has been devoted to human detection andtracking from a single camera [2, 9]. The most challengingproblem in monocular detection and tracking is occlusion.To address this problem, multiple cameras are used. Newalgorithms have been developed, and existing ones extended,for the multi-view scenario [10, 11]. Researchers attemptingto understand human motion have also adopted multi-cameraframework, including human pose recovery [12] and activityrecognition [13]. Human behavior in an office environment hasbeen studied by using prior knowledge about the layout of theenvironment to associate places to activities [14]. Instead ofcomprehensive understanding of human activities, we focuson the interactive behavior of people. Social interactions canbe identified by analyzing people’s walking pattern [15]. In[16], social groups and their leaders are identified by linkingpeople using their similarity in motion. We address a morestatic work environment where social links are determined bypeople’s location and focus of attention.

Existing research on head pose estimation mainly falls intoone of the following categories. Template-based [17] methodsmatch the incoming image to a set of templates, and estima-tions are assigned by the most similar ones. Learning-basedmethods [18, 19] learn the function between the input imageand the pose, either via classifiers or regressors. Embedding-based methods [20, 21] discover the low dimensional manifoldand the embedding. A comprehensive survey can be found in[22]. Because of the nature of the applications, e.g. humancomputer interaction (HCI) and driver visual attention moni-toring [23], most research focuses on head pose in the rangeof −90 to 90. Our pose estimation considers full pose range,where people move around freely in the test environment.We learn randomized ferns using cues from skin color andbackground segmentation, and extend it to the framework ofmultiple cameras.

III. SOCIAL BEHAVIOR ANALYSIS

In this section we describe our approach to extract the nec-essary information for social behavior analysis. In particular,we detail the tracking algorithm, review the random fernsclassifiers and their application to head pose estimation, and

finally discuss how location and head orientation are used todetect human interaction.

A. Tracking

We first explain how tracking is done on each individualcamera. Instead of full body tracking, only the head is trackedbecause of its direct applicability for head pose estimation.Moreover, the body is often occluded in a work environment,e.g. behind the desk or chair. To track multiple people in theenvironments, we adopt a tracking-by-detection approach asin [24]. The tracker integrated two methods complementary toeach other: detection association and low-level tracking. Fordetection, foreground mask is first obtained by constructing anadaptive Gaussian-mixture background model for the incom-ing video stream. An Ω-shape head and shoulder silhouettetemplate is then matched against the edges from foregroundobjects. The template matching is performed at multiple scalesusing Chamfer distance, and thus is robust to backgroundclutters and appearance variations.

Detection is associated with the current track based on bothits geometric overlap with the current bounding box, and itssimilarity with the current appearance template. The similarityis measured by the normalized correlation of the two imagepatches. This appearance attribute is also the feature our low-level tracking is based on. When there is no detection, wesearch in a local region near the current track location, andfind the image patch that achieves maximum correlation withthe current track. If the maximum score is below a threshold,the previous image patch is kept as the appearance template.Otherwise the appearance template is updated by the imagepatch at the new location. The two modes are complementary:the low-level mode fills the gap between detections (due toocclusion or static foreground merging into the backgroundmodel), and the detection mode recovers the track once thetarget reappears.

While biometric recognition methods have enabled theidentification of people from visual sensors, we incorporate anRFID system to help the initialization of the tracker. An RFIDsensor and an auxiliary camera are installed at the entrance ofthe office. When a person enters the office, the RFID he carriestriggers the recording system, and a color histogram is built tomodel the appearance of that person on that day. The personis then tracked throughout the day. Our setup also enabled thedataset collection of people’s appearance over time, which canbe subsequently used to learn a recognition system.

B. Localization

Once the head location of the target person in the image isknown, his location can be estimated by mapping the 2D imagelocation to real world 3D coordinates. In monocular view, witha calibrated camera, this can be done if the height of the targetis known. In an office environment, people are either standingor seated most of the time. Other poses only appear brieflyand do not contribute much to the overall statistics of thebehavior analysis. Therefore, two heights are used to estimatethe location, one for standing and one for seated people. Most

of the time only one of the two hypotheses would be valid.In the case of ambiguity, the location closer to the previousestimated position is used.

When multiple views are available, we use the follow-ing method to combine information from all views. LetI0, I1, In−1 be the images obtained from n cameras. Withoutloss of generality, let I0 be the reference image. Denotethe observation at location l in the reference image x. Itscorresponding observations in other views, x1, . . . , xn−1 canbe found by computing the homography to the respectiveimages. Let X be the event that the pixel at location llies in the target’s head H . Following the formulation in[10], the probability of such an event given the observationsx0, x1, . . . , xn−1 can be written as

P (X|x0, x1, . . . , xn−1) ∝ P (x0, x1, . . . , xn−1|X)P (X)

∝ P (x0|X)P (x1|X) . . . P (xn−1|X)

where Bayes’ law is applied in the first line, and the sec-ond line follows assuming conditional independence amongdifferent cameras. By the homography constraint in [10], theprobability of P (xi|X) is proportional to f(xi), the probabil-ity that xi belongs to the foreground head. In [10] the planarhomorgraphy between cameras is derived from ground planeto identify the feet. In our case, the homography is calculatedfrom the plane parallel to the ground at the target’s height,since the region of interest is the person’s head.

In our framework, a probability is assigned to the headlocation based on the mode the tracker operates in. In de-tection association mode, the detection score is converted toa probability. During low-level tracking, the probability isproportional to the correlation score. The probability indicatesthe likelihood that a region belongs to the foreground head ofthe target person, and is combined across cameras using themethod above. More specifically, a synergy map is constructedby choosing a reference frame, warping the likelihood mapof other views to this reference view, and multiplying thelikelihood together. An example is shown in Figure 1.

C. Head Pose Estimation

Here we treat the head pose estimation as a classificationproblem, where the 360 degrees yaw rotation space is uni-formly discretized into N distinct views. We show in Figure 2the 8 classes classification, and example head images for eachclass. Head pose estimation is sensitive to image alignment.To ameliorate the problem, an ellipse is fitted to the contourof the head, and the image patch centered at the ellipse is usedas the input to the classifier. The scale of the ellipse is varieddepending on its distance to the camera.

Randomized ferns [7] have been applied to the estimation ofhead poses [25]. A fern is a special type of decision tree wherethe tests at equal depth are the same. Assume F1, F2, . . . , Fn

are the ensemble of randomized ferns, I the head image tobe classified, and C = c1, . . . , cN the class label. Eachleaf node in the fern carries a posterior probability learnedfrom the training examples. The image I traverses down fernsaccording to the decisions at each interior node. Suppose the

(a) Multiview localization.

(b) Synergy map and thresholded result.

Fig. 1: Human localization from multiview cameras. (a) Whilethe target subject is occluded in Cam 0 , as illustrated by theyellow bounding box, she is visible in the other two viewsand is correctly tracked by detection , as shown by the redrectangles. Foreground head likelihood is assigned to eachbounding box by the tracker. (b) We show the synergy mapby warping the likelihood maps to the reference view, Cam 0,and multiplying them. By applying an appropriate threshold,the correct location of the occluded person can be recoveredas illustrated by the green bounding polygon.

image reaches a leaf node in the kth fern with the posteriorP (C|Fk, I). The head patch is then classified to class h suchthat,

h = arg maxjP (cj |F1, . . . , Fn, I)

= arg maxj

n∑i=1

P (cj |Fi, I).

Arithmetic mean is taken to combine estimations from all fernsbecause of its proven effectiveness and robustness to bias [25].

We now describe how the ferns are constructed and how theposteriors are learned. The training examples are cropped headimages normalized to a predefined size, and manually labeledthe ground truth orientation. Each pixel in the image patchesis segmented into one of the three labels: skin, non-skinforeground, and background. Background pixels are obtainedfrom background segmentation, and skin pixels are identifiedby a color-based skin detector in HSV color space. The test at

each fern node is then based on the label. For example, the testcould be whether or not the pixel at location (x, y) is a skinpixel. The locations and pixel labels are randomly selected toconstruct the random ferns. Examples of the segmented headimages and randomized ferns can be found in Figure 3.

The training examples are passed down the kth fern to reachthe leaf nodes. Each leaf node keeps a histogram of the classlabel. After all training examples are processed by the fern,the posterior P (C|Fk, I) for a given image I is computed bynormalizing the histogram at the node I reaches. The classlabel can then be estimated from the equations above.

Fig. 2: Head images are classified to one of the eight orienta-tion classes. Examples of head images tracked by our systemare shown. The resolution of the head image is low, and theappearance of the target people exhibits a large variation.

Fig. 3: Head pose estimation. Left: The input image patch,with a fitted ellipse, and the segmented image. From lightcolor to dark, the three labels are: skin, non-skin foreground,and background. Right: An ensemble of random ferns, andexamples of tests at each depth. The thick edges indicate thepath taken by the input. Each leaf node carries a posteriorprobability distribution, which is combined across the ferns togive the classification result.

D. Interaction Classification

We propose to train a discriminative classifier for dyadicprimitive interactions. In particular, a support vector machine(SVM) classifier is trained. The features include the distancebetween the pair of people and their relative head poses.The selection of the feature is based on the observations thatpeople interacting tend to look at each other in the face andstay close to each other. Formally, the feature vector is theconcatenation of the relative distance and the relative headangle. The head pose estimation in our system generates aprobability distribution over the N orientations, which canbe expressed as an N -dimensional vector. Denote the head

orientation estimations of a pair of people from a camerao1, o2 ∈ <N , respectively. Assume the two subjects forman angle θ with respect to the camera. The relative headorientation of the two is then |o1 − o2 − θ|.

IV. DISTRIBUTED SMART CAMERAS APPROACHES

In this section we describe how our algorithms are applied,or extended, to the distributed smart cameras framework.Location, head pose, and interaction estimated by each smartcamera are combined to reach a more accurate decision.Information from distinct views can be fused at various levels,each requiring different amounts of data to be transmittedover the network. We explore several possibilities and theircorresponding algorithms.

A. Centralized Processing

In a traditional setup, all cameras stream the raw data toa central processor. Therefore, the scalability of the sensornetwork is limited to the bandwidth of the underlying networkand the processing capacity of the central processor.

B. Location and Head Pose Fusion

The localization method we employ does not require imagesfrom different views be aggregated at a central processor.In fact, only the homography and the tracked head locationfrom the cameras that share an overlapping field of vieware needed for a local camera to leverage synergies amongindividual sensors and obtain a more robust estimation. Sinceonly static cameras are considered, the field of view overlap isdetermined by the geometric configuration of the cameras anddoes not change with the dynamic scene. When a target is notdetected in a view, an uniform distribution will be returnedif an estimation is requested by other cameras. Therefore thedecision will not be affected by missed detections.

The randomized fern can be readily extended to a multi-view scenario. Using the same notation from the previoussection, a head patch is classified to the angle class h thatsatisfies

h = arg maxj

∑k∈N

n∑i=1

P (φk(cj)|Fi, I). (1)

where N is the set of cameras where the target is observable,including the local camera itself. The function φk(cj) mapsthe orientation class in the local camera to the correct repre-sentation in the kth camera, relative to its line of view. Thedistributions estimated by each camera are averaged together.An angle class is given a higher posterior probability if morecameras agree on it.

Equation 1 is equivalent to having multiple observations,and classifying them using sets of randomized ferns. Withoutactually sending the image patches, the proposed methodachieves the same performance as a centralized approach whiledistributing the computation across different cameras.

C. Interaction Fusion

Finally, we consider information fusion at the interactiondecision level. That is, each smart camera detects interactions,and only detection results are combined. This requires the leastamount of data be shared for the cameras to collaborate ininteraction detection. The intuition is that false positives by asingle camera can be recovered if other cameras have correctclassification results.

A common approach to combine estimation from multiplecamera is to take the average [8]. However, we argue thatnot all estimations should have equal importance in the finaldecision. For example, if from one view the camera hasdetected that the target is occluded, the pose estimation, andhence the interaction detection, are likely to be erroneous.We therefore propose a probabilistic method where eachcamera reports a confidence score for itself, as well as for itsneighboring cameras based on the knowledge of their relativegeometric configuration. The final decision is made by takinginto account all the estimations and their respective confidencescores.

Assume that for an test example, the output score from theith view’s SVM classifier is hi. Its confidence score for itselfis the same probability used for constructing the synergy mapduring joint localization. This score is given by the tracker andreflects its confidence in localization. Denote this score wii.Notice that the dyadic interaction classifier operates on a pairof people, so the average of the two localization probabilitiesis taken as the score. Based on its own estimation of thehead poses and their relative location, the jth camera makesa decision on the interaction and has an SVM score of hj .The weight on the jth camera assessed by the ith view, wij ,is proportional to the classification accuracy of the head poseestimator for a particular pose. The weight is learned duringtraining of the head pose estimator, and has a multimodaldistribution for different local head pose estimation. The ideais that some head poses are more difficult than others, e.g. theback of the head is less discernible than frontal face. Again theweight is the mean of two weights, for each of the person inthe interaction pair to be evaluated. The final score combiningall n views is then

H =1

n

n∑i=1

n∑j=1

wijhj . (2)

V. EXPERIMENTAL RESULTS

As opposed to experiments conducted in controlled environ-ments with people instructed to act in a certain manner, weevaluate our methods on a challenging dataset [24] of videoscaptured in a real office where people behave naturally. ThreeAXIS network cameras are installed in 3 different cornersin the office. The schematic layout of the office, as well assample frames from the cameras, are shown in Figure 5. Thecameras capture VGA (640× 480 pixels) video at 30 fps, andare synchronized to a Network Time Protocol (NTP) server.

Test data. The system captures the daily life of peopleworking in the office environment, where people greet, talk

and have meetings occasionally. Tens of hours of videohave been recorded. A short sequence is extracted from therecordings for testing. The test video sequence is about 11minutes and 30 seconds long, and contains 5 people interactingin diverse ways. The 5 subjects have very different appearanceattributes, such as hair style and facial features, making headpose estimation a challenging task. The locations, head posesand primitive interactions in the test video are mansuallylabeled for evaluation.

Localization. We quantify the localization accuracy. A trackis considered correct if the overlap of the ground truth and thetracked bounding box is at least 50% of both rectangles. Thereare 5 tracks for the 5 people in the video. Ground truth headbounding boxed are annotated for all 5 tracks in all 3 views forthe entire video, sampled at 6 fps. Each view contains 4150synchronized frames.

We use Cam 0 as the reference frame. The synergy map iswarped to it, and we report the performance of the multi-viewmethod using the ground truth label on the reference frame.Using information from multi-view cameras, the precisionimproves from 93.8% to 97.8%. We show the number of falsepositives for each view in Figure 4. The reduction in falsepositives reflects the multi-view system’s ability of resolveocclusion.

Fig. 4: Localization results. We plot the false positives of allfive tracks. We compare the performance of each individualmonocular camera to the results obtained from using all3 views. Using information from multi-view system greatlyreduces the number of false positives.

Head pose estimation. We compare the head pose es-timation performance from single-view and multi-view. Wecompare our method to several baseline methods. First of all,the head pose can be estimated from monocular view. Thatis, each camera classifies the head patch it discovers. Theoverall accuracy for all 3 cameras is 0.53. A closer look at thefailure cases suggests that incorrect localization by the trackercontributes to many of the misclassified cases. We thereforeconsider a second baseline, where each camera first agreeson the best location of the target. The synergy map is thenprojected back to each view, and an ellipse is fitted to the newlocation. The classifier then uses the new location as the input.The accuracy is improved to 0.61. While a better localizationmitigates the problem of misalignment, it still cannot recoverthe correct orientation if the target person is occluded in a local

Fig. 5: Examples of discovered interaction. We show people’s locations estimated by our multi-view system as colored nodes.A red edge connects two nodes if our classifier detects a direct dyadic interaction.

view. Multi-view system must be used to obtain a more reliableestimation. Our multi-view head pose estimation achieves anaccuracy of 0.69.

Interaction. The test sequence contains several instancesof different forms of interactions. At the beginning of thesequence, two people talked to each other in their office chairs.Then the other three started a meeting among themselves, andpeople turned their faces when engaging in conversations withdifferent people. After both meetings were dismissed, twopeople began working together at the same desk. Finally agroup discussion involving all people took place at the centerof the office. We sample dyadic interaction instances fromthe sequence. About 200 frames are used as the test set, andwe run our interaction classifier on it. Examples frames anddiscovered interactions are shown in Figure 5.

The average precision from a single view camera, Cam 0, is0.71. The multi-view system that uses the aggregated locationand head pose estimation from all cameras as the input tothe trained SVM achieves an average precision of 0.83. Ifonly interaction decisions, made independently by each camerabased on its own estimation of location and head poses, arecombined, the average precision is 0.79. Information fusionat interaction decision level has comparable performance withthe full multi-view method, and only requires little informationbe shared among the cameras in the network.

VI. CONCLUSION

We introduce a multi-view framework for the understandingof human social behavior. The performance of the proposedmethods has been demonstrated on a challenging dataset. Inparticular, we have seen that information fusion at a highly ab-stract level can achieve promising results, making distributedsmart cameras a compelling framework for its robustness andscalability.

Currently interactions are detected independently on eachframe. Temporal information can be incorporated to removenoise, and to provide more insight about the interactions. Asfuture work, we plan to further analyze the structure of theinteractions, e.g. group meetings, presentations, and the socialstructure, such as each individual’s social role, based on ourresearch results.

REFERENCES[1] E. T. Hall, “A System for the Notation of Proxemic Behavior,” American

Anthropologist, vol. 65, no. 5, pp. 1003–1026, 1963.

[2] T. Zhao, R. Nevatia, and B. Wu, “Segmentation and tracking of multiplehumans in crowded environments.” IEEE PAMI, vol. 30, no. 7, pp. 1198–211, Jul. 2008.

[3] X. Ren, “Finding people in archive films through tracking,” in IEEECVPR, 2008.

[4] G. Brogefors, “Hierarchical Chamfer Matching: A Parametric EdgeMatching Algorithm,” IEEE PAMI, pp. 849–865, 1988.

[5] T. Zhao and R. Nevatia, “Stochastic human segmentation from a staticcamera,” in IEEE WMVC, 2002, pp. 9–14.

[6] R. Stiefelhagen and J. Zhu, “Head orientation and gaze direction inmeetings,” in ACM CHI, 2002.

[7] M. Ozuysal, P. Fua, and V. Lepetit, “Fast keypoint recognition in tenlines of code,” in IEEE CVPR, 2007.

[8] M. Voit, K. Nickel, and R. Stiefelhagen, “A Bayesian Approach forMulti-view Head Pose Estimation,” in IEEE MFI, 2006.

[9] B. Leibe, E. Seemann, and B. Schiele, “Pedestrian Detection in CrowdedScenes,” in IEEE CVPR, 2005.

[10] S. M. Khan and M. Shah, “A multiview approach to tracking people incrowded scenes using a planar homography constraint,” in ECCV, 2006.

[11] D. Focken and R. Stiefelhagen, “Towards vision-based 3-D peopletracking in a smart room,” in ICMI, 2002.

[12] C. Wu, H. Aghajan, and R. Kleihorst, “Real-Time Human PostureReconstruction in Wireless Smart Camera Networks,” in IPSN, 2008.

[13] C. Wu, A. Khalili, and H. Aghajan, “Multiview activity recognition insmart homes with spatio-temporal features,” in ACM/IEEE ICDSC, 2010.

[14] D. Ayers and M. Shah, “Monitoring human behavior from video takenin an office environment,” Image and Vision Computing, vol. 19, no. 12,pp. 833–846, Oct. 2001.

[15] H. Habe, K. Honda, and M. Kidode, “Human Interaction Analysis Basedon Walking Pattern Transitions,” in ACM/IEEE ICDSC, 2009.

[16] T. Yu, S.-N. Lim, K. Patwardhan, and N. Krahnstoever, “Monitoring,recognizing and discovering social networks,” in IEEE CVPR, 2009.

[17] D. J. Beymer, “Face recognition under varying pose,” in IEEE CVPR,1994.

[18] S. Z. Li, L. Gu, B. Schlkopf, H. Zhang, Q. Fu, and Y. Cheng,“Kernel machine based learning for multi-view face detection and poseestimation,” in ICCV, 2001.

[19] Y. Li, S. Gong, J. Sherrah, and H. Liddell, “Support vector machinebased multi-view face detection and recognition,” Image and VisionComputing, vol. 22, no. 5, pp. 413–427, May 2004.

[20] V. N. Balasubramanian, J. Ye, and S. Panchanathan, “Biased ManifoldEmbedding: A Framework for Person-Independent Head Pose Estima-tion,” IEEE CVPR, 2007.

[21] D. Huang, M. Storer, F. De La Torre, and H. Bischof, “Supervised LocalSubspace Learning for Continuous Head Pose Estimation,” in IEEECVPR, 2011.

[22] E. Murphy-Chutorian and M. Trivedi, “Head pose estimation in com-puter vision: A survey,” IEEE PAMI, vol. 31, no. 4, pp. 607–626, Apr.2009.

[23] P. Smith, M. Shah, and N. da Vitoria Lobo, “Determining driver visualattention with one camera,” Intelligent Transportation Systems, vol. 4,no. 4, pp. 205–218, 2003.

[24] C.-W. Chen, R. C. Ugarte, C. Wu, and H. Aghajan, “Discovering SocialInteractions in Real Work Environments,” in IEEE FG, Workshop onSocial Behavior Analysis, 2011.

[25] B. Benfold and I. Reid, “Colour invariant head pose classification in lowresolution video,” in BMVC, 2008.