attentive human action recognition - harvard university
TRANSCRIPT
Attentive Human Action RecognitionMinh Hoai Nguyen [email protected]
2
Human Behavior Analysis
Action/Interaction/Activity
3
Human Behavior Analysis
Facial BehaviorAction/Interaction/Activity
Gaze and visual attention
1. Recognition (Past)2. Prediction (Future)3. Early Recognition (Present)
Why should we care about human behavior?
4
Applications of Human Behavior Analysis
Security
UK has > 1.8M CCTV cameras
Healthcare
US: 3.4% suffer major depression
5
Human-robot interaction
Is she initiating
a handshake?
Network optimization
Human Behavior Analysis
6
Supporting problemsHuman/Hand Detection
Crowd counting
512 people
Machine Learning problems:Main problems:Human actionFace behaviorVisual Attention
RecognitionEarly Recognition
Prediction
• Unlabeled samples• Weak/noisy annotated samples• Complementary samples
Interdisciplinary problems
• Fundamental questions: • Human behavior• Human perception • Human learning
Human Behavior Analysis
7
Supporting problemsHuman/Hand Detection
Crowd counting
512 people
Machine Learning problems:Main problems:Human actionFace behaviorVisual Attention
RecognitionEarly Recognition
Prediction
• Unlabeled samples• Weak/noisy annotated samples• Complementary samples
Interdisciplinary problems
• Fundamental questions: • Human behavior• Human perception • Human learning
8
Talk Outline
• Pulling Actions of out Context
• Active Vision for Early Recognition
9
Goal: Recognizing Human ActionsDriveRun Kiss
• Human action recognition is challenging due to: • The subtlety of human actions• The complexity of video space
• It’s difficult to identify and attend to the relevant information
10
A Common Approach• Collect positive and negative data and train a classifier• For example, consider training a KISS classifier
kiss
kiss
kiss
Positive Examples
Sit Up
Drive
Handshake
Negative Examples
SVMSVM
11
A Common Approach• Collect positive and negative data and train a classifier• For example, consider training a KISS classifier
kiss
kiss
kiss
Positive Examples
Sit Up
Drive
Handshake
Negative Examples
+ Context
+ Context
+ ContextSVM
Problem
kiss + ContextSVM
• Various types of context– Background context, e.g., living room– Camera context, e.g., slow panning camera– Situation context, e.g., a dance
• Context often co-occurs with the actions– The learned classifier focuses on the contextual
cues instead of the action content– The learned classifier does not generalize well
12
Context
13
Our Goal
kiss
kiss
kiss
Positive Examples
Sit Up
Drive
Handshake
Negative Examples
+ Context
+ Context
+ Contextkiss + ContextSVM
14
Technical Challenges
kiss
kiss
kiss
Positive Examples
Sit Up
Drive
Handshake
Negative Examples
+ Context
+ Context
+ Contextkiss + ContextSVM
Co-occurring
Co-occurring
Co-occurring
15
One Approach: Provide Detailed Annotation
Action Context
• But it is difficult to obtain detailed annotation:• Laborious process => unscalable• Ambiguous
Handshake
Context
16
Difference
Action Context
Action sample
Conjugate sample
Our Idea
Context
Action
17
Action Context
Action sample
Conjugate sample
Our Idea
Context
Similarity
Context
18
Action Context
Action sample
Conjugate sample
Our Idea
Context
Similarity
Context
Difference
Action
How to obtain Conjugate Samples?
19
ActionPre-Action Post-Action
Action SampleConjugate Sample
20
How to use conjugate samples effectively?
Action Context
Action sample
Conjugate sample
Context
1. Subtract conjugate sample from action sample
- No direct pixel to pixel correspondence
Possible approaches
2. Use conjugate samples as negative examples
- Ignore the importance of context for recognition
3. Use conjugate samples as positive examples
- Noisy labels!
Proposed Framework
21
Action sample
a
Conjugate sample
c
f(a): action extractorf
g(a): contextextractorg
f(c): action extractorf
g(c): contextextractorg
SimilaritymeasureLs
Difference measure
Ld
Action is different
Context is similar
• Separate Action & Context factors
22
Action sample
a
Conjugate sample
c
f(a): action extractorf
g(a): contextextractorg
f(c): action extractorf
g(c): contextextractorg
: classifierh Classification loss
Lc
SimilaritymeasureLs
Difference measure
Ld
Proposed Framework• Separate Action & Context factors• Combine Action & Context factors for classification
23
Action sample
a
Conjugate sample
c
f(a): action extractorf
g(a): contextextractorg
f(c): action extractorf
g(c): contextextractorg
SimilaritymeasureLs
Difference measure
Ld
Classification loss
: classifier Lch
Combined objective
Proposed Framework• Joint learning of:• Action extractor f• Context extractor g• Classifier h
24
Action sample
a
Conjugate sample
c
f(a): action extractorf
g(a): contextextractorg
f(c): action extractorf
g(c): contextextractorg
SimilaritymeasureLs
Difference measure
Ld
Classification loss
: classifier Lch
Combined objective
For Prediction (Testing)
Our Framework Enables Detailed Analysis
25
f(a): action extractorf
g(a): contextextractorg
: classifierh
Action sample
a
f(a): action extractorf
g(a): contextextractorg
: classifierh
Action sample
a
• Understand the contribution of the action factors
• Understand the contribution of the context factors
26
Summary of Our Approach
kiss
kiss
kiss
Positive Examples
Sit Up
Drive
Handshake
Negative Examples
+ Context
+ Context
+ Contextkiss + ContextSVM
Automatically
Separately model action & context
Combine them for action recognition
Conjugate samples are easy to collect
Enable fine-grained analysis (action/context channel)
27
Experiment 1
• ActionThread dataset [Hoai & Zisserman, ACCV14]• 3035 video clips• 13 action classes
• C3D Network and features [Tran et al., ICCV15]• 3D convolution network• Input: block of 16 RBG frames• Feature dimension: 4096
28
Experiment 1 Results
Pulling Actions out of Context: Explicit Separation for Effective Combination
Yang Wang and Minh HoaiStony Brook University, Stony Brook, NY 11794, USA
{wang33, minhhoai}@cs.stonybrook.edu
How conjugate samples are used
NotUsed AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0
DriveCar 51.2 51.6 48.9 53.1
Eat 36.9 37.7 35.7 42.1
Fight 45.7 48.6 41.2 61.1
GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2
Hug 43.9 44.1 45.5 54.7
Kiss 67.4 66.4 67.0 72.6
Run 82.0 80.6 79.9 85.6
SitDown 36.3 35.7 36.0 45.2
SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 1. Action recognition results on the ActionThread
dataset. The table shows average precision values; a higher num-ber indicates a better performance. All four settings use the sameC3D architecture [? ]. NotUsed is the baseline method that doesnot use conjugate samples. AsNegative and AsPositive are themethods that use conjugate samples as negative and positive train-ing examples respectively. Our method achieves significantly bet-ter performance than the other methods.
Method No Pruning Pruning
Pretrained C3D [? ] 35.4 -Finetuned C3D [? ] 42.0 -Factor-C3D [Proposed] 47.9 -
DTD [? ] 45.3 52.1Non-Action [? ] 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 2. Benefits and complementary benefits of the proposed
method with other state-of-the-art methods. ‘Pruning’ meansthe non-action classifier is used. The fairest comparison is betweenFactor-C3D and Finetuned C3D because they both use the samefeature descriptors. Factor-C3D provides a large complementarybenefit to DTD.
Significant improvement on average
29
Experiment 1 Results
Pulling Actions out of Context: Explicit Separation for Effective Combination
Yang Wang and Minh HoaiStony Brook University, Stony Brook, NY 11794, USA
{wang33, minhhoai}@cs.stonybrook.edu
How conjugate samples are used
NotUsed AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0
DriveCar 51.2 51.6 48.9 53.1
Eat 36.9 37.7 35.7 42.1
Fight 45.7 48.6 41.2 61.1
GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2
Hug 43.9 44.1 45.5 54.7
Kiss 67.4 66.4 67.0 72.6
Run 82.0 80.6 79.9 85.6
SitDown 36.3 35.7 36.0 45.2
SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 1. Action recognition results on the ActionThread
dataset. The table shows average precision values; a higher num-ber indicates a better performance. All four settings use the sameC3D architecture [? ]. NotUsed is the baseline method that doesnot use conjugate samples. AsNegative and AsPositive are themethods that use conjugate samples as negative and positive train-ing examples respectively. Our method achieves significantly bet-ter performance than the other methods.
Method No Pruning Pruning
Pretrained C3D [? ] 35.4 -Finetuned C3D [? ] 42.0 -Factor-C3D [Proposed] 47.9 -
DTD [? ] 45.3 52.1Non-Action [? ] 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 2. Benefits and complementary benefits of the proposed
method with other state-of-the-art methods. ‘Pruning’ meansthe non-action classifier is used. The fairest comparison is betweenFactor-C3D and Finetuned C3D because they both use the samefeature descriptors. Factor-C3D provides a large complementarybenefit to DTD.
Large performance gap
30
Experiment 1 Results
Pulling Actions out of Context: Explicit Separation for Effective Combination
Yang Wang and Minh HoaiStony Brook University, Stony Brook, NY 11794, USA
{wang33, minhhoai}@cs.stonybrook.edu
How conjugate samples are used
NotUsed AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0
DriveCar 51.2 51.6 48.9 53.1
Eat 36.9 37.7 35.7 42.1
Fight 45.7 48.6 41.2 61.1
GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2
Hug 43.9 44.1 45.5 54.7
Kiss 67.4 66.4 67.0 72.6
Run 82.0 80.6 79.9 85.6
SitDown 36.3 35.7 36.0 45.2
SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 1. Action recognition results on the ActionThread
dataset. The table shows average precision values; a higher num-ber indicates a better performance. All four settings use the sameC3D architecture [? ]. NotUsed is the baseline method that doesnot use conjugate samples. AsNegative and AsPositive are themethods that use conjugate samples as negative and positive train-ing examples respectively. Our method achieves significantly bet-ter performance than the other methods.
Method No Pruning Pruning
Pretrained C3D [? ] 35.4 -Finetuned C3D [? ] 42.0 -Factor-C3D [Proposed] 47.9 -
DTD [? ] 45.3 52.1Non-Action [? ] 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 2. Benefits and complementary benefits of the proposed
method with other state-of-the-art methods. ‘Pruning’ meansthe non-action classifier is used. The fairest comparison is betweenFactor-C3D and Finetuned C3D because they both use the samefeature descriptors. Factor-C3D provides a large complementarybenefit to DTD.
Using conjugate samples as positive or negative examples might hurt the performance
31
Experiment 1: Combining with DTD• Combine with Dense Trajectory Descriptors
How conjugate samples are used
NotUsed [29] AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0
DriveCar 51.2 51.6 48.9 53.1
Eat 36.9 37.7 35.7 42.1
Fight 45.7 48.6 41.2 61.1
GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2
Hug 43.9 44.1 45.5 54.7
Kiss 67.4 66.4 67.0 72.6
Run 82.0 80.6 79.9 85.6
SitDown 36.3 35.7 36.0 45.2
SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 2. Action recognition results on the ActionThread
dataset. The table shows average precision values; a higher num-
ber indicates a better performance. All four settings use the same
C3D architecture [29]. NotUsed is the baseline method that does
not use conjugate samples. AsNegative and AsPositive are the
methods that use conjugate samples as negative and positive train-
ing examples respectively. Our method achieves significantly bet-
ter performance than the other methods.
Method No Pruning Pruning
Pretrained C3D [29] 35.4 -Finetuned C3D [29] 42.0 -Factor-C3D [Proposed] 47.9 -
DTD [32] 45.3 52.1Non-Action [36] 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 3. Benefits and complementary benefits of the proposed
method with other state-of-the-art methods. ‘Pruning’ means
the non-action classifier is used. The fairest comparison is between
Factor-C3D and Finetuned C3D because they both use the same
feature descriptors. Factor-C3D provides a large complementary
benefit to DTD.
For instance, for classifying between two contextually sim-ilar actions Kiss and Hug, removing the confounding con-text features (i.e., using f(a) instead of f(a) + g(a)) leadsto better classification result. However, for two actions withvery different context, e.g., Kiss and Eat, the classifier per-forms worse when the context component is removed, be-cause the context is useful for separating contextually dis-similar actions. These experiments show that the contextshould not be blindly used or removed. Instead, we can de-termine the importance of context with a weighting scheme
Action+Context Action Weighting
70
75
80
85
90
Ave
rage
Pre
cisi
on (%
)
contextually similar actions
Action+Context Action Weighting
95
96
97
98
99
contextually dissimilar actions
Kiss + ; Eat -
Fight + ; Run -
Kiss + ; Hug -
Fight + ; Situp -
Figure 4. Performance change for binary classification after re-
moving context features. +/- stands for positive/negative sam-
ples. Left: for distinguishing between actions with similar context,
removing the confounding context component improves the clas-
sification results. Right: for distinguishing contextually dissimi-
lar actions, removing the context component decreases the perfor-
mance. These results show the importance of context, but context
should not be blindly used or removed. It is beneficial to use con-
text selectively, which requires explicit action-context separation
as proposed here. Given the explicit separation, we can tune the
amount of context to use in a weighted combination, as success-
fully done here.
f(a)+γg(a), where γ is a tunable parameter. This is effec-tive, as shown in Figure 4, where γ is tuned using validationdata. For this experiment, we divide the videos in the testset into disjoint test/validation subsets using 80/20 split.
Visualizing action and context components. We also vi-sualize the video clips that excite the action and context ex-tractors the most. For each video in the test set, We sam-ple 20 clips from the action sequence and another 20 fromthe pre- or post-action sequences. We feed each clip intoour final network and measure the action and context acti-vation strengths separately. For each action class, we iden-tify the video clips that have the highest action and contextactivation strengths. Figure 5 depicts some of these videoclips for actions AnswerPhone, Hug, ShakeHand and SitUp.As can be seen, the most helpful contextual cues for rec-ognizing Hug and ShakeHand appear to be a group of hu-mans, whereas bed scene is contextually associated to ac-tion SitUp. This is the visual evidence that our network hasindeed learned to factorize action and context components.
Pairwise context difference. Since we have the context ex-tractors for every class, we can analyze the pairwise contextsimilarity between action classes. Consider the context ex-tractor of Class R and all action samples of Class X , wecalculate the mean activation of the context extractor andlet c(X,R) denote this quantity. Using c(R,R) as a ref-erence, we consider the context difference between ClassR and Class X as: d(X,R) = c(R,R) − c(X,R). Thecontext difference indicates the amount of context similar-ity between two classes. Figure 6 shows the context differ-
7049
Average precision
32
Video clips that excite the action and context channels the most
Action Components Context ComponentsSi
tUp
Shak
eHan
dHu
gAn
swer
Phon
e
Figure 5. Representative patterns with highest action and con-
text activation values.
HighFiveStandUp
SitUpSitDown
RunKissHug
ShakeHandGetOutCar
FightEat
DriveCarAnswerPhone
GetOutCar Classifier Kiss Classifier Run Classifier
Average Activation Gap from Reference Classes (Context Channel)
Figure 6. Pairwise context difference between GetOutCar (left),
Kiss (middle), and Run (right) with other action classes. Smaller
context difference indicates higher context similarity. In terms of
contextual similarity, DriveCar is close to GetOutCar, Hug is close
to Kiss, while Fight is close to Run.
ence for three action classes: GetOutCar, Kiss, and Run. Ascan be seen, GetOutCar is contextually similar to DriveCar,Kiss is similar to Hug, and Fight is similar to Run.
4.2. Imperfect conjugate samples
So far, we have assumed that we can retrieve perfect con-jugate samples that do not contain the target human actionsin consideration. This assumption is true in general, un-less we are forced to exclusively work with a dataset thathave no corresponding pre- or post-action sequences such asUCF101 [27] and Hollywood2 [21]. We question whether itis necessary to have a black-and-white separation betweenconjugate and action samples with respect to the action con-tent. We want to study the benefits and drawbacks of theproposed framework when the action samples may not con-tain the entire human action and the conjugate samples can-not be guaranteed to exclude all of the action. In this sec-tion, we describe the experiments to study this scenario, us-ing UCF101 [27] and Hollywood2 [21] datasets.
Given a training video in either UCF101 or Hollywood2dataset, we extract a sequence of 16 video frames and use it
Method UCF101 Hollywood2
C3D [29] 82.3 49.8Factor-C3D [Proposed] 84.5 54.7
DTD [32] 85.9 67.5DTD + C3D 90.8 69.5DTD + Factor-C3D [Proposed] 91.3 71.3
EigenTSN [37] 95.3 75.5EigenTSN + C3D 95.8 76.1EigenTSN + Factor-C3D [Proposed] 95.8 76.7
Table 4. Action recognition results on UCF101 and Holly-
wood2. The comparison between Factor-C3D and C3D is the
direct measurement for the advantage of the proposed frame-
work with conjugate samples. State-of-the-art performance can be
achieved when combining the proposed method with others. We
report accuracy for UCF101 and mean AP for Hollywood2.
as the action sample. From the same video, we extract an-other sequence of 16 frames before or after the action sam-ple to create the corresponding conjugate sample. Thesetwo video samples have the same context, and what dis-tinguish them is the difference between the two dynamicsstages of a human action.
Using the generated conjugate samples, the proposedframework achieves a mean average precision of 54.7% onthe Hollywood2 dataset, outperforming the baseline C3Dnetwork (49.8%) where conjugate samples are not used. Onthe three splits of the UCF101 dataset, the proposed frame-work achieves an average accuracy of 84.5%, outperform-ing the baseline C3D network (82.3%) that does not usethe conjugate samples. On both Hollywood2 and UCF101,the performance gains for using conjugate samples are sig-nificant, even though the approach of generating conjugatesamples is not ideal. The performance gains can be at-tributed to the ability of the framework to force the actionextractor to focus on the dynamics of the action, rather thanthe scene context. On the other hand, the performance gainson Hollywood2 and UCF101 are not as high as the perfor-mance gain obtained on the ActionThread dataset, wherewe could extract proper conjugate samples.
The results reported in previous paragraph should not becompared directly to the highest reported numbers in pre-vious publications, which have been obtained by combin-ing multiple features and methods [6, 35]. The proposedmethod provides complementary benefits to other methods,and the state-of-the-art results can be achieved by combin-ing them, as shown in Table 4.
4.3. Separating action and context in still images
The proposed framework for separating action and con-text is not exclusive to video data. In this section, we per-form some controlled experiments on still images. Action
7050
33
Video clips that excite the action and context channels the most
Action Components Context ComponentsSi
tUp
Shak
eHan
dHu
gAn
swer
Phon
e
Figure 5. Representative patterns with highest action and con-
text activation values.
HighFiveStandUp
SitUpSitDown
RunKissHug
ShakeHandGetOutCar
FightEat
DriveCarAnswerPhone
GetOutCar Classifier Kiss Classifier Run Classifier
Average Activation Gap from Reference Classes (Context Channel)
Figure 6. Pairwise context difference between GetOutCar (left),
Kiss (middle), and Run (right) with other action classes. Smaller
context difference indicates higher context similarity. In terms of
contextual similarity, DriveCar is close to GetOutCar, Hug is close
to Kiss, while Fight is close to Run.
ence for three action classes: GetOutCar, Kiss, and Run. Ascan be seen, GetOutCar is contextually similar to DriveCar,Kiss is similar to Hug, and Fight is similar to Run.
4.2. Imperfect conjugate samples
So far, we have assumed that we can retrieve perfect con-jugate samples that do not contain the target human actionsin consideration. This assumption is true in general, un-less we are forced to exclusively work with a dataset thathave no corresponding pre- or post-action sequences such asUCF101 [27] and Hollywood2 [21]. We question whether itis necessary to have a black-and-white separation betweenconjugate and action samples with respect to the action con-tent. We want to study the benefits and drawbacks of theproposed framework when the action samples may not con-tain the entire human action and the conjugate samples can-not be guaranteed to exclude all of the action. In this sec-tion, we describe the experiments to study this scenario, us-ing UCF101 [27] and Hollywood2 [21] datasets.
Given a training video in either UCF101 or Hollywood2dataset, we extract a sequence of 16 video frames and use it
Method UCF101 Hollywood2
C3D [29] 82.3 49.8Factor-C3D [Proposed] 84.5 54.7
DTD [32] 85.9 67.5DTD + C3D 90.8 69.5DTD + Factor-C3D [Proposed] 91.3 71.3
EigenTSN [37] 95.3 75.5EigenTSN + C3D 95.8 76.1EigenTSN + Factor-C3D [Proposed] 95.8 76.7
Table 4. Action recognition results on UCF101 and Holly-
wood2. The comparison between Factor-C3D and C3D is the
direct measurement for the advantage of the proposed frame-
work with conjugate samples. State-of-the-art performance can be
achieved when combining the proposed method with others. We
report accuracy for UCF101 and mean AP for Hollywood2.
as the action sample. From the same video, we extract an-other sequence of 16 frames before or after the action sam-ple to create the corresponding conjugate sample. Thesetwo video samples have the same context, and what dis-tinguish them is the difference between the two dynamicsstages of a human action.
Using the generated conjugate samples, the proposedframework achieves a mean average precision of 54.7% onthe Hollywood2 dataset, outperforming the baseline C3Dnetwork (49.8%) where conjugate samples are not used. Onthe three splits of the UCF101 dataset, the proposed frame-work achieves an average accuracy of 84.5%, outperform-ing the baseline C3D network (82.3%) that does not usethe conjugate samples. On both Hollywood2 and UCF101,the performance gains for using conjugate samples are sig-nificant, even though the approach of generating conjugatesamples is not ideal. The performance gains can be at-tributed to the ability of the framework to force the actionextractor to focus on the dynamics of the action, rather thanthe scene context. On the other hand, the performance gainson Hollywood2 and UCF101 are not as high as the perfor-mance gain obtained on the ActionThread dataset, wherewe could extract proper conjugate samples.
The results reported in previous paragraph should not becompared directly to the highest reported numbers in pre-vious publications, which have been obtained by combin-ing multiple features and methods [6, 35]. The proposedmethod provides complementary benefits to other methods,and the state-of-the-art results can be achieved by combin-ing them, as shown in Table 4.
4.3. Separating action and context in still images
The proposed framework for separating action and con-text is not exclusive to video data. In this section, we per-form some controlled experiments on still images. Action
7050
34
Pairwise Context Differences
Action Components Context Components
SitU
pSh
akeH
and
Hug
Answ
erPh
one
Figure 5. Representative patterns with highest action and con-
text activation values.
HighFiveStandUp
SitUpSitDown
RunKissHug
ShakeHandGetOutCar
FightEat
DriveCarAnswerPhone
GetOutCar Classifier Kiss Classifier Run Classifier
Average Activation Gap from Reference Classes (Context Channel)
Figure 6. Pairwise context difference between GetOutCar (left),
Kiss (middle), and Run (right) with other action classes. Smaller
context difference indicates higher context similarity. In terms of
contextual similarity, DriveCar is close to GetOutCar, Hug is close
to Kiss, while Fight is close to Run.
ence for three action classes: GetOutCar, Kiss, and Run. Ascan be seen, GetOutCar is contextually similar to DriveCar,Kiss is similar to Hug, and Fight is similar to Run.
4.2. Imperfect conjugate samples
So far, we have assumed that we can retrieve perfect con-jugate samples that do not contain the target human actionsin consideration. This assumption is true in general, un-less we are forced to exclusively work with a dataset thathave no corresponding pre- or post-action sequences such asUCF101 [27] and Hollywood2 [21]. We question whether itis necessary to have a black-and-white separation betweenconjugate and action samples with respect to the action con-tent. We want to study the benefits and drawbacks of theproposed framework when the action samples may not con-tain the entire human action and the conjugate samples can-not be guaranteed to exclude all of the action. In this sec-tion, we describe the experiments to study this scenario, us-ing UCF101 [27] and Hollywood2 [21] datasets.
Given a training video in either UCF101 or Hollywood2dataset, we extract a sequence of 16 video frames and use it
Method UCF101 Hollywood2
C3D [29] 82.3 49.8Factor-C3D [Proposed] 84.5 54.7
DTD [32] 85.9 67.5DTD + C3D 90.8 69.5DTD + Factor-C3D [Proposed] 91.3 71.3
EigenTSN [37] 95.3 75.5EigenTSN + C3D 95.8 76.1EigenTSN + Factor-C3D [Proposed] 95.8 76.7
Table 4. Action recognition results on UCF101 and Holly-
wood2. The comparison between Factor-C3D and C3D is the
direct measurement for the advantage of the proposed frame-
work with conjugate samples. State-of-the-art performance can be
achieved when combining the proposed method with others. We
report accuracy for UCF101 and mean AP for Hollywood2.
as the action sample. From the same video, we extract an-other sequence of 16 frames before or after the action sam-ple to create the corresponding conjugate sample. Thesetwo video samples have the same context, and what dis-tinguish them is the difference between the two dynamicsstages of a human action.
Using the generated conjugate samples, the proposedframework achieves a mean average precision of 54.7% onthe Hollywood2 dataset, outperforming the baseline C3Dnetwork (49.8%) where conjugate samples are not used. Onthe three splits of the UCF101 dataset, the proposed frame-work achieves an average accuracy of 84.5%, outperform-ing the baseline C3D network (82.3%) that does not usethe conjugate samples. On both Hollywood2 and UCF101,the performance gains for using conjugate samples are sig-nificant, even though the approach of generating conjugatesamples is not ideal. The performance gains can be at-tributed to the ability of the framework to force the actionextractor to focus on the dynamics of the action, rather thanthe scene context. On the other hand, the performance gainson Hollywood2 and UCF101 are not as high as the perfor-mance gain obtained on the ActionThread dataset, wherewe could extract proper conjugate samples.
The results reported in previous paragraph should not becompared directly to the highest reported numbers in pre-vious publications, which have been obtained by combin-ing multiple features and methods [6, 35]. The proposedmethod provides complementary benefits to other methods,and the state-of-the-art results can be achieved by combin-ing them, as shown in Table 4.
4.3. Separating action and context in still images
The proposed framework for separating action and con-text is not exclusive to video data. In this section, we per-form some controlled experiments on still images. Action
7050
• Smaller difference indicates higher context similarity• DriveCar vs. GetOutCar
35
Pairwise Context Differences
Action Components Context Components
SitU
pSh
akeH
and
Hug
Answ
erPh
one
Figure 5. Representative patterns with highest action and con-
text activation values.
HighFiveStandUp
SitUpSitDown
RunKissHug
ShakeHandGetOutCar
FightEat
DriveCarAnswerPhone
GetOutCar Classifier Kiss Classifier Run Classifier
Average Activation Gap from Reference Classes (Context Channel)
Figure 6. Pairwise context difference between GetOutCar (left),
Kiss (middle), and Run (right) with other action classes. Smaller
context difference indicates higher context similarity. In terms of
contextual similarity, DriveCar is close to GetOutCar, Hug is close
to Kiss, while Fight is close to Run.
ence for three action classes: GetOutCar, Kiss, and Run. Ascan be seen, GetOutCar is contextually similar to DriveCar,Kiss is similar to Hug, and Fight is similar to Run.
4.2. Imperfect conjugate samples
So far, we have assumed that we can retrieve perfect con-jugate samples that do not contain the target human actionsin consideration. This assumption is true in general, un-less we are forced to exclusively work with a dataset thathave no corresponding pre- or post-action sequences such asUCF101 [27] and Hollywood2 [21]. We question whether itis necessary to have a black-and-white separation betweenconjugate and action samples with respect to the action con-tent. We want to study the benefits and drawbacks of theproposed framework when the action samples may not con-tain the entire human action and the conjugate samples can-not be guaranteed to exclude all of the action. In this sec-tion, we describe the experiments to study this scenario, us-ing UCF101 [27] and Hollywood2 [21] datasets.
Given a training video in either UCF101 or Hollywood2dataset, we extract a sequence of 16 video frames and use it
Method UCF101 Hollywood2
C3D [29] 82.3 49.8Factor-C3D [Proposed] 84.5 54.7
DTD [32] 85.9 67.5DTD + C3D 90.8 69.5DTD + Factor-C3D [Proposed] 91.3 71.3
EigenTSN [37] 95.3 75.5EigenTSN + C3D 95.8 76.1EigenTSN + Factor-C3D [Proposed] 95.8 76.7
Table 4. Action recognition results on UCF101 and Holly-
wood2. The comparison between Factor-C3D and C3D is the
direct measurement for the advantage of the proposed frame-
work with conjugate samples. State-of-the-art performance can be
achieved when combining the proposed method with others. We
report accuracy for UCF101 and mean AP for Hollywood2.
as the action sample. From the same video, we extract an-other sequence of 16 frames before or after the action sam-ple to create the corresponding conjugate sample. Thesetwo video samples have the same context, and what dis-tinguish them is the difference between the two dynamicsstages of a human action.
Using the generated conjugate samples, the proposedframework achieves a mean average precision of 54.7% onthe Hollywood2 dataset, outperforming the baseline C3Dnetwork (49.8%) where conjugate samples are not used. Onthe three splits of the UCF101 dataset, the proposed frame-work achieves an average accuracy of 84.5%, outperform-ing the baseline C3D network (82.3%) that does not usethe conjugate samples. On both Hollywood2 and UCF101,the performance gains for using conjugate samples are sig-nificant, even though the approach of generating conjugatesamples is not ideal. The performance gains can be at-tributed to the ability of the framework to force the actionextractor to focus on the dynamics of the action, rather thanthe scene context. On the other hand, the performance gainson Hollywood2 and UCF101 are not as high as the perfor-mance gain obtained on the ActionThread dataset, wherewe could extract proper conjugate samples.
The results reported in previous paragraph should not becompared directly to the highest reported numbers in pre-vious publications, which have been obtained by combin-ing multiple features and methods [6, 35]. The proposedmethod provides complementary benefits to other methods,and the state-of-the-art results can be achieved by combin-ing them, as shown in Table 4.
4.3. Separating action and context in still images
The proposed framework for separating action and con-text is not exclusive to video data. In this section, we per-form some controlled experiments on still images. Action
7050
• Smaller difference indicates higher context similarity• DriveCar vs. GetOutCar• Kiss vs. Hug
36
Role of Context Features• Being able to separate action components f(a) from context components g(a) brings flexibility•We can tune the contribution of the context component for classification: f(a) + �g(a)
<latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="aS+tNoeEISyAdBPhTj/Tet3568w=">AAACBXicbZDNSsNAFIUn/tZYta7dBIvgqk3c6FJw40apYH+gDWUyuWmHTmbCzI1YQl/ArT6NO/EpfBlxmnahrRcGzpwzl3vnizLBDfr+l7OxubW9s1vZc/er7sHhUa3aMSrXDNpMCaV7ETUguIQ2chTQyzTQNBLQjSY387z7BNpwJR9xmkGY0pHkCWcUrdUa1up+wy/LWxfBUtTJsoa170GsWJ6CRCaoMf3AzzAsqEbOBMzcQW4go2xCR9C3UtIUTFiUa868M+vEXqK0PRK90v3dUdDUmGka2ZcpxbFZzebmf1k/x+QqLLjMcgTJFoOSXHiovPmfvZhrYCimVlCmud3VY2OqKUNLxh2UjUWzbeytOVaUN++m9wrBNGNIuORzUqaB8DxzLa9glc666Fw0Ar8RPPikQk7IKTknAbkk1+SWtEibMBKTF/LqvDnvzseC64azBHxM/pTz+QOu+5v7</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="YicS702jzFfiZCIpF7K9rV4krTw=">AAACJ3icbVDLSiNBFK12fGYczejSTWEQHAaSbje6DAyCmxEFo0ISwu3K7aSwHk3V7cHQ5Fdmq/M1sxNd+iNiJWbh60DBuefcQ11OmivpKY4forkv8wuLS8srla+r39bWq983zr0tnMCWsMq6yxQ8KmmwRZIUXuYOQacKL9KrXxP/4g86L605o1GOXQ0DIzMpgILUq25ku/CD/+SdAWgNfBCmXrUW1+Mp+EeSzEiNzXDSqz51+lYUGg0JBd63kzinbgmOpFA4rnQKjzmIKxhgO1ADGn23nN4+5jtB6fPMuvAM8an6OlGC9n6k07CpgYb+vTcRP/PaBWUH3VKavCA04uWjrFCcLJ8UwfvSoSA1CgSEk+FWLobgQFCoq9KZBstGy4epMbQgG79Hx5bQN/qYSSMn9fk64fW4EgpL3tfzkZzv1ZO4npzGtebhrLpltsW22S5L2D5rsiN2wlpMsGv2l92w2+hf9D+6i+5fVueiWWaTvUH0+AxYLaRo</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit>
How conjugate samples are used
NotUsed [29] AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0
DriveCar 51.2 51.6 48.9 53.1Eat 36.9 37.7 35.7 42.1
Fight 45.7 48.6 41.2 61.1GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2Hug 43.9 44.1 45.5 54.7Kiss 67.4 66.4 67.0 72.6Run 82.0 80.6 79.9 85.6
SitDown 36.3 35.7 36.0 45.2SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 2. Action recognition results on the ActionThread
dataset. The table shows average precision values; a higher num-
ber indicates a better performance. All four settings use the same
C3D architecture [29]. NotUsed is the baseline method that does
not use conjugate samples. AsNegative and AsPositive are the
methods that use conjugate samples as negative and positive train-
ing examples respectively. Our method achieves significantly bet-
ter performance than the other methods.
Method No Pruning Pruning
Pretrained C3D [29] 35.4 -Finetuned C3D [29] 42.0 -Factor-C3D [Proposed] 47.9 -
DTD [32] 45.3 52.1Non-Action [36] 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 3. Benefits and complementary benefits of the proposed
method with other state-of-the-art methods. ‘Pruning’ means
the non-action classifier is used. The fairest comparison is between
Factor-C3D and Finetuned C3D because they both use the same
feature descriptors. Factor-C3D provides a large complementary
benefit to DTD.
For instance, for classifying between two contextually sim-ilar actions Kiss and Hug, removing the confounding con-text features (i.e., using f(a) instead of f(a) + g(a)) leadsto better classification result. However, for two actions withvery different context, e.g., Kiss and Eat, the classifier per-forms worse when the context component is removed, be-cause the context is useful for separating contextually dis-similar actions. These experiments show that the contextshould not be blindly used or removed. Instead, we can de-termine the importance of context with a weighting scheme
Action+Context Action Weighting
70
75
80
85
90
Aver
age
Prec
isio
n (%
)
contextually similar actions
Action+Context Action Weighting
95
96
97
98
99
contextually dissimilar actions
Kiss + ; Eat -
Fight + ; Run -
Kiss + ; Hug -
Fight + ; Situp -
Figure 4. Performance change for binary classification after re-
moving context features. +/- stands for positive/negative sam-
ples. Left: for distinguishing between actions with similar context,
removing the confounding context component improves the clas-
sification results. Right: for distinguishing contextually dissimi-
lar actions, removing the context component decreases the perfor-
mance. These results show the importance of context, but context
should not be blindly used or removed. It is beneficial to use con-
text selectively, which requires explicit action-context separation
as proposed here. Given the explicit separation, we can tune the
amount of context to use in a weighted combination, as success-
fully done here.
f(a)+γg(a), where γ is a tunable parameter. This is effec-tive, as shown in Figure 4, where γ is tuned using validationdata. For this experiment, we divide the videos in the testset into disjoint test/validation subsets using 80/20 split.
Visualizing action and context components. We also vi-sualize the video clips that excite the action and context ex-tractors the most. For each video in the test set, We sam-ple 20 clips from the action sequence and another 20 fromthe pre- or post-action sequences. We feed each clip intoour final network and measure the action and context acti-vation strengths separately. For each action class, we iden-tify the video clips that have the highest action and contextactivation strengths. Figure 5 depicts some of these videoclips for actions AnswerPhone, Hug, ShakeHand and SitUp.As can be seen, the most helpful contextual cues for rec-ognizing Hug and ShakeHand appear to be a group of hu-mans, whereas bed scene is contextually associated to ac-tion SitUp. This is the visual evidence that our network hasindeed learned to factorize action and context components.
Pairwise context difference. Since we have the context ex-tractors for every class, we can analyze the pairwise contextsimilarity between action classes. Consider the context ex-tractor of Class R and all action samples of Class X , wecalculate the mean activation of the context extractor andlet c(X,R) denote this quantity. Using c(R,R) as a ref-erence, we consider the context difference between ClassR and Class X as: d(X,R) = c(R,R) − c(X,R). Thecontext difference indicates the amount of context similar-ity between two classes. Figure 6 shows the context differ-
7049
For contextually similar pairs For contextually dissimilar pairs
f(a) + �g(a)<latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="aS+tNoeEISyAdBPhTj/Tet3568w=">AAACBXicbZDNSsNAFIUn/tZYta7dBIvgqk3c6FJw40apYH+gDWUyuWmHTmbCzI1YQl/ArT6NO/EpfBlxmnahrRcGzpwzl3vnizLBDfr+l7OxubW9s1vZc/er7sHhUa3aMSrXDNpMCaV7ETUguIQ2chTQyzTQNBLQjSY387z7BNpwJR9xmkGY0pHkCWcUrdUa1up+wy/LWxfBUtTJsoa170GsWJ6CRCaoMf3AzzAsqEbOBMzcQW4go2xCR9C3UtIUTFiUa868M+vEXqK0PRK90v3dUdDUmGka2ZcpxbFZzebmf1k/x+QqLLjMcgTJFoOSXHiovPmfvZhrYCimVlCmud3VY2OqKUNLxh2UjUWzbeytOVaUN++m9wrBNGNIuORzUqaB8DxzLa9glc666Fw0Ar8RPPikQk7IKTknAbkk1+SWtEibMBKTF/LqvDnvzseC64azBHxM/pTz+QOu+5v7</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="YicS702jzFfiZCIpF7K9rV4krTw=">AAACJ3icbVDLSiNBFK12fGYczejSTWEQHAaSbje6DAyCmxEFo0ISwu3K7aSwHk3V7cHQ5Fdmq/M1sxNd+iNiJWbh60DBuefcQ11OmivpKY4forkv8wuLS8srla+r39bWq983zr0tnMCWsMq6yxQ8KmmwRZIUXuYOQacKL9KrXxP/4g86L605o1GOXQ0DIzMpgILUq25ku/CD/+SdAWgNfBCmXrUW1+Mp+EeSzEiNzXDSqz51+lYUGg0JBd63kzinbgmOpFA4rnQKjzmIKxhgO1ADGn23nN4+5jtB6fPMuvAM8an6OlGC9n6k07CpgYb+vTcRP/PaBWUH3VKavCA04uWjrFCcLJ8UwfvSoSA1CgSEk+FWLobgQFCoq9KZBstGy4epMbQgG79Hx5bQN/qYSSMn9fk64fW4EgpL3tfzkZzv1ZO4npzGtebhrLpltsW22S5L2D5rsiN2wlpMsGv2l92w2+hf9D+6i+5fVueiWWaTvUH0+AxYLaRo</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit>
f(a) + g(a)<latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit>
f(a)<latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="aS+tNoeEISyAdBPhTj/Tet3568w=">AAACBXicbZDNSsNAFIUn/tZYta7dBIvgqk3c6FJw40apYH+gDWUyuWmHTmbCzI1YQl/ArT6NO/EpfBlxmnahrRcGzpwzl3vnizLBDfr+l7OxubW9s1vZc/er7sHhUa3aMSrXDNpMCaV7ETUguIQ2chTQyzTQNBLQjSY387z7BNpwJR9xmkGY0pHkCWcUrdUa1up+wy/LWxfBUtTJsoa170GsWJ6CRCaoMf3AzzAsqEbOBMzcQW4go2xCR9C3UtIUTFiUa868M+vEXqK0PRK90v3dUdDUmGka2ZcpxbFZzebmf1k/x+QqLLjMcgTJFoOSXHiovPmfvZhrYCimVlCmud3VY2OqKUNLxh2UjUWzbeytOVaUN++m9wrBNGNIuORzUqaB8DxzLa9glc666Fw0Ar8RPPikQk7IKTknAbkk1+SWtEibMBKTF/LqvDnvzseC64azBHxM/pTz+QOu+5v7</latexit><latexit sha1_base64="D9L+MZ1FkWLZNrdGDQapija7IB4=">AAACDXicbZDPSgMxEMZn/W/9V716CYpQL+2uFz0KXrwoCq4V2lKy6awNZpMlmRXL0mfwqj6NN/ERfBkxrT1odSDw5fsyzOSX5Eo6CsOPYGZ2bn5hcWm5srK6tr5R3Vy9dqawAmNhlLE3CXeopMaYJCm8yS3yLFHYTO5ORnnzHq2TRl/RIMdOxm+1TKXg5K04rfF91q3uhvVwXOyviCZiFyZ10a1+tntGFBlqEoo714rCnDoltySFwmGlXTjMubjjt9jyUvMMXaccLztke97psdRYfzSxsfuzo+SZc4Ms8S8zTn03nY3M/7JWQelRp5Q6Lwi1+B6UFoqRYaOfs560KEgNvODCSr8rE31uuSDPp9IeN5aN2Plbo2+4bJwNzg2ha/QwlVqOeLk64cOw4oFF03j+iuuDehTWo8sQlmAbdqAGERzCMZzCBcQgQMIjPMFz8BK8Bm/faGeCCeMt+FXB+xdf5J5Q</latexit><latexit sha1_base64="D9L+MZ1FkWLZNrdGDQapija7IB4=">AAACDXicbZDPSgMxEMZn/W/9V716CYpQL+2uFz0KXrwoCq4V2lKy6awNZpMlmRXL0mfwqj6NN/ERfBkxrT1odSDw5fsyzOSX5Eo6CsOPYGZ2bn5hcWm5srK6tr5R3Vy9dqawAmNhlLE3CXeopMaYJCm8yS3yLFHYTO5ORnnzHq2TRl/RIMdOxm+1TKXg5K04rfF91q3uhvVwXOyviCZiFyZ10a1+tntGFBlqEoo714rCnDoltySFwmGlXTjMubjjt9jyUvMMXaccLztke97psdRYfzSxsfuzo+SZc4Ms8S8zTn03nY3M/7JWQelRp5Q6Lwi1+B6UFoqRYaOfs560KEgNvODCSr8rE31uuSDPp9IeN5aN2Plbo2+4bJwNzg2ha/QwlVqOeLk64cOw4oFF03j+iuuDehTWo8sQlmAbdqAGERzCMZzCBcQgQMIjPMFz8BK8Bm/faGeCCeMt+FXB+xdf5J5Q</latexit><latexit sha1_base64="HLmiya1o3Z365eF1CwAT1D9CE/k=">AAACGHicbVDLSgMxFM3Ud31VXboJFqFu2hk3uhREcKNUcLTQlpJJ79jQTDIkd8Qy9Bvcql/jTty682fE9LHQ6oHAyTn3kJsTpVJY9P1PrzA3v7C4tLxSXF1b39gsbW3fWJ0ZDiHXUptGxCxIoSBEgRIaqQGWRBJuo/7pyL+9B2OFVtc4SKGdsDslYsEZOimMK+yAdkplv+qPQf+SYErKZIp6p/TV6mqeJaCQS2ZtM/BTbOfMoOAShsVWZiFlvM/uoOmoYgnYdj5edkj3ndKlsTbuKKRj9WciZ4m1gyRykwnDnp31RuJ/XjPD+LidC5VmCIpPHoozSVHT0c9pVxjgKAeOMG6E25XyHjOMo+un2BoH81po3a3W00zULgaXGsHWuhALJUZ92SrCw7DoCgtm6/lLbg6rgV8Nrvzyydm0umWyS/ZIhQTkiJyQc1InIeFEkEfyRJ69F+/Ve/PeJ6MFb5rZIb/gfXwDQQ2f0g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit>
f(a) + �g(a)<latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="aS+tNoeEISyAdBPhTj/Tet3568w=">AAACBXicbZDNSsNAFIUn/tZYta7dBIvgqk3c6FJw40apYH+gDWUyuWmHTmbCzI1YQl/ArT6NO/EpfBlxmnahrRcGzpwzl3vnizLBDfr+l7OxubW9s1vZc/er7sHhUa3aMSrXDNpMCaV7ETUguIQ2chTQyzTQNBLQjSY387z7BNpwJR9xmkGY0pHkCWcUrdUa1up+wy/LWxfBUtTJsoa170GsWJ6CRCaoMf3AzzAsqEbOBMzcQW4go2xCR9C3UtIUTFiUa868M+vEXqK0PRK90v3dUdDUmGka2ZcpxbFZzebmf1k/x+QqLLjMcgTJFoOSXHiovPmfvZhrYCimVlCmud3VY2OqKUNLxh2UjUWzbeytOVaUN++m9wrBNGNIuORzUqaB8DxzLa9glc666Fw0Ar8RPPikQk7IKTknAbkk1+SWtEibMBKTF/LqvDnvzseC64azBHxM/pTz+QOu+5v7</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="rPMRX3pSYIfkMYVALrtEdtnDbdY=">AAACHHicbZBNSyNBEIZr3A/drLtmvXppVhZcFpIZL3oUvHhxUTAqJCHUdGqSxv4YumskYchf8br6a7yJ+2cWOzGHXd2Chrfet4sqnrzUKnCa/k5W3rx993517UPj4/qnzxvNL+vnwVVeUkc67fxljoG0stRhxZouS09ock0X+dXhPL+4Jh+Us2c8LalvcGRVoSRytAbNzWIHv4sfojdCY1CMYjdobqetdFHitciWYhuWdTJo/ukNnawMWZYaQ+hmacn9Gj0rqWnW6FWBSpRXOKJulBYNhX69uH0mvkVnKArn47MsFu7fEzWaEKYmjz8N8ji8zObm/7JuxcV+v1a2rJisfF5UVFqwE3MQYqg8SdbTKFB6FW8VcoweJUdcjd5isG53QuzaY4eqfTz96ZhCe0iFsmqOL7SYJrNGBJa9xPNanO+2srSVnaawBlvwFXYggz04gCM4gQ5ImMAN/ILb5C65Tx6e0a4kS8ab8E8lj09HRKLX</latexit><latexit sha1_base64="YicS702jzFfiZCIpF7K9rV4krTw=">AAACJ3icbVDLSiNBFK12fGYczejSTWEQHAaSbje6DAyCmxEFo0ISwu3K7aSwHk3V7cHQ5Fdmq/M1sxNd+iNiJWbh60DBuefcQ11OmivpKY4forkv8wuLS8srla+r39bWq983zr0tnMCWsMq6yxQ8KmmwRZIUXuYOQacKL9KrXxP/4g86L605o1GOXQ0DIzMpgILUq25ku/CD/+SdAWgNfBCmXrUW1+Mp+EeSzEiNzXDSqz51+lYUGg0JBd63kzinbgmOpFA4rnQKjzmIKxhgO1ADGn23nN4+5jtB6fPMuvAM8an6OlGC9n6k07CpgYb+vTcRP/PaBWUH3VKavCA04uWjrFCcLJ8UwfvSoSA1CgSEk+FWLobgQFCoq9KZBstGy4epMbQgG79Hx5bQN/qYSSMn9fk64fW4EgpL3tfzkZzv1ZO4npzGtebhrLpltsW22S5L2D5rsiN2wlpMsGv2l92w2+hf9D+6i+5fVueiWWaTvUH0+AxYLaRo</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit><latexit sha1_base64="p/bjSifymKnHQSvtxwkU60gq72w=">AAACJ3icbVBdSxtBFJ219St+pfroy9BQUIRkVwT7KIjgi8VCo4EkhLuTu8ngfCwzd8Ww5K/4av01fSv62D8iTmIe2tgDA+eecw9zOWmupKc4fo4WPnxcXFpeWa2srW9sblU/bV95WziBTWGVda0UPCppsEmSFLZyh6BThdfpzenEv75F56U1P2iUY1fDwMhMCqAg9arb2R7s8wPeGYDWwAdh6lVrcT2egr8nyYzU2AyXvepLp29FodGQUOB9O4lz6pbgSAqF40qn8JiDuIEBtgM1oNF3y+ntY/4lKH2eWReeIT5V/06UoL0f6TRsaqChn/cm4v+8dkHZ124pTV4QGvH2UVYoTpZPiuB96VCQGgUCwslwKxdDcCAo1FXpTINlo+nD1BhakI2L0TdL6Bt9zKSRk/p8nfBuXAmFJfP1vCdXh/Ukriffj2onZ7PqVtgu+8z2WMKO2Qk7Z5esyQS7Y/fsgf2MHqNf0e/o6W11IZpldtg/iP68AlltpGw=</latexit>
f(a) + g(a)<latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit><latexit sha1_base64="0vJp1n3a2eoyODYJ5DLirc9LOvk=">AAACHnicbVDLSgMxFM34rPVVdekmWARFaGdE0GVBBDdKBfuAtkgmvdMGM8mQ3BHL0M9wq36NO3GrPyOmj4WvAyEn597DvTlhIoVF3//wZmbn5hcWc0v55ZXVtfXCxmbd6tRwqHEttWmGzIIUCmooUEIzMcDiUEIjvD0d1Rt3YKzQ6hoHCXRi1lMiEpyhk1rRHtunB7TnrptC0S/5Y9C/JJiSIpmielP4bHc1T2NQyCWzthX4CXYyZlBwCcN8O7WQMH7LetByVLEYbCcbrzyku07p0kgbdxTSsfrdkbHY2kEcus6YYd/+ro3E/2qtFKOTTiZUkiIoPhkUpZKipqP/064wwFEOHGHcCLcr5X1mGEeXUr49NmblmnWvcl8zUb4YXGoEW+5CJJQYpWZLCPfDvAss+B3PX1I/LAV+Kbg6KlbOptHlyDbZIXskIMekQs5JldQIJ5o8kEfy5D17L96r9zZpnfGmni3yA977F4R1oXY=</latexit>
f(a)<latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="aS+tNoeEISyAdBPhTj/Tet3568w=">AAACBXicbZDNSsNAFIUn/tZYta7dBIvgqk3c6FJw40apYH+gDWUyuWmHTmbCzI1YQl/ArT6NO/EpfBlxmnahrRcGzpwzl3vnizLBDfr+l7OxubW9s1vZc/er7sHhUa3aMSrXDNpMCaV7ETUguIQ2chTQyzTQNBLQjSY387z7BNpwJR9xmkGY0pHkCWcUrdUa1up+wy/LWxfBUtTJsoa170GsWJ6CRCaoMf3AzzAsqEbOBMzcQW4go2xCR9C3UtIUTFiUa868M+vEXqK0PRK90v3dUdDUmGka2ZcpxbFZzebmf1k/x+QqLLjMcgTJFoOSXHiovPmfvZhrYCimVlCmud3VY2OqKUNLxh2UjUWzbeytOVaUN++m9wrBNGNIuORzUqaB8DxzLa9glc666Fw0Ar8RPPikQk7IKTknAbkk1+SWtEibMBKTF/LqvDnvzseC64azBHxM/pTz+QOu+5v7</latexit><latexit sha1_base64="D9L+MZ1FkWLZNrdGDQapija7IB4=">AAACDXicbZDPSgMxEMZn/W/9V716CYpQL+2uFz0KXrwoCq4V2lKy6awNZpMlmRXL0mfwqj6NN/ERfBkxrT1odSDw5fsyzOSX5Eo6CsOPYGZ2bn5hcWm5srK6tr5R3Vy9dqawAmNhlLE3CXeopMaYJCm8yS3yLFHYTO5ORnnzHq2TRl/RIMdOxm+1TKXg5K04rfF91q3uhvVwXOyviCZiFyZ10a1+tntGFBlqEoo714rCnDoltySFwmGlXTjMubjjt9jyUvMMXaccLztke97psdRYfzSxsfuzo+SZc4Ms8S8zTn03nY3M/7JWQelRp5Q6Lwi1+B6UFoqRYaOfs560KEgNvODCSr8rE31uuSDPp9IeN5aN2Plbo2+4bJwNzg2ha/QwlVqOeLk64cOw4oFF03j+iuuDehTWo8sQlmAbdqAGERzCMZzCBcQgQMIjPMFz8BK8Bm/faGeCCeMt+FXB+xdf5J5Q</latexit><latexit sha1_base64="D9L+MZ1FkWLZNrdGDQapija7IB4=">AAACDXicbZDPSgMxEMZn/W/9V716CYpQL+2uFz0KXrwoCq4V2lKy6awNZpMlmRXL0mfwqj6NN/ERfBkxrT1odSDw5fsyzOSX5Eo6CsOPYGZ2bn5hcWm5srK6tr5R3Vy9dqawAmNhlLE3CXeopMaYJCm8yS3yLFHYTO5ORnnzHq2TRl/RIMdOxm+1TKXg5K04rfF91q3uhvVwXOyviCZiFyZ10a1+tntGFBlqEoo714rCnDoltySFwmGlXTjMubjjt9jyUvMMXaccLztke97psdRYfzSxsfuzo+SZc4Ms8S8zTn03nY3M/7JWQelRp5Q6Lwi1+B6UFoqRYaOfs560KEgNvODCSr8rE31uuSDPp9IeN5aN2Plbo2+4bJwNzg2ha/QwlVqOeLk64cOw4oFF03j+iuuDehTWo8sQlmAbdqAGERzCMZzCBcQgQMIjPMFz8BK8Bm/faGeCCeMt+FXB+xdf5J5Q</latexit><latexit sha1_base64="HLmiya1o3Z365eF1CwAT1D9CE/k=">AAACGHicbVDLSgMxFM3Ud31VXboJFqFu2hk3uhREcKNUcLTQlpJJ79jQTDIkd8Qy9Bvcql/jTty682fE9LHQ6oHAyTn3kJsTpVJY9P1PrzA3v7C4tLxSXF1b39gsbW3fWJ0ZDiHXUptGxCxIoSBEgRIaqQGWRBJuo/7pyL+9B2OFVtc4SKGdsDslYsEZOimMK+yAdkplv+qPQf+SYErKZIp6p/TV6mqeJaCQS2ZtM/BTbOfMoOAShsVWZiFlvM/uoOmoYgnYdj5edkj3ndKlsTbuKKRj9WciZ4m1gyRykwnDnp31RuJ/XjPD+LidC5VmCIpPHoozSVHT0c9pVxjgKAeOMG6E25XyHjOMo+un2BoH81po3a3W00zULgaXGsHWuhALJUZ92SrCw7DoCgtm6/lLbg6rgV8Nrvzyydm0umWyS/ZIhQTkiJyQc1InIeFEkEfyRJ69F+/Ve/PeJ6MFb5rZIb/gfXwDQQ2f0g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit><latexit sha1_base64="QAhWZf3vpk2g65ae/b8U0NM0oUw=">AAACGHicbVDLSgMxFM34tr6qLt0Ei6CbdkYEXRZEcKMoOK3QFsmkd9rQTDIkd8Qy9Bvcql/jTty682fE9LFQ64HAyTn3kJsTpVJY9P1Pb2Z2bn5hcWm5sLK6tr5R3NyqWZ0ZDiHXUpvbiFmQQkGIAiXcpgZYEkmoR73ToV+/B2OFVjfYT6GVsI4SseAMnRTG++yA3hVLftkfgU6TYEJKZIKru+JXs615loBCLpm1jcBPsZUzg4JLGBSamYWU8R7rQMNRxRKwrXy07IDuOaVNY23cUUhH6s9EzhJr+0nkJhOGXfvXG4r/eY0M45NWLlSaISg+fijOJEVNhz+nbWGAo+w7wrgRblfKu8wwjq6fQnMUzCuhdbdKVzNRuehfagRbaUMslBj2ZcsID4OCKyz4W880qR2WA78cXB+VqmeT6pbIDtkl+yQgx6RKzskVCQkngjySJ/LsvXiv3pv3Ph6d8SaZbfIL3sc3Qk2f1g==</latexit>
37
Experiments on Hollywood2 & UCF101 Datasets• Hollywood2 dataset• 12 action classes• Around 1600 clips from various movies
• UCF101 dataset• 101 action classes• Around 13K clips collected from YouTube
• But the video clips are trimmed to the action parts only• No preceding/following context sequences
Action sample
16 frames
Conjugate sample
16 frames
•We consider imperfect conjugate samples
38
Using Imperfect Conjugate Samples
How conjugate samples are used
NotUsed AsNegative AsPositive Proposed
AnswerPhone 27.3 27.3 28.7 43.0DriveCar 51.2 51.6 48.9 53.1Eat 36.9 37.7 35.7 42.1Fight 45.7 48.6 41.2 61.1GetOutCar 29.7 31.4 30.1 27.2ShakeHand 26.9 26.0 26.6 35.2Hug 43.9 44.1 45.5 54.7Kiss 67.4 66.4 67.0 72.6Run 82.0 80.6 79.9 85.6SitDown 36.3 35.7 36.0 45.2SitUp 17.1 13.4 15.2 15.7StandUp 31.9 31.0 31.4 28.5HighFive 49.3 40.6 48.8 58.5
Mean 42.0 41.1 41.2 47.9
Table 1. Action recognition results on the ActionThreaddataset. All settings use the same C3D architecture. NotUsed isthe baseline method that does not use conjugate samples. AsNega-tive and AsPositive are the methods that use conjugate samples asnegative and positive training examples respectively. Our methodachieves significantly better performance than the other methods.
Method No Pruning Pruning
Pretrained C3D 35.4 -Finetuned C3D 42.0 -Factor-C3D [Proposed] 47.9 -
DTD 45.3 52.1Non-Action 48.0 55.0DTD + C3D 52.0 56.3DTD + Factor-C3D [Proposed] 55.3 58.5
Table 2. Benefits and complementary benefits of the proposedmethod with other state-of-the-art methods. ‘Pruning’ meansthe non-action classifier is used. The fairest comparison is betweenFactor-C3D and Finetuned C3D because they both use the samefeature descriptors. Factor-C3D provides a large complementarybenefit to DTD.
Method UCF101 Hollywood2
C3D 82.3 49.8Factor-C3D [Proposed] 84.5 54.7
DTD 85.9 67.5DTD + C3D 90.8 69.5DTD + Factor-C3D [Proposed] 91.3 71.3
EigenTSN 95.3 75.5EigenTSN + C3D 95.8 76.1EigenTSN + Factor-C3D [Proposed] 95.8 76.7
Table 3. Action recognition results on UCF101 and Holly-wood2. The comparison between Factor-C3D and C3D is thedirect measurement for the advantage of the proposed frame-work with conjugate samples. State-of-the-art performance can beachieved when combining the proposed method with others. Wereport accuracy for UCF101 and mean AP for Hollywood2.
77
Summary
• First part• Task: recognition• A method to separate action from context
• Second part• Task: early recognition• Consider multiple cameras and learn a selection policy
• Learn to attend to the relevant information
Thank You!