arX

iv:1

901.

0533

5v2

[cs

.LG

] 2

4 Ju

l 201

91

A review of domain adaptationwithout target labels

Wouter M. Kouw, Marco Loog

Abstract—Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the

question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of

approaches, divided into, what we refer to as, sample-based, feature-based and inference-based methods. Sample-based methods

focus on weighting individual observations during training based on their importance to the target domain. Feature-based methods

revolve around on mapping, projecting and representing features such that a source classifier performs well on the target domain and

inference-based methods incorporate adaptation into the parameter estimation procedure, for instance through constraints on the

optimization procedure. Additionally, we review a number of conditions that allow for formulating bounds on the cross-domain

generalization error. Our categorization highlights recurring ideas and raises questions important to further research.

Index Terms—Machine Learning, Pattern Recognition, Domain Adaptation, Transfer Learning, Covariate Shift, Sample Selection Bias.

✦

1 INTRODUCTION

G ENERALIZATION is the process of observing a finitenumber of samples and making statements about all

possible samples. In the case of machine learning andpattern recognition, samples are used to train classifiersto make predictions for future samples. However, if theobserved labeled samples are not an accurate reflection ofthe underlying distribution on which the learner should op-erate, the system will not generalize well to new samples. Inpractice, collected data is hardly ever a completely unbiasedrepresentation of the operating setting.

Data is biased if certain outcomes are systematically morefrequently observed than they would be for a uniformly-at-random sampling procedure. For example, data sampledfrom a single hospital can be biased with respect to theglobal population due to differences in living conditions ofthe local patient population. Statisticians have long studiedsampling biases under the term sample selection bias [1], [2].Corrections are based on estimating - or in cases wherethere is control of the experimental design, knowing - theprobability that an instance will be selected for observation[3], [4], [5]. But many modern data collection procedures,such as internet crawlers, are less structured than samplingfrom patients in a hospital, for instance. It is thereforedifficult, if not impossible, to estimate the probability thata sample is selected for observation and by extension, howthe biased sample differs from the general population. Onthe other hand, it might not be necessary to generalizeto the whole population. It could be more important togeneralize to a specific target subpopulation. For example,can data collected in European hospitals be used to train anintelligent prognosis system for hospitals in Africa?

• W.M. Kouw is with the Datalogisk Institut, University of Copenhagen, atUniversitetsparken 1, DK-2100 Copenhagen, Denmark.E-mail: wmkouw@gmail.com

• M. Loog is with the Pattern Recognition Laboratory, Delft Universityof Technology, in Delft, the Netherlands and the Datalogisk Institut,University of Copenhagen, in Copenhagen, Denmark.

In order to target specific distributions over samplespace, henceforth referred to as domains, we need at leastsome information. Unlabeled data from a target domain canusually be collected, but labels are more difficult to obtain.Nevertheless, unlabeled data gives an indication in whatway a source domain and a target domain differ from eachother. This information can be exploited to make a classifieradapt, i.e. change its decisions such that it generalizes bettertowards the target domain.

The important question to ask is: how can a classifierlearn from a source domain and generalize to a targetdomain? We present a categorization of methods into threeparts, each containing a subcategorization on a finer level.First, there are sample-based methods, which are based oncorrecting for biases in the data sampling procedure throughindividual samples. Methods in this category focus on dataimportance-weighting [6], [7] or class importance-weighting[8]. Secondly, there are feature-based methods, which focuson reshaping feature space such that a classifier trainedon transformed source data will generalize to target data.A further distinction can be made into finding subspacemappings [9], [10], optimal transportation techniques [11],learning domain-invariant representations [12] or construct-ing corresponding features [13]. Thirdly, we consider whatwe call inference-based approaches. These methods focus onincorporating the adaptation into the parameter estimationprocedure. It is a diverse category, containing algorithmicrobustness [14], minimax estimators [15], self-learning [16],empirical Bayes [17] and PAC-Bayes [18]. Clearly, the aboveclassification is not necessarily mutually exclusive, but webelieve it offers a comprehensible overview.

Our categorization reveals a small number of conditionsthat permit performance guarantees for domain adaptiveclassifiers. In practice, one has to assume that a condi-tion holds, which means that for any adaptive classifiera problem setting exists for which it fails. We discuss theimportance of hypothesis tests and causal information todomain-adaptive classifier selection.

2

1.1 Scope

Our scope is limited to the single-source / single-targetadaptation without labeled data from the target domain.This setting is the minimal form to study cross-domain gen-eralization. Incorporating multiple source domains raisesadditional questions such as: Should each source domainbe weighted based on its similarity to the target domain?Or should domains be selected? Should there be sometemporal or spatial ordering? Questions concerning multi-source adaptation are not considered here, but the interestedreader may refer to [19], [20], [21]. Similarly, incorporatingtarget labels would raise additional questions relating tosemi-supervised learning, active learning and multi-tasklearning [22], [23]. For example, how can the unlabeledtarget data improve the classifier’s estimation? Are somelabeled target samples more informative than others? Arethere features that are useful to classification in both do-mains? These questions will remain outside our scope aswell. Topics that are well covered by other reviews will notbe discussed in great detail here either. First, there are twoarticles covering visual domain adaptation [24], [25], with athird one specializing in deep learning [26]. Secondly, thereis an empirical comparison of domain adaptation methodsfor genomic sequence analysis [27] and thirdly, a surveypaper on, amongst others, transfer learning in biomedicalimaging [28].

Other reviews are available: there is a book on data setshift in machine learning [29], an excellent paper on thetypes and causes of data set shift [30], a technical reporton domain adaptation with unlabeled samples [31] and twopapers focusing on variants of transfer learning [32], [33].Our work complements these with more recent studies.

1.2 Outline

Before the end of this section, we go through some moti-vating examples. These demonstrate that this problem isrelevant to a wide variety of scientific and engineeringfields. In section 2 we turn to more precise definitions ofdomains and adaptation. Furthermore, we discuss briefly anumber of assumptions that permit performance guaranteeson the target domain. Additionally, an example setting ispresented that will serve to provide some intuition of howparticular types of methods work. Our categorization startswith sample-based methods in Section 3. Following thatare feature-based methods in Section 4 and inference-basedmethods in Section 5. The discussion of methods includesequations whenever they facilitate comparisons, such as be-tween divergences or estimators. Lastly, we discuss commonthemes and open questions in Section 6, and summarize ourfindings in Section 7.

1.3 Motivating examples

Sample selection bias has been studied in statistics andeconometrics for quite some time [2], [34]. For example, inthe 70s, it was of interest to find predictors for wage rates ofwomen. These were estimated by measuring characteristicsof working women and their salaries [35], [36]. However,working women differed from non-working women interms such as age, number of children, and education [35].

The predictor did not generalize to the total population,which raised awareness of sample selection bias as an issue.

In clinical studies, randomized controlled trials are usedto study the effects of treatments [37]. The average treatmenteffect is estimated by comparing the difference betweenan experimental and a control group [5]. The treatmenteffect is expected to hold for patients outside of the study,but that is not necessarily the case if certain patients weresystematically excluded from the study. For example, thefactor that makes certain patients non-compliant – a reasonfor exclusion – can also have an effect on the treatment [37].

In medical imaging, radiologists manually annotate tis-sues, abnormalities, and pathologies to obtain training datafor computer-aided-diagnosis systems. But due to the me-chanical configuration, calibration, vendor or acquisitionprotocol of MRI, CT or PET scanners, there are large varia-tions between data sets from different medical centers [38],[39]. Consequently, diagnosis systems often fail to performwell across centers [40], [41], [42].

Computer vision deals with such rich, high-dimensionaldata that even large image data sets are essentially biasedsamplings of visual objects [43]. As a result, cross-datasetgeneralization is low for systems that do not employ someform of adaptation [24], [25], [43]. Examples of adaptationstudies include recognizing objects in photos based on com-mercial images [44], event recognition in consumer videosthrough training on web data [45], recognizing activitiesacross viewpoints [46], and recognizing movements acrosssensors [47] and persons [48], [49].

In robotics, simulations can be employed as an addi-tional source of data [50], [51]. Physics simulators have beenextensively developed for fields such as computer graphicsor video gaming and one could potentially generate a vastamount of data. Through adaptation from the simulateddata to the real data, domain adaptive methods can behelpful in improving lane and pedestrian detection forself-driving cars [52], [53] or improving hand grasping forstationary robots [54].

Speech differs strongly across speakers, but is instantlyrecognizable for humans. Learning algorithms expect newvocal data to be similar to training data, and struggle togeneralize across speakers [55]. With the adoption of com-mercial speech recognition systems in homes, there is anincreasing need to adapt to specific speakers [56].

In natural language processing, authors are known touse different styles of expression on different publicationplatforms. For instance, biomedical science articles containwords like ’oncogenic’ and ’mutated’, which appear far lessoften in financial news articles [13]. Similarly, online moviereviews are linguistically different from tweets [57] andproduct reviews differ per category [58]. Natural languageprocessing tasks such as sentiment classification or named-entity recognition become much more challenging underchanges in word frequencies.

In bioinformatics, adaptive approaches have been suc-cessful in sequence classification [59], [60] and biologicalnetwork reconstruction [61]. For some problem settings, thegoal is to generalize from one model organism to another[62], such as from nematodes to fruit flies [27].

Radiotelescopes measure spectral signals arriving toearth. Astronomers use these signals to, for example, detect

3

quasars or to determine the amount of photometric redshiftin galaxies [63], [64]. These signals are costly to label, and as-tronomers therefore choose the ones they consider the mostpromising. But this selection constitutes a biased samplingprocedure and various domain adaptation techniques havebeen employed to tackle it [65], [66].

Fairness-aware machine learning focuses on ensuringthat algorithms and automated decision-making systems donot discriminate based on gender, race or other protectedattributes [67], [68]. For instance, if a data set contains manyexamples of men with higher salaries, then a recommendersystem could learn to suggest mostly men as candidates forpositions with high salaries. To tackle this kind of unwantedbehavior in algorithms, fairness has to be built into thelearning process [69]. Aspects of fairness, such as equalityof opportunity, can be formulated as constraints on thelearning process [67]. Some of these constraints lead to aneed for distribution matching techniques, which have beenextensively developed for domain adaptation [70].

2 DOMAIN ADAPTATION

We go through some definitions relevant to domain adapta-tion, followed by the introduction of a running example, aremark on domain discrepancy metrics and a brief reviewof assumptions that permit generalization error bounds foradaptive classifiers. The reader is assumed to be familiarwith supervised learning and risk minimization. For exten-sive overviews of these topics, see [71], [72].

2.1 Definitions & notation

Consider an input or feature space X , a subset of RD , and

an output or label space Y , either binary {−1,+1} or multi-class {1, . . .K} with K as the number of classes. We definedifferent domains, in this context, as different probabilitydistributions p(x, y) over the same feature-label space pairX × Y . The domain of interest is called the target domainpT (x, y) and the domain with labeled data is called thesource domain pS(x, y). Domain adaptation refers to predict-ing the labels of samples drawn from a target domain, givenlabeled samples drawn from a source domain and unlabeledsamples drawn from the target domain itself. We considerdomain adaptation a special case of transfer learning, wheredifferences between feature spaces and label spaces are al-lowed. For example, transferring from the ”speech domain”to the ”text domain”.

We assume that a data set of size n from the sourcedomain is given. Source samples are marked xi and sourcelabels are marked yi, for i = 1, . . . , n. Likewise, a dataset of size m drawn from the target domain, where onlythe target samples zj for j = 1, . . . ,m are given and thelabels uj are not known. A problem setting with at least oneobserved target label is usually called semi-supervised domainadaptation. There are two potential goals: firstly, to predictthe labels of the given target samples. The second goal isto predict the labels of new samples from the target domain.The first one is called the transductive setting, and the secondgoal is called the inductive setting.

Functions and distributions related to the source domainare marked with the subscript S, e.g. the source posterior

distribution pS(y | x). Similarly, functions related to the tar-get domain are marked with T . Classification functions mapsamples to a real number, h : X → R, where h is an elementof an hypothesis space H. A loss function ℓ compares aclassifier prediction with the true label ℓ : R × Y → R. Arisk function R is the expected loss of a particular classifier,with respect to a distributionR(h) = E[ℓ(h(x), y)]. The errorfunction e is a special case of a risk function, correspondingto the expected 0/1-loss. D is reserved for discrepancymeasures, W for weights, M for transformation matrices,φ for basis functions and κ for kernel functions.

2.2 Example setting

We use a running example to illustrate the problem setting,and later on, to illustrate the behavior of some domain-adaptive classifiers. Figure 1 shows a scatter plot of mea-surements of patients from a hospital in Budapest, Hungary(left; the source domain) and patients from a hospital inLong Beach, California (right; the target domain). This asubset of the Heart Disease data set from UCI machinelearning repository [73]. The measurements consist of thepatients’ age (x-axis) and cholesterol level (y-axis) and thetask is to predict whether they will develop heart disease(blue = healthy, red = heart disease).

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Budapest, Hungaryhealthydisease

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Long Beach, CA, USAhealthydisease

Fig. 1: Example of a domain adaptation problem, in whichpatients are diagnosed with heart disease based on theirage and cholesterol. (Left) Data from the source domain, ahospital in Budapest. (Right) Data from the target domain,a hospital in Long Beach, California.

Figure 2 (left) shows the decision boundary of a linearclassifier trained on the source samples (solid black line).It is not suited well to classifying samples from the targetdomain (Figure 2 right), which are shifted in terms of age.As can be imagined, its performance would decrease as thedifference between the domains increases.

2.3 Domain dissimilarity metrics

In order to characterize generalization across domains, ameasure of domain dissimilarity is necessary. Many met-rics of of differences between probability distributions ordata sets exist, such as the Kullback-Leibler divergence,the total variation distance, the Wasserstein metric or theKolmogorov-Smirnoff statistic [74], [75]. The choice of met-ric will often affect the behavior of a domain-adaptiveclassifier. We will discuss two measures in more detail, asthey will appear later on in the paper.

4

20 30 40 50 60 70 80age

100

200

300

400

500

600ch

oles

terol

Budapest, Hungaryhealthydisease

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Long Beach, CA, USApredict healthypredict disease

Fig. 2: Linear classifier (solid black line) trained on sourcedata (left) and applied to target data (right).

The symmetric difference hypothesis divergence (DH∆H)takes two classifiers and looks at to what extent they dis-agree with each other on both domains [76]:

DH∆H[pS , pT ] =

2 suph,h′∈H

| PrS [h(x) 6= h′(x)] − PrT [h(x) 6= h′(x)] | . (1)

In this context, it finds the pair of classifiers h, h′ for whichthe difference in disagreements between the source andtarget domain is largest [76], [77]. Its value increases as thedomains become more dissimilar. Another example is theRenyi divergence [78]:

DRα [pT , pS ] =1

α− 1log2

∫

XpαT (x)/p

α−1S (x)dx , (2)

where α denotes its order. For α = 1, the Renyi divergenceequals the Kullback-Leibler divergence [78]. In words, itcorresponds to the expected value, with respect to the sourcedistribution, of a power of the ratio of data distributions ineach domain [20], [79].

2.4 Generalization error

Before one starts designing algorithms, one should considerwhether it is at all possible to generalize across probabilitydistributions. Chiefly, such questions are studied by exam-ining the difference between the true error of an estimated

classifier h and the true error of the optimal classifier h∗,known as the generalization error [71]. Bounding the gen-eralization error often leads to insights on what conditionshave to be satisfied to achieve certain levels of performance.

First, we consider a generalization error bound for do-main adaptation that does not incorporate an adaptationstrategy. Instead, it characterizes the capacity of the sourceclassifier to perform on the target domain in terms ofdomain dissimilarity, sample size and classifier complexity.Specifically, it relies on the error of the ideal joint hypothesis:e∗S,T = minh∈H [eS(h) + eT (h)] [76], [77], [80]. If theerror of the ideal joint hypothesis is too large, then thereare no guarantees that the source classifier will performwell in the target domain. Given e∗S,T and the symmetricdifference hypothesis divergence DH∆H, one can state that

with probability 1 − δ, for δ ∈ (0, 1), the following holds(Theorem 3 in [76] for β = 0 and α = 0):

eT (hS)− eT (h∗T ) ≤ 2e∗S,T +

DH∆H(pS , pT ) + 4

√

2

n

(

ν log(2(n+1))+log8

δ

)

, (3)

where ν is the VC-dimension of the hypothesis space H [71].In words: the generalization error of the source classifierwith respect to the target domain is upper bounded by idealjoint hypothesis error, the domain dissimilarity and a factorconsisting of classifier complexity and sample size. Thisbound is specific to error functions, but can be generalizedto bounded real-valued loss functions [81].

The above bound is loose because it does not incorpo-rate an adaptation strategy. Now, the challenge is to findconditions that permit tighter bounds. In the methods wereview, we have found assumptions that lead to bounds foradaptive classifiers. Firstly, if one assumes that the posteriordistributions are equal in both domains pS(y |x) = pT (y |x)(see Section 3.1), then the difference between the targeterrors of the optimal target classifier and an importance-weighted classifier can be bounded in terms of the Renyidivergence between domains, the source sample size andclassifier complexity [79], [82]. Secondly, if one assumes thatthere exists a set of components, obtainable through a non-trivial transformation of the features t(x), that matches theclass-conditional distributions pS(t(x)|y) = pT (t(x)|y) (seeSection 4.3), then one can bound the generalization error of aclassifier trained on the transformed source data in terms ofthe divergence between the target data and the transformedsource data [83]. Thirdly, assuming that a robust algorithmcan achieve limited variation in loss for each partition itcreates (see Section 5.1), then one can bound the differencebetween the target error and the average maximal loss, withrespect to a shift in the posterior distributions, weightedby the probability of a sample falling in each partition [14].Lastly, if the assumption is made that the risk is small inthe part of the target domain where the source domain isuninformative (see Section 5.5), then one can bound thedifference between the source error weighted by the Renyidivergence and the target error, of the Gibbs classifier [84].

These assumptions provide some theoretical under-standing of the domain adaptation problem, and moremight exist still. They are insightful in that they reveal thatgeneralization depends strongly on the particular form ofdomain dissimilarity. We will discuss them in more detail intheir respective method categories.

3 SAMPLE-BASED APPROACHES

In sample-based approaches, we are interested in minimiz-ing the target risk through data from the source domain.One way to relate the source distribution to the target riskRT , at least superficially, is to consider:

RT (h) =∑

y∈Y

∫

Xℓ(h(x), y) pT (x, y) dx

=∑

y∈Y

∫

Xℓ(h(x), y)

pT (x, y)

pS(x, y)pS(x, y) dx . (4)

5

In order to deal with Equation 4, we need to determine theratio pT (x, y)/pS(x, y). Estimating that ratio would requirelabeled data from both domains. However, as discussed,target labels are considered unavailable. We therefore haveto make simplifying assumptions so that risk estimationbecomes possible without target labels.

Here we consider constrained forms of domain shifts:prior, covariate and concept shift [30], [85]. Concept shift, alsoknown as concept drift or conditional shift, will requireobservations of labeled data in both domains and is there-fore out of our scope [86]. Covariate shift corresponds todecomposing the joint distributions into p(y | x)p(x) andassuming that the posteriors remain equal in both domains,pS(y|x) = pT (y|x) [30]. Conversely, prior shift, also referredto as label or target shift, corresponds to decomposingthe joints into p(x | y)p(y) and assuming the conditionaldistributions remain equal pS(x | y) = pT (x | y) [30]. It hasbeen remarked in the causality community that covariateshift corresponds to causal learning (predicting effects fromcauses) and prior shift corresponds to anti-causal learning(predicting causes from effects) [87].

Of course, in practice, the underlying probability dis-tributions are unknown and assumptions on posteriors orconditionals cannot be verified. So, when are the posterioror conditional distributions equal? These assumptions areknown to hold in cases of sample selection bias [1], [85].In the sample selection bias setting, one differentiates be-tween a true underlying data-generating distribution anda sampling distribution [85]. If the sampling distributionis not uniform, then the resulting data will be biased withrespect to the underlying distribution. Since the underlyingdata-generating distribution remains constant, the underly-ing posteriors and conditionals remain equivalent betweenthe biased and unbiased samples [7]. Note that there aresubtle differences between sample selection bias, covariateshift and domain adaptation: sample selection bias is aspecial case of covariate shift because it involves a samplingdistribution and covariate shift is a special case of domainadaptation because the posteriors are assumed to be equal.

3.1 Data importance-weighting

The knowledge that the posteriors remain equivalent canbe exploited by decomposing the joint distributions andcanceling terms:

RW (h) =∑

y∈Y

∫

Xℓ(h(x), y)✘

✘✘✘pT (y | x)pT (x)

✘✘✘✘pS(y | x)pS(x)

pS(x, y)dx . (5)

In this case, the importance weights consist of the ratio ofthe data marginal distributions, w(x) = pT (x)/pS(x). Alarge weight indicates that the sample has high probabilityunder the target distribution, but low probability under thesource domain. As such, it is considered more importantto the target domain than samples with small weights. Theweights influence a classification model by increasing theloss for certain samples and decreasing the loss for others.Importance weighting is most often used in applicationsinvolving clinical or social science, but has been applied tonatural language processing as well [88].

Figure 3 shows a scatterplot of the example settingfrom Section 2.2. The importance of the source samples is

indicated with marker size. Training on these importance-weighted samples produces a decision boundary (dashedblack line) that is different from the one belonging tothe source classifier (solid black line) (see Figure 2). Theimportance-weighted classifier is said to have adapted to thedata from the hospital in Long Beach.

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Budapest, Hungaryhealthydisease

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Long Beach, CA, USApredict healthypredict disease

Fig. 3: Example of importance-weighting. (Left) The sourcesamples from Figure 1 have been weighted (larger dotsize is larger weight) based on their relative importanceto the target domain, producing the importance-weightedclassifier (black dashed line). (Right) Predictions made bythe adapted classifier.

Equation 5 is interesting, but does not tell us anythingabout the finite sample size case. For that, we will have alook at the difference between the true target error eT andthe empirical weighted source error eW of a given classifierh. With probability 1− δ for δ > 0 (Theorem 3, [79]):

eT (h)− eW(h)

≤ 25/4 2DR2 [pT ,pS ]/2 3/8

√

c

nlog

2ne

c+

1

nlog

4

δ, (6)

where DR2 [pT , pS ] is the 2-order Renyi divergence [79],and c is the so-called pseudo-dimension of the hypothe-sis space [89]. This bound indirectly tells us how far animportance-weighted classifier is from the optimal targetclassifier. Moreover, as the divergence between the domainsincreases, the sample size needs to increase at a certainrate as well, in order to maintain the same difference inerror. Analysis of importance-weighting further reveals thatan importance-weighted classifier will only converge if theexpected squared weight is finite, i.e. ES [w(x)

2] < ∞ [79].If the domains are too dissimilar, then this will not be thecase. In addition, asymptotically, weighting is only effectivefor a mis-specified classification model, e.g. a linear classifierfor a non-linear classification problem [15], [90], [91]. With acorrectly specified model, as the sample size goes to infinity,the unweighted estimator will find the optimal classifieras well [90]. In fact, with a correctly specified model, theweighted estimator will converge to the optimal classifierfor any fixed set of non-negative weights that sums to 1 [90].

Given that we have mis-specified our model, how shouldwe find appropriate importance weights? Indirect weightestimators first estimate the marginal data distribution ofeach domain separately and subsequently compute the ra-tio. Estimating each data distribution can be done paramet-rically, i.e. using a probability distribution with a fixed set ofparameters, or non-parametrically, i.e. using a distribution

6

with a variable set of parameters. With parametric weightestimators, one obtains a functional form of the resultingratio of distributions. For example, one could assume Gaus-sian distributions for each domain [91]. Then the weightfunction consists of:

w(xi) =N (xi | µT , ΣT )

N (xi | µS , ΣS). (7)

Such weight functions can be analyzed and often showinteresting properties. For example, should you choosemembers of the exponential family to estimate the datadistributions, then you can expect the variance of the im-portance weights to increase drastically [79], [92]. Highweight variance means that it is probable that a few sampleswill receive large weights. Consequently, at training time,the classifier will focus on those few samples designatedas important, and ignore the others. The result is often apathological classifier that will not generalize well.

The non-parametric alternative is to use kernel densityestimators [93], [94], [95]:

w(xi) =m−1

∑mj=1 κσT

(xi − zj)

n−1∑n

i′=1 κσS(xi − xi′ )

, (8)

where κ is a kernel function and σ denotes its bandwidth.The density of a source sample xi depends on the distanceto each kernel’s center. Bandwidths are considered hyperpa-rameters and can be tuned to produce smoother densities.Kernel density estimators have the advantage that they canbecome multi-modal: when samples are clustered in tworegions, the overlayed kernel functions naturally form amode on the cluster centers.

Instead of estimating the data distributions in each do-main and taking their ratio, the weights can be viewed asa separate variable [96], [97]. Following this interpretation,the weights are directly estimated using an optimizationprocedure where the discrepancy between the weightedsource and the target distribution is minimized [98]. Differ-ent methods use different discrepancy measures. Two con-straints are added to the optimization procedure. Firstly, allimportance weights should be non-negative. Secondly, theweighted source distribution should be a valid probabilitydistribution:

1 =

∫

XpT (x)dx =

∫

Xw(x)pS (x)dx ≈

1

n

n∑

i=1

w(xi) . (9)

The above approximate equality can be enforced by con-straining the absolute deviation of the weight average from1 to be less than some small ǫ. With these two constraints,the optimization problem becomes:

w = argminw∈R

n+

D[

w, pS(x), pT (x)]

(10)

such that |n−1∑n

i=1 wi − 1| ≤ ǫ. D refers to the discrepancymeasure, of which we discuss a few below. An advantage isthat additional constraints can be incorporated to achievecertain desired effects, such as low weight variance. Alimitation is that one needs to re-estimate weights when newsource samples are added as it produces no weight functionw(·), although some methods avoid this limitation [93], [99].

The most popular measure of domain dissimilarity isthe Maximum Mean Discrepancy (MMD) [100], [101]. It is

based on the distance between the expected values of twodistributions under the continuous function that pulls themmaximally apart [100]. But it can be approximated usinguniversal kernels [101]. Minimizing the empirical MMDwith respect to the importance weights, is called KernelMean Matching (KMM) [102], [103]:

DMMD

[

w, pS(x), pT (x)]

= ‖ ES [wφ(x)] − ET [φ(x)] ‖H

∝1

n2

n∑

i,i′

wiκ(xi, xi′ )wi′)−2

mn

n∑

i

m∑

j

wiκ(xi, zj) , (11)

where φ is the basis expansion associated with the Gaussiankernel. Constant terms are dropped as they are not rele-vant to the optimization procedure. Depending on how theweights are further constrained, algorithmic computationalcomplexities and convergence criteria can be derived as well[94], [103]. Optimization consists of a quadratic program inthe number of samples, which means that KMM in this formdoes not scale well to large data sets.

Another popular direct importance-weight estimatoris the Kullback-Leibler Importance Estimation Procedure(KLIEP) [104], [105]. The KL-divergence between the truetarget distribution and the importance-weighted source dis-tribution can be simplified to:

DKL

[

pS(x), pS(x)w(x)]

=

∫

XpT (x) log

pT (x)

pS(x)dx−

∫

XpT (x) logw(x)dx

∝ −1

m

m∑

j

logw(zj) . (12)

Note that this formulation requires weighting target sam-ples. New weights would have to be estimated for each newtarget sample. To avoid this, a functional model is proposed.It consists of the inner product of parameters α and basisfunctions φ, i.e. w(x) = φ(x)α [93]. The objective nowsimplifies to m−1

∑mj logφ(zj)α, where α is not dependent

on an individual sample. So, KLIEP can be applied to newtarget samples without additional weight estimation.

A third discrepancy is the L2-norm between the weightsand the ratio of data distributions [99], [106]. The squareddifference can be expanded and terms not involving theweights can be dropped. Using a functional model of theweights, w(x) = φ(x)α, and approximating expectationswith sample averages, the empirical discrepancy becomes:

DLS

[

w, pS(x), pT (x)]

=1

2

∫

X

(

w(x) −pT (x)

pS(x)

)2

pS(x)dx

∝1

2α⊤

( 1

n

n∑

i,i′

φ(xi)⊤φ(xi′ )

)

α−( 1

m

m∑

j=1

φ(zj))

α , (13)

where φ(xi)⊤φ(xi′ ) denotes the outer product of apply-

ing the basis functions to a single source sample [99].This method is called the Least-Squares Importance Fittingprocedure. Note that although this form resembles KernelMean Matching, it estimates α’s, which can have a differentdimensionality than the wi’s. KMM can be very impracticalin large data sets, because one needs to solve a quadraticprogram with n variables, one for each wi. Suppose the

7

dimensionality of φ is d, and d < n. Then, there are d α’s,and d variables for the quadratic program. As such, thisapproach can be computationally cheaper than kernel meanmatching. It can also be extended to be kernel-based [106].

There are also direct weight estimators that do notemploy optimization. A simple procedure is to use logis-tic regression to discriminate between samples from eachdomain [107]. In that case, the inverse estimated posteriorprobabilities become the importance weights. Additionally,Nearest-Neighbour Weighting is based on tessellating thefeature space into Voronoi cells, which approximate a prob-ability distribution function in the same way as a multi-dimensional histogram [108], [109]. To obtain weights, oneforms the Voronoi cell Vi of each source sample xi with thepart of feature space that lies closest to xi [109]. The ratioof target over source is then approximated by counting thenumber of target samples zj that lie within each Voronoicell:

wi = |Vi ∩ {zj}mj=1| , (14)

where | · | denotes cardinality. Counting can be done bya nearest-neighbours algorithm [64], [109]. This estimatordoes not require hyperparameter optimization, but Laplacesmoothing, which adds a 1 to each cell, can additionally beperformed [109].

Lastly, it is also possible to avoid the two-step procedure,and optimize the weights simultaneously with optimizingthe classifier [110], [111].

3.2 Class importance-weighting

Similar to before, the knowledge that the conditionals areequivalent can be exploited by canceling them. This pro-duces the following weighted risk function for prior shiftcorrection:

RW (h)=∑

y∈Y

∫

Xℓ(h(x), y)✘

✘✘✘pT (x | y)pT (y)

✘✘✘✘pS(x | y)pS(y)

pS(x, y)dx . (15)

where w(y) = pT (y)/pS(y) are the class weights, correctingfor the change in class priors between the domains. Weight-ing in this manner is related to cost-sensitive learning [112]and class imbalance [113]. The same effect can be achievedby under- or over-sampling data points from one class [114].

Nonetheless, weighting based on target labels is outsidethe scope of this review, and we will not discuss it further.In contrast, there are a variety of methods that estimateclass importance weights without requiring target labels [8],[82], [115]. In Black Box Shift Estimation (BBSE), one takesa black-box predictor h, computes the confusion matrixCh(x),y on a validation split of the source data and makespredictions for the target data [8]. The class weights can thenbe obtained by taking the product of the inverse confusionmatrix and the predicted target prior:

wk = C−1h(x),k pT (h(z) = k) , (16)

where pT (h(z)) are the empirical proportions of the classi-fier’s predictions. The difference between the estimated andthe true class weights can be bounded tightly [8]. The maindifficulty in estimating class weights lies in the fact that esti-mating the confusion matrix becomes unstable if it is nearlysingular, which is especially a problem when the sample size

is small [82]. In BBSE, this is avoided by using soft labelsinstead of hard ones [8]. Alternatively, one could regularizethe weight estimator [82]. Regularized Learning of LabelShift first reparameterizes the class weights to υ = w−1 andthen estimates the reparametrized weights using a stableprocedure. They are transformed back into the weights,w = 1 + λυ, using a regularization term λ that dependson the source sample size. The procedure lends itself wellto analysis and a generalization error bound on the classimportance-weighted estimator can be formed (Theorem 1,[82]). This bound also utilizes the Renyi divergence, similarto the one for data importance-weighting [79].

Kernel Mean Matching has also been extended to esti-mating weights for prior shift [115]. The class-conditionaldistributions in the source domain can be estimated andaveraged over the weighted source priors to produce anew data marginal distribution. Using the Maximum MeanDiscrepancy between the target data distribution and thenew data marginal, the following class weight estimator isobtained:

wk=argminw∈R

K+

‖U [pS(x | y)]ES [wψ(y)] − ET [φ(x)]‖2 , (17)

where U [pS(x | y)] is the product of the cross-covarianceCX ,Y and the inverse class covariance C−1

Y,Y , and ψ(y) is abasis expansion of the weights. The prior shift assumptionthat the class-conditionals are equal can be even relaxed toallow for location-scale changes [115].

4 FEATURE-BASED APPROACHES

In some problem settings, there potentially exists a transfor-mation that maps source data onto target data [24], [116]. Inthe example setting, the average age of patients in Budapestis lower than that of the patients in Long Beach. One couldconsider reducing the discrepancy between the hospitalsby shifting the age of the Hungarian patients upwardsand training a classifier on the shifted data. Figure 4 (left)visualizes such a translation with the vector field in thebackground. The adapted decision boundary (black dashedline) is a translated version of the original naive sourceclassifier (black solid line).

It has been argued that matching source and target datais justified by the generalization error bound from Section2.4 [10], [117], [118], [119]. That would shrink the domaindiscrepancy and produce a tighter bound on the general-ization error. However, note that matching the data distri-butions, pS(t(x)) ≈ pT (x) where t(x) is the transformationfunction, does not imply that the conditional distributionswill be matched as well, pS(y | t(x)) ≈ pT (y | x) [83], [115].To achieve such a result, stronger conditions are necessary(see Sections 4.2 and 4.3).

4.1 Subspace mappings

In certain problems, the domains contain domain-specificnoise but common subspaces. Adaptation would consist offinding these subspaces and matching the source data tothe target data along them. One of the most straightforwardof such techniques, called Subspace Alignment, computesthe first d principal components in each domain, CS andCT , where d < D [10]. A linear transformation matrix

8

20 30 40 50 60 70 80age

100

200

300

400

500

600ch

oles

terol

Budapest, Hungaryhealthydisease

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Long Beach, CA, USApredict healthypredict disease

Fig. 4: Example of a feature-based method. (Left) The sourcesamples from Figure 1 have been translated to match thedata from the target domain. Subsequently, a classifier istrained on the mapped source data (black dashed line).The original naive classifier (black solid line) is shown forcomparison. (Right) The adapted classifier is applied to thetarget samples and predictions are shown.

is then computed that aligns source components to targetcomponents: M = C⊤

S CT . The adaptive classifier projectsdata in each domain to their components, maps the pro-jected source data using M and trains on the transformedsource data. Extensions of this approach include basing thealignment on particular landmarks (Landmarks Selection-based Subspace Alignment) [120], basing the alignment onboth the data distributions and the subspaces (Subspace Dis-tribution Alignment) [121] and training a classifier jointlywith learning a subspace (Joint cross-domain Classificationand Subspace Learning) [122].

A slightly more complicated technique is to modelthe structure of the data with graph-based methods, in-stead of aligning directions of variance [123], [124]. Datais first summarized using a subset called the exemplars,obtained through clustering. From the set of exemplars, twohyper-graphs are constructed. These two hyper-graphs arematched along their first, second and third orders usingtensor-based algorithms [123], [124]. Higher-order momentsconsider other forms of geometric and structural informa-tion, beyond pairwise distances between exemplars [123],[125].

A further step can be taken by assuming that thereexists a manifold of transformations between the source andtarget domain [95], [126], [127]. This manifold consists ofa space of parameters, where each point would generate apossible domain. For example, the manifold might consistof a set of camera optics parameters, where each settingwould produce data in a different part of feature space[95], [128]. The manifold assumption is interesting, becauseit implies that there exists a path along the manifold’ssurface from the source to the target domain [129], [130].Every point along that path could generate an intermediatedomain [131], [132]. For example, the space of all linearsubspace transformations is termed the Grassmann manifold[133]. A path along the Grassmannian generates many lineartransformations, which could, in turn, generate intermediatedomains [132]. These intermediate domains can be usedduring training to inform a classifier on how to adapt itsdecision boundary.

But it is also possible to incorporate the entire path.

Geodesic Flow Kernel forms a kernel consisting of the innerproduct between two feature vectors xi and xj projectedonto the t-th subspace, for t ∈ [0, 1] [9]. It then integratesover t to account for all intermediate subspaces:G(xi, xj) =∫ 10 xiM(t)M(t)⊤x⊤j dt where M(t) is the projection matrix

at time t. At t = 0, the projection consists of purely thesource components, M(0) = CS , and at t = 1, it consistspurely of the target components, M(1) = CT . The resultingkernel can be used in combination with a support vectormachine or a kernel nearest-neighbour [9], [131].

An alternative is to consider statistical manifolds [95].A path on the statistical manifold may describe a sequenceof parameters that turns one distribution into another, forinstance a sequence of means between two Gaussian distri-butions. The length of the geodesic path along the statisticalmanifold is called the Hellinger distance, and can be used asa measure of domain discrepancy [134]. Adaptation consistsof finding a sequence of parameters based on minimumHellinger distance, and use these in a similar fashion as theGeodesic Flow Kernel [95], [135].

Lastly, if one believes that the transformation fromsource to target is stochastic, one could consider a stochasticmapping. Feature-level domain adaptation (FLDA) mini-mizes risk under a transfer model describing the probabilitywhere a source sample would lie in feature space, if it werepart of the target domain [136]. For example, suppose thatdata is collected with a rich set of features in the sourcedomain, but in the target domain, there is a chance thatsome features will not be collected. A distribution could befitted to source and target data to estimate the probability ofa feature dropping out. The FLDA classifier trained underthis transfer model will effectively not rely on features witha high probability of dropping out [136].

4.2 Optimal transport

Another class of transformation-based techniques is optimaltransport. In optimal transport, one estimates probabilitymeasures from data and finds a mapping of minimal costthat equates the measures [11], [137]. For domain adap-tation, one assumes that there exists a transformation t(·)that matches the source and target posterior distributionsin the following way: pT (y | t(x)) = pS(y | x) [11]. Thetransportation map t is also called a push-forward, in this casefrom pS to pT . After applying the transformation, the sourcesamples resemble the target samples and a classifier trainedon the transformed source samples would be suitable toclassifying target samples.

Finding the optimal transportation map between twomeasures among the set of all possible transformationsis intractable [11]. Therefore, the optimization problem isrelaxed into a search over a joint probability measure γ, withmarginals pS(x) and pT (x), that has minimal distance be-tween points. The joint measure couples the two marginalstogether, allowing one to be obtained from the other. Thisnew objective corresponds to the Wasserstein distance:

DW [pS(x), pT (x)] = infγ∈Γ

∫

X×Xd(x, z)dγ(x, z) , (18)

where Γ refers to the set of all joint measures on X ×X withmarginals pS(x) and pT (x). The function d(·, ·) refers to a

9

distance metric between two points, usually taken to be theEuclidean norm [11]. The joint measure γ∗ that minimizesthe Wasserstein distance between the two marginals is calledthe transportation plan.

In practice, the data marginals are estimated using theempirical distributions, pS(x) =

∑ni=1 P

Si δ(x − xi) and

pT (x) =∑m

j=1 PTj δ(z − zj) where δ(·) is a Dirac function

and Pi is the probability mass assigned to the i-th sample[138]. The set Γ becomes the set of all non-negative matricesof size n × m where the sum over columns correspondsto the empirical source distribution pS and the sum overrows corresponds to the empirical target distribution pT[11]. The transportation plan γ∗ can now be found byminimizing the inner product of γ and the distance matrix:γ∗ = argminγ ‖γ⊤C‖F where ‖ · ‖F denotes the Frobeniusnorm and Cij = d(xi, zj) is the distance matrix [11]. Thisminimization is expensive due to its combinatorial nature,but it can be relaxed through regularization. To actuallytransform the source samples, one takes the barycentricmapping, xi = argminx

∑mj=1 γ

∗i d(x, zj) [11].

The advantage of the optimal transport formulation isthat it is possible to exploit structure in the problem set-ting. For instance, if the source data conforms to a graph,that graph can directly be incorporated through Laplacianregularization [11]. Additionally, the measures need not bedata marginals, but could also be joint distrbutions [137].Furthermore, generalization error bounds can be formed,with forms similar to the one described in Section 2.4 [137],[138]. The Wasserstein distance is important to generativeadversarial networks, and there have been a number ofworks combining optimal transport with deep learning[139], [140], [141].

4.3 Domain-invariant spaces

The problem with transforming data from one domain tomatch another, is that the representation remains domain-specific. But variation due to domains is often more of anuisance than an interesting factor. Ideally, we would like torepresent the data in a space which is domain-invariant. Theadvantage of this approach is that classification becomesthe same as standard supervised learning. Most domain-invariant projection techniques stem from the computervision and image processing communities, where domainsare often caused by image acquisition procedures [43].

In cases where there is a causal relationship betweenthe variables X and Y such that Y causes X , it is possi-ble to bound the difference between the class-conditionaldistributions in terms of the difference between the datadistributions [83], [115]. Under the conditions specifiedby the bound, mapping data to a domain-invariant spacewould also match class-conditional distributions. To explainthis result, we must first define the notion of ConditionalInvariant Components: Xci are d-dimensional components ofX , obtained by transforming the data Xci = t(x), such thatpS(x

ci | y) = pT (xci | y) [83]. Now, it is necessary to make

two assumptions. Firstly, the transformation t is assumedto be non-trivial. A trivial transformation is one where theconditional distributions of the transformed data are depen-dent for each class. For example, multiplying each featurewith 0. Secondly, all linear combinations of the source class-conditional and the target class-conditional distributions

are assumed to be linearly independent of each other, forany two classes. If these two assumptions hold, then thefollowing holds (adapted from Lemma 1 and Theorem 2from [83], assuming equal class priors pS(y) = pT (y)):

eT (h)−eS(h) ≤ Jci1(0,π/2](θ)+

2

sin2 θJci

1(π/2,π)(θ), (19)

where eS(h) is the error on the transformed data,pS(t(x), y), and 1 is the indicator function. Jci =‖ES [φ(t(x))] − ET [φ(t(x))‖2 is the MMD divergence be-tween the transformed source and the transformed tar-get data. When Jci is 0, the source distribution is trans-formed perfectly in the target distribution. The differencebetween the error on the transformed source data andthe target error will then be 0. Above, θ is the anglebetween two Kronecker delta kernels, where that kernelis: ∆c = pS(y = c) Exci∼pS(xci | y=c)

[

φ(xci)]

− pT (y =c) Exci∼pT (xci | y=c)

[

φ(xci)]

for c ∈ Y . Each delta kernelconsists of the difference between the source and targetdistributions over the invariant components for that class,weighted by the class prior. If θ is between π/2 and π, thebound becomes looser as θ goes to π. At θ = π, Jci cannotbe used as an upper bound.

A simple approach to finding a domain-invariant spaceis to find principal components based on joint directions ofvariation [12], [142]. In order to do so, the joint domain ker-nel, K = [κS,S κS,T ; κT ,S κT ,T ], is first constructed [143].Data projected onto components C should have minimaldistance to the empirical means in each domain [12]. Assuch, components are extracted by minimizing the trace ofthe projected joint domain kernel:

minimizeC

tr(C⊤KLKC)

s.t. C⊤KHKC = I , (20)

where L is the normalization matrix that divides each entryin the joint kernel by the sample size of the domain fromwhich it originated, and H is the matrix that centers K[12]. The constraint is necessary to avoid trivial solutions,such as projecting all data to 0. This technique is known asTransfer Component Analysis [12]. It resembles kernel PCAand, likewise, its optimization consists of an eigenvaluedecomposition [144]. A regularization term tr(C⊤C) can beincluded to control the complexity of the components andavoid rank deficiencies in the eigendecomposition.

Learning domain-invariant components can be done ina number of ways. The Domain-Invariant Projection ap-proach from [129], later renamed to Distribution-MatchingEmbedding (DME) [135], aims to find a projection matrixthat minimizes the MMD:

DDME[M,pS , pT ] = ‖ ES [φ(xM)] − ET [φ(xM)] ‖H , (21)

where M is the projection matrix that is being minimizedover, with the additional constraint that it remains orthonor-mal;M⊤M = I . This constraint is necessary to avoid patho-logical solutions to the minimization problem. It is possibleto add a regularization term that punishes the within-classvariance in the domain-invariant space, to encourage classclustering. Alternatively, the same authors have also pro-posed the same technique, but with the Hellinger distanceinstead of the MMD [135]. This approach resembles Transfer

10

Component Analysis, but minimizes discrepancy instead ofmaximizing joint domain variance [12]. It has been extendedto nonlinear projections as well [135].

Alternatively, [145] proposed to learn the values of theMMD kernel itself: instead of weighting or projecting sam-ples and then using a universal kernel to measure theirdiscrepancy, it is also possible to find a basis function forwhich the two sets of distributions are as similar as possible.The space spanned by this learned kernel then correspondsto the domain-invariant space. Considering that differentdistributions generate different means in kernel space, it ispossible to describe a distribution of kernel means. The vari-ance of this meta-distribution, termed distributional variance,should then be minimized to obtain the proposed learnedMMD kernel [145]. The functional relationship between theinput and the classes can be preserved by incorporating acentral subspace in which the input and the classes are con-ditionally independent [146]. Constraining the optimizationobjective while maintaining this central subspace, ensuresthat classes remain separable in the new domain-invariantspace. This technique is coined Domain-Invariant Compo-nent Analysis (DICA) [145]. It has been expanded on for thespecific case of spectral kernels by [117]. Other techniquesof this type include information-theoretic learning, wherethe authors assume that classes across domains are stillclustered together in high-dimensional space [147].

One could also find the subspace within the data fromwhich the reconstruction to both domains is optimal [148],[149]. Transfer Subspace Learning minimizes the Bregmandivergence to both domains, with respect to a projectionto a subspace [148]. In practice, the source data is mappedto a lower-dimensional representation, and then mappedback to the original dimensionality. The reconstruction errorthen consists of the mismatch between the reconstructedsource samples and the target samples, measured throughthe squared error or the Frobenius norm for instance [150].

4.4 Deep domain adaptation

Reconstructing the target data from the source data can bedone through an autoencoder. Autoencoders are forms ofneural networks that reconstruct the input from their hiddenlayers: ‖ψ(ψ(xM)M−1)−x‖ whereM is a projection matrixand ψ is a nonlinear activation function [151]. There aremultiple ways of avoiding trivial solutions to this objective;one formulation will extract meaningful information bypushing the input through a bottleneck (contractive autoen-coders) while another adds artificial noise to the input whichthe network has to remove (denoising autoencoders) [152].Deep autoencoders can stack multiple layers of nonlinearfunctions on top of each other to create flexible transforma-tions [153], [154]. Stacking many layers can increase compu-tational cost, but computations for denoising autoencoderscan be simplified through noise marginalization [155].

Autoencoders are not the only neural networks thathave been applied to domain adaptation. One of the mostpopular adaptive networks is the Domain-Adverserial Neu-ral Network (DANN), which aims to find a representationsuch that the domains cannot be distinguished from eachother while correctly classifying the source samples [119].It does so by including two loss layers: one loss layer

classifies samples based on their labels, while the otherloss layer classifies samples based on their domains. Duringoptimization, DANN will minimize label classification losson while maximizing domain classification loss. Domain-adversarial networks mostly rely on the generalization errorbound from Section 2.4: if the domain discrepancy is small,the target error of a source classifier will be small as well.

Maximizing the classification error of samples from dif-ferent domains is equivalent to minimizing the proxy A-distance:

DA[x, z] = 2(

1− 2 e(x, z))

, (22)

where e(x, z) is the cross-validation error of a classifiertrained to discriminate source samples x from target sam-ples z [76], [156]. The proxy A-distance is derived fromthe total variation distance [156]. Other distances have beenemployed for classifying domains, most notably those basedon moment-matching [157] or using the Wasserstein dis-tance [158]. These can be computationally expensive, andthere is work on developing less expensive metrics such asthe Central Moment Discrepancy or the Paired HypothesesDiscrepancy [159], [160].

A limitation of domain-adversarial networks is thatmatched data distributions do not imply that the class-conditional distributions will be matched as well, as alreadydiscussed at the start of this section. Furthermore, the twoloss layers produce gradients that are often in differentdirections. Because of this, DANNs can be harder to trainthan standard deep neural nets. But recent work has lookedat stabilizing the learning process. The DIRT-T approachaccelerates gradient descent by incorporating the naturalgradient and guiding the network to avoid crossing high-density data regions with its decision boundary [161]. Theobjective can also be stabilized by replacing the domain-confusion maximizing part of the objective with its dualformulation [162].

The idea of maximizing domain-confusion while mini-mizing classification error has been explored with numer-ous network architectures, such as with residual layers[163], through generative adversarial networks [164], bytying weights at different levels [165], embedding kernelsin higher layers [166], aligning moments [157] and model-ing domain-specific subspaces [167]. They have also beenapplied to a variety of problem settings, such as speechrecognition [168], medical image segmentation [41] andcross-language knowledge transfer [169]. We refer to [24],[25], [26] for extensive reviews.

4.5 Correspondence learning

High-dimensional data with correlated features is commonin natural language processing as well. In a bag-of-words(BoW) encoding, each document is described by a vectorconsisting of the word occurrence counts. Words that signaleach other, tend to co-occur and lead to correlating features.Now, suppose a particular word is a strong indicator ofpositive or negative sentiment and only occurs in the sourcedomain. One could find a correlating word, referred toas a pivot word, that occurs frequently in both domains[13], [170]. Then find the word in the target domain thatcorrelates most with the pivot word. This target domain

11

word is most likely the corresponding word to the originalsource domain word and will be a good indicator of positiveversus negative sentiment as well [13]. Thus, by augmentingthe bag-of-words encoding with pivot words and learningcorrespondences, it is possible to construct features commonto both domains [13], [171]. Later approaches generalizecorrespondence learning through kernelization [172].

5 INFERENCE-BASED APPROACHES

In this section, we cover methods that incorporate adap-tation in the inference procedure. There are many suchways: reformulating the optimization objective, incorporat-ing constraints based on properties of the target domain orincorporating uncertainties through Bayesian inference. Asa result, this category is more diverse than the others.

5.1 Algorithmic robustness

An algorithm is deemed robust if it can partition a labeledfeature space into disjoint sets, or regions, such that the vari-ation of the classifier’s loss is bounded [173]. This impliesthat when a training sample is removed from a region, thechange in loss is small. Algorithmic robustness naturallyextends to the case of data set shift, if one considers the shiftto be the addition and removal of samples between trainingand testing [14]. A classifier that is robust to such changes,can be trained on the source domain and would generalizewell to the target domain.

Changes over posterior probabilities in regions of fea-ture space can be described by λ-shift. It is said that thetarget posterior is λ-shifted from the source if the follow-ing holds for all x ∈ Xr and for all K classes: pT (y =k | x) ≤ pS(y = k | x) + λpS(y 6= k | x) andpT (y = k | x) ≥ pS(y = k | x)(1 − λ) where Xr refersto the r-th region. Note that for λ = 0, the posteriors areexactly equal, and for λ = 1, the posteriors can be arbitrarilydifferent. In essence, this measure is a generalization of theassumption of equal posterior distributions that was used inimportance-weighting methods for covariate shift. λ-shift isa less strict assumption and, hence, algorithms based on λ-shift are more widely applicable than importance-weightingmethods.

Using the notion of λ-shift and a robust algorithm, thefollowing holds with probability 1 − δ, for δ > 0 (Theorem2, [14]):

eT (h)−∑

Xr∈X

pT (Xr) ℓλ(h(x), Xr)

≤ ξ

√

2K log 2 + 2 log 1δ

n+ ǫS . (23)

where ξ is the upper bound on the loss function, ǫS isthe bound on the change in loss when a source sample isreplaced,K refers to the number of regions and pT (Xr) con-sists of the empirical probability of a target sample fallingin region Xr . ℓλ(h(x), Xr) corresponds to the maximal lossthe classifier h incurs with respect to the source samples inthe region Xr, taking into account the λ-shift in posteriorprobability in that region.

Incorporating λ-shift into SVM’s produces λ-shift SVMAdaptation (λ-SVMA) [14]. In its most pessimistic variant,

no restrictions are put on the difference between the poste-rior distributions (i.e. λ is set to 1). Pessimistic λ-SVMA’sdual optimization problem formulation is identical to astandard SVM, except for the addition of an additional con-straint: the sum of weights belonging to source samples xifalling in region Xr has to be less than or equal to the trade-off parameter τ times the empirical target probability of ther-th region;

∑

i∈Xrai ≤ τ pT (Xr). This could be interpreted

as that the weights are not allowed to concentrate on regionsof feature space where there are few target samples. Robustforms of ridge and lasso regression have been formed [14],and this approach has been extended to boosting [174]. Italso resembles another robust SVM approach that aims tofind support vectors for the target domain that producestable labelings [175].

5.2 Minimax estimators

One could view domain adaptation as a classification prob-lem where an adversary changes the properties of the testdata set with respect to the training set. Adversarial settingsare formalized as minimax optimization problems, wherethe classifier minimizes risk, with respect to the classifier’sparameters, and an adversary maximizes it, with respectto an uncertain quantity [176], [177], [178]. By workingunder maximal uncertainty, the classifier adapts in a moreconservative manner.

A straightforward example of such a minimax estima-tor is the Robust Bias-Aware classifier [176]. Its uncertainquantity corresponds to the target domain’s posterior dis-tribution. It minimizes risk for one classifier h while theadversary maximizes risk with respect to another classifierg. However, given full freedom, the adversary would alwaysproduce the opposite classifier and its optimization proce-dure would not converge. A constraint is imposed, tellingthe adversary that it needs to pick posteriors that match themoments of the source distribution’s feature statistics. Theestimator is now written as [176]:

h = argminh∈H

maxg∈H∩Ξ

1

m

m∑

j=1

ℓ(h(zj), g(zj)) , (24)

where Ξ represents the set of feature statistics that character-ize the source domain’s distribution and H∩Ξ indicates therestriction of the hypothesis class to functions that producefeature statistics equivalent to that of the source domain.This estimator produces a classifier with high-confidencepredictions in regions with large probability mass underthe source distribution and low-confidence predictions inregions with small source probability mass.

Another minimax estimator, called the Target Con-trastive Robust risk estimator [178], focuses on the perfor-mance gain that can be achieved with respect to the sourceclassifier. By contrasting the empirical target risk of a newclassifier with that of the source classifier, one can effectivelyexclude classifier parameters that are already known toproduce worse risks than that of the source classifier. Theestimator is formulated as:

hT =argminh∈H

maxq

1

m

m∑

j=1

ℓ(

h(zj), qj)

− ℓ(

hS(zj), qj)

, (25)

12

where hS is the source classifier and q denotes soft targetlabels. If no parameters can be found that are guaranteedto perform better than that of the source classifier’s, then itwill not adapt. It thereby explicitly avoids negative transfer[179], [180]. For discriminant analysis models, it can even beshown that the empirical target risk of the TCR estimate is

strictly smaller than that of the source classifier, RT (hT ) <R(hS), for the given target samples (Theorem 1, [178]). Itis hence a transductive classifier. Its weakness is that it canperform poorly if the source classifier is a bad choice for thetarget domain to begin with.

Figure 5 presents an example of the Target ContrastiveRobust risk estimator. By taking into account the uncertaintyover the labeling of the target samples, the classifier adaptsconservatively (black dashed line). In this case, it is minimiz-ing the empirical risk on the target samples for the worst-case labeling.

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Budapest, Hungaryhealthydisease

20 30 40 50 60 70 80age

100

200

300

400

500

600

choles

terol

Long Beach, CA, USAq(disease|x)

0.2

0.4

0.6

0.8

Fig. 5: Example of a inference-based method. The targetsamples are soft-labeled through an adversary. The adaptedclassifier (dashed line) deviates from the unadapted sourceclassifier (solid line) such that the empirical risk on the targetsamples (right) is lower than that of the source classifier.

An alternative quantity with uncertainty is the setof weights from importance-weighting. Small changes inweights could have large effects on the resulting set ofclassifier parameters and, by extension, generalization per-formance. One can be less sensitive to poor weights byminimizing risk with respect to worst-case weights [15]:

h = argminh∈H

maxw∈W

1

n

n∑

i=1

ℓ(h(xi), yi)w(xi) , (26)

where W is restricted to the set of non-negative weightsthat average to 1 (see Section 3.1). Minimizing the adversar-ially weighted loss produces more conservative estimates asthe domain dissimilarity increases. Worst-case importance-weights have also been studied from a distributionallyrobust optimization perspective [181].

Finally, variations on minimax estimators have beenproposed in the context of domain-adversarial networks[182], multiple-kernel support vector machines [183] andwith using Wasserstein distances [184].

5.3 Self-learning

A prominent approach in semi-supervised learning is toconstruct a classifier based on the labeled data and makepredictions some of the unlabeled data. These predictionsare then treated as labeled data in a following iteration and

used for re-training the classifier. This is known as iterativeself-labeling or self-learning [185]. A similar procedure canbe set up for domain adaptation: train a classifier on thesource data, predict the labels of the some of the targetsamples, and use these labeled target samples to re-trainthe classifier [16], [186].

But which target samples to use first? Co-training is anapproach that uses a logistic regressor to soft-label targetsamples [187]. The ones with a posterior probability largerthan some threshold, are self-labeled and used in the nextiteration of training the classifier. The threshold on the pos-terior will drop in each iteration, ensuring that the classifierbecomes stable.

Another method, called domain adaptation support vec-tor machine (DASVM) [16], will progressively find 2k sourcesamples and replace them by 2k newly labeled target sam-ples. The k samples closest to the margins spanned bythe current support vectors are selected. DASVM has twocost factors that trade-off the source error and the targeterror. By increasing the cost factor for the target error whiledecreasing the cost for the source error over time, the clas-sifier focuses increasingly on its performance in the targetdomain. DASVM has to re-train each iteration, which meansit has to solve multiple quadratic optimization problems.That can be computationally expensive. If the data is in theform of strings or trees, these optimization problems can beeven more expensive [188]. The problem in this case is thatthe underlying kernel has to be positive semi-definite (PSD)and symmetric. By relaxing this constraint and employingsimilarity functions that need not be PSD or symmetric, asignificant speed-up can be achieved [188].

If the target labels are treated as hidden variables, theninference can be done using an Expectation-Maximizationprocedure [186], [189]. Adaptation with Randomized Expec-tation Maximization (Ad-REM) uses the source classifica-tion model to compute the expected labeling of the targetsamples [190]. Given the expected labels, it maximizes theclassification model’s likelihood with respect to its parame-ters. It is hence an iterative self-labeling procedure. Whereit differs from previous approaches, such as DASVM, is thatit enforces class balance in the target samples: after eachexpectation step, a roughly equal amount of target samplesassigned to each class is taken for the maximization step[190]. Class balance ensures more stable predictions.

Self-labeling can be combined with a transformation-based technique. Balanced Distribution Adaptation (BDA)uses a source classifier to obtain soft labels for the targetsamples [191]. These soft labels are then used to estimatethe conditional distributions in the target domain, pT (x | y).In turn, the conditionals are used to match the domainsvia a linear transformation. After transforming the sourcedomain, the source classifier is re-trained and the targetsamples are assigned new soft labels. This procedure is re-iterated until convergence.

An alternative procedure is based on PAC-Bayesianweighted majority votes [192], [193]. First, the notion ofperturbed-variation (PV) self-labeling is introduced, a mea-sure that returns the proportion of target samples that arenot within an ǫ-radius of any source sample [192]. PV is0 when all target samples are close to a source sample.The matched target samples, i.e. those within range, are

13

assigned the same label as their closest source sample.This constitutes the self-labeled set, which is later used forestimating the weighted majority vote for an ensemble ofclassifiers. In total, PV-minCq performs one iteration of self-labeling followed by ensemble training [192].

Existing domain adaptation approaches can be extendedto include self-learning. Joint Distribution Adaptation (JDA)is an extension of Transfer Component Analysis [12], [194].It pseudo-labels the target samples in order to find transfercomponents based on both the data marginal distributionsand the class-conditional distributions [194]. JDA can avoidfinding transfer components where the class-conditionaldistributions differ strongly. Similarly, graph-based match-ing can be extended to include a pseudo-labeling stage [124].

The advantage of a self-labeling approach is that it al-lows for validation: using the self-labeled target samples asa validation set, the current classifier’s hyperparameters canbe optimized [16]. This is also known as the reverse validationstrategy [16], [192], [193].

5.4 Empirical Bayes

In Bayesian inference, one forms a data likelihood functiongiven a set of parameters and poses a prior distributionover these parameters [195]. Once data has been observed,one can derive the posterior distribution of the parameters,and make decisions based on Maximum A Posteriori (MAP)estimation. Crucial to this inference process is the choice ofprior distribution. Often, the shape of this prior distributiondepends on what the expert believes are suitable values. Inthe absence of an expert, so-called uninformative priors canbe constructed.

In domain adaptation problems, the labeled source do-main acts as a form of prior knowledge [17], [42], [196].One way to incorporate this knowledge in the inferenceprocess, is by fitting the prior distribution to the sourcedata. For example, in NLP classification tasks with bag-of-words feature vectors, we know that features correlateheavily with each other [17]. Suppose a prior distribution isset on parameters of a linear classifier where each featurecorresponds to a word. An uninformative prior for theclassifier’s parameters might look like a diagonal covariancematrix, i.e. no word correlations. But if one has access to alarge data set of text documents, e.g. Wikipedia, one couldestimate correlations between words. These estimates canreplace the covariance matrix of the prior distribution. It isnow informative as it contains knowledge gained from thesource domain, and can improve performance of the targetclassifier. Note that this is a form of empirical Bayes as theprior is estimated from data [197]. The main difference fromstandard Empirical Bayes is that the data used for fitting theprior originates from a different distribution than the datathat will be used for the likelihood function.

In a sense, the source domain informs the model on whatkind of feature structure could be expected in the targetdomain [198]. A limitation of this approach is that, if thedomains differ too much, then the informative prior willconcentrate on a region of parameter space that is furtheraway from the optimal posterior than the uninformativeprior, and produce more uncertain predictions.

Constructing informative priors using the source domainhas been done in a number of settings: the recognition of

speech utterances of variable length [199], script knowl-edge induction [200], cross-domain action recognition [201],cross-center brain tissue segmentation [42], using decision-tree priors [74], [202] and using hierarchical priors [196].

5.5 PAC-Bayes

The PAC-Bayesian framework is a combination of PAC-learning and Bayesian inference [18], [192], [193]. One de-fines a prior distribution π over an hypothesis space Hof classification functions [18], [193]. Through observingdata, one aims to obtain a posterior distribution ρ over H.The expected prediction with respect to hypotheses drawnfrom ρ is known as the ρ-weighted majority vote, or Bayesclassifier [18], [192].

PAC-Bayes departs from standard Bayesian inference byreplacing the consideration of ρ over the entire hypothesisspace by, what is known as, the Gibbs classifier [193]. TheGibbs classifier Gρ draws, according to ρ, a single hypothe-sis from H to make a prediction. With it, a domain-adaptivePAC-Bayesian generalization error bound can be derivedsimilar to the one from Section 2.4 [193]. It contains thesource risk of the Gibbs classifier plus a discrepancy termand the error of the ideal-joint-hypothesis. The discrepancyconsists of the absolute difference between the disagree-ments on each domain with respect to ρ [193]:

Dρ[pT , pS ] = | dS(ρ)− dT (ρ)| , (27)

where dS(ρ) = ESEh,h′∼ρ1[h(x) 6= h′(x)] and likewise fordT , are the domain disagreements.

Building on the previous results, a new PAC-Bayesianbound was derived that is not dependent on the error ofthe ideal joint hypothesis, Instead, it utilizes a term, ηT \S ,describing the risk in the part of the target domain for whichthe source domain is uninformative and a factor weightingthe source error [84]:

RT (Gρ)− βα(pT ‖pS)(

eS(ρ))1− 1

α ≤1

2dT (ρ) + ηT \S . (28)

This factor is an exponentiated Renyi divergencelog2 βα(pT ‖pS) = α−1

α DαR[pT ‖pS ]. For α = 2, it wouldequal the Renyi divergence from the bound for importanceweighting (c.f. Equation 6) if not for the fact that it is withrespect to the joint distribution in each domain instead of thedata distributions. This divergence focuses on those parts offeature space where the source domain’s support is withinthe target domain’s support. Effectively, the source error haslittle to no influence in parts of feature space supported bythe source domain but not the target domain.

The above bound can be expressed in computable termsby taking a linear Gibbs classifier and assuming a Gaussiandistribution over classifier parameters for ρ [84]. The ex-pected disagreement is bounded by the empirical disagree-ment and the Kullback-Leibler divergence between the priorπ and the posterior ρ over the hypothesis space. For spher-ical Gaussian distributions, the KL-divergence between πand ρ constitutes the squared norm of the parameters.Similarly, the expected source error can be upper boundedby the empirical error and, again, the Kullback-Leiblerdivergence between π and ρ. The resulting optimizationobjective, coined Domain Adaptation for Linear Classifiers

14

(DALC), trades off the disagreement of the classifier on thetarget samples, the classification error on the source samplesand the squared norm of the classifier parameters:

θ=argminθ

τ1

m∑

j=1

Φ( zjθ

‖zj‖

)

+τ2

n∑

i=1

Φ2(yixiθ

‖xi‖

)

+‖θ‖2, (29)

where Φ is a probit loss function and Φ(x) = 2Φ(−x)Φ(x).τ1 and τ2 are trade-off parameters based on normalizationfactors. DALC resembles a standard linear classifier in thatit minimizes an average loss. However, it includes an addi-tional term that penalizes predictions on the target samplesthat are far removed from zero.

6 DISCUSSION

We discuss insights, limitations and points of attention.

6.1 Assumptions, tests & no-free-lunch

The problem with assumptions on the relationship betweendomains, such as covariate or prior shift, is that they cannotbe verified without target labels. In order to check that theposteriors in each domain are actually equal, one wouldneed labeled data from each domain. The inability to checkfor the validity of assumptions means that it is impossible topredict how well a method will perform on a given data set.For any adaptive classifier, it is possible to find a case wherelearning fails dramatically. Clearly, in domain adaptationthere is no free lunch either [203].

Advising practitioners on classifier selection is thereforedifficult. Ideally, there would be some hypothesis test thatgives an impression of how likely an assumption is to bevalid for a given problem setting. A number of such testshave been developed for domain adaptation procedures.Firstly, a test exists for regression under sample selec-tion bias [204]. Secondly, the necessity of data importance-weighting can be tested by comparing the adversarial lossof the classifier parameters obtained by training with ver-sus without data importance-weights, where the adversar-ial loss consists of maximizing loss given fixed weights[15], [90]. The larger the difference, the more the model ismis-specified and the larger the necessity for importance-weighting. Thirdly, testing for label shift can be donethrough a two-sample test comparing the empirical sourceprior and the empirical predicted target prior, as it can beshown that, under weak assumptions, pT (y) = pS(y) if andonly if pT (y) = pS(y) [8]. Lastly, in some cases, a largeempirical discrepancy can indicate that no method has thecapacity to generalize well to the target domain [7].

Given the difficulty of domain adaptation [205], thesetests are crucial to developing practical solutions. Withoutsome indication that your proposed method relies on a validassumption, the risk of negative transfer is very real [180].

6.2 Insights & interpretability

We argue that interpretability, in the sense that one caneasily inspect the inner mechanics of a method and gainsome intuition on why it is likely to succeed or fail, is animportant property. Interpretations lead to novel insights,which deepen our understanding of domain adaptation.

In models with explicit descriptions of transfer, typicalsuccess and failure cases can be compared and interpreted.For example, in sample-based methods, the importanceweights describe transfer because they indicate how muchcorrection is necessary. In successful adaptation, the weightstend to vary smoothly around the value one. In one set offailure cases, we observe a bimodal distribution with a smallset of weights having large values and the remainder beingclose to zero. Upon reflection, we realize that the adaptiveclassifier under-performs because the effective sample sizedrops for such a bi-modal weight distribution. Similarly, intransformation-based methods, the transformation functionitself describes transfer. In success cases, there are fewtransformations that fit well. If many transformations fit,such as when the class-conditional distributions overlapsubstantially in each domain, then chances are low that theadaptive classifier recovers the correct one. Lastly, some ofthe inference-based methods fail when facing problems withsubstantial target probability mass outside the support ofthe source data distribution. This can be observed throughthe adversarial loss becoming large or through the disagree-ment of the learned prior with the posterior relative to anuninformative prior.

6.3 Shrinking search space

In a number of methods the source domain is considereda means of narrowing the search space. For example, inempirical Bayes approaches, the informed prior will assignlow probabilities to parameters that would not be suitablefor the given task. Hence, the search space is effectivelysmaller than for the uninformative prior. Another exampleis fine-tuning, where a neural network is first trained on alarge generic data set and then trained on the smaller dataset of the problem of interest. Starting with more plausibleparameters than random ones means that it will avoid manyimplausible parameters [206].

Ignoring parts of parameter space is an intuitive notionof how the source domain assists the learning process.This interpretation shows an interesting link to naturalintelligence, where agents use knowledge from previoustasks to complete new tasks [207]. But ignoring parameterscan be dangerous as well. If parameters are ignored thatare actually useful for the target domain, then the sourcedomain effectively interferes with learning.

6.4 Multi-site studies

It is notoriously difficult to integrate data from researchgroups working in the same field. The argument is thatanother group uses different experimental protocols or mea-suring devices, or is located in a different environment,and that their data is therefore not compatible [208]. Forexample, in biostatistics, gene expression micro-array datacan exhibit batch effects [209]. These can be caused by the am-plification reagent, time of day, or even atmospheric ozonelevel [210]. In some data sets, batch effects are actually themost dominant source of variation and are easily identifiedby clustering algorithms [209].

Domain adaptation methods are useful tools for this typeof problem. Additional information such as local weather,

15

laboratory conditions, or experimental protocol, can be ex-ploited to correct for the data shift. Considering the financialcosts of some types of experiments, the ability to removebatch effects and integrate data sets from multiple researchcenters would be valuable. Proper cross-domain generaliza-tion techniques could indirectly increase the incentive formaking data sets publicly available.

6.5 Causality

Domain shifts can be viewed as the effect of one or morevariables on the data-generating process [87], [115]. Knowl-edge of how those variables affect features can clarifywhether assumptions, such as covariate shift or prior shift,are reasonable to make [83], [211]. Given the causal struc-ture, one could even relax the covariate shift assumption tohold only for a subset of the features [211]. Furthermore,taking the causal relationships between variables into ac-count can lead to better predictions of invariant conditionaldistributions [115], [212]. For example, recent work con-siders the domains to be context variables outside of thesystem of interest [212]. By making certain assumptions onthe confounding nature of these context variables, one couldidentify a set of features that is invariant under transfer fromthe source to target domain. These assumptions on contextvariables can often be linked to real-world environmentalvariables, such as the organism in a genetics study or theenvironment in a data-collecting research institute [212],[213]. The main limitation of causal inference procedures isthat they, currently, do not scale beyond dozens of variables.As the causal structure resolves ambiguities on the typeof domain shift occurring in any given problem, advancesin causal discovery could lead to automatic adaptationstrategy selection and eventually, to much more practicaldomain-adaptive systems.

6.6 Limitations

Methods have general limitations. Firstly, importance-weighting suffers from the curse of dimensionality. Theweights are supposed to align distributions, but measuringalignment in high-dimensional settings is tricky. Secondly,a variety of methods require that the support of the tar-get distribution is contained within the support of thesource distribution, where ”support” should be interpretedas the collection of areas of high probability mass. If thisis not the case, then pathological solutions such as highweight variance or worse-than-uninformative priors canoccur. Thirdly, unfortunately, combining transformationsthat bring the supports of the distributions closer togetherwith methods that involve support requirements, is notstraightforward: subjecting the source and target domainto different transformations can break the equivalence ofposterior or conditional distributions. Fourthly, in manypractical high-dimensional settings, there are a multitude oftransformations that would align the data distributions butnot the posterior or class-conditional distributions. Heuris-tics and knowledge of the particular application of interestis required to pick an appropriate transformation.

Our survey is limited in that it is not exhaustive. This isdue to two reasons: firstly, we selected papers with the goalof presenting what we believe are original ideas, thereby

disregarding special cases and particular applications. Sec-ondly, inconsistent terminology in the literature makes ithard to find papers that adhere to our problem definitionbut follow alternative naming conventions. Some such arti-cles have been found by chance and are included here, suchas [148], but we cannot exclude the possibility that somehave escaped our search criteria. Nonetheless, we do believeto have presented the reader a thorough summary.

6.7 Terminology

The literature contains a wide variety of terminology de-scribing the current problem setting. Our setting ”domainadaptation without target labels” has been referred to as”unsupervised domain adaptation”, ”transductive transferlearning”, and terms involving sampling bias, correctingdistributional shifts, robustness to data shifts or out-of-sample / out-of-distribution generalization. We tried to beconsistent in our terminology, but the reader should beaware that, in general, the literature is not.

7 CONCLUSION

We reviewed work in domain adaptation to answer thequestion: how can a classifier learn from a source andgeneralize to a target domain? We have made a coarse-level categorization into three categories: those that operateon individual observations (sample-based), those that op-erate on the representation of sets of observations (feature-based) and those that operate on the parameter estimator(inference-based). On a finer level, we can split sample-based methods into data importance-weighting, based onassuming covariate shift, and class importance-weighting,based on assuming prior shift. A variety of weight es-timators has been proposed, suitable to diverse problemsettings and data types. Similarly, we can split feature-based methods into subspace mappings, optimal transport,domain-invariant spaces, deep domain adaptation and cor-respondence learning. Subspace mappings focus on trans-formations from source to target based on subsets of featurespace. Optimal transport also focuses on transformationsfrom source to target, but does so on the level of probabilitydistributions. Domain-invariant representations, where bothsource and target data are mapped to a new space, canbe learned. Deep domain adaptation is the neural networkequivalent of that. Correspondence learning constructs com-mon features through feature inter-dependencies in eachdomain. As can be imagined, these methods require high-dimensional data, such as images or text. Subspace projec-tions are more common to computer vision while learningdomain-invariant representations is popular in both theimage and the natural language processing community.Inference-based methods is a diverse category consisting ofalgorithmic robustness, minimax estimators, self-learning,empirical Bayes and PAC-Bayes. Robust algorithms aim topartition feature space such that the loss is small when re-moving a training sample and minimax estimators optimizefor worst-case shifts under constraints imposed on the ad-versary. Self-learning procedures provisionally label targetsamples and include the pseudo-labeled samples in trainingthe target classifier. In empirical Bayes, the prior distribution

16

is fit to the source data and in PAC-Bayes, one considersdisagreement between hypotheses drawn from the posteriorover the hypothesis space while ignoring source sampleslying outside the support of the target data distribution.

Furthermore, we find that assumptions are a necessarycomponent to domain adaptation without target labels andthat these strongly influence when a particular method willsucceed or fail. It is crucial to develop hypothesis testsfor the validity of such assumptions. Moreover, to developdomain adaptation methods, it is important to study in-terpretable procedures. Comparing explicit descriptions oftransfer between success and failure cases can producenovel insights. Additionally, domain adaptation is not onlyrelevant to many scientific and engineering disciplines, it isof value to integrating multi-site data sets and to the com-putational expense of existing algorithms. Automaticallydiscovering causal structure in data is an exciting possibilitythat could resolve many ambiguities on adaptive classifierselection. In general, domain adaptation is an importantproblem, as it explores outside the standard assumptionsof independently and identically distributed data. It shouldbe studied in further detail.

REFERENCES

[1] J. J. Heckman, “Sample selection bias as a specification error (withan application to the estimation of labor supply functions),” 1977.

[2] J. Heckman, “Varieties of selection bias,” American EconomicReview, vol. 80, no. 2, pp. 313–318, 1990.

[3] W. G. Cochran and D. B. Rubin, “Controlling bias in observa-tional studies: a review,” Sankhya: Indian Journal of Statistics, SeriesA, pp. 417–446, 1973.

[4] R. J. Little and D. B. Rubin, Statistical analysis with missing data.John Wiley & Sons, 2014.

[5] G. W. Imbens and D. B. Rubin, Causal inference in statistics, social,and biomedical sciences. Cambridge University Press, 2015.

[6] B. Zadrozny, “Learning and evaluating classifiers under sampleselection bias,” in International Conference on Machine Learning,2004, p. 114.

[7] C. Cortes and M. Mohri, “Domain adaptation and sample biascorrection theory and algorithm for regression,” Theoretical Com-puter Science, vol. 519, pp. 103–126, 2014.

[8] Z. Lipton, Y.-X. Wang, and A. Smola, “Detecting and correctingfor label shift with black box predictors,” in International Confer-ence on Machine Learning, 2018, pp. 3128–3136.

[9] B. Gong, Y. Shi, F. Sha, and K. Grauman, “Geodesic flow kernelfor unsupervised domain adaptation,” in Conference on ComputerVision and Pattern Recognition, 2012, pp. 2066–2073.

[10] B. Fernando, A. Habrard, M. Sebban, and T. Tuytelaars, “Unsu-pervised visual domain adaptation using subspace alignment,”in International Conference on Computer Vision, 2013, pp. 2960–2967.

[11] N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy, “Optimaltransport for domain adaptation,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 39, no. 9, pp. 1853–1865,2016.

[12] S. Pan, I. Tsang, J. Kwok, and Q. Yang, “Domain adaptation viatransfer component analysis,” Neural Networks, vol. 22, no. 2, pp.199–210, 2011.

[13] J. Blitzer, R. McDonald, and F. Pereira, “Domain adaptationwith structural correspondence learning,” in Empirical Methodsin Natural Language Processing, 2006, pp. 120–128.

[14] Y. Mansour and M. Schain, “Robust domain adaptation,” Annalsof Mathematics and Artificial Intelligence, vol. 71, no. 4, pp. 365–380,2014.

[15] J. Wen, C.-N. Yu, and R. Greiner, “Robust learning under uncer-tain test distributions: relating covariate shift to model misspeci-fication,” in International Conference on Machine Learning, 2014, pp.631–639.

[16] L. Bruzzone and M. Marconcini, “Domain adaptation problems:A DASVM classification technique and a circular validationstrategy,” IEEE Transactions on Pattern Analysis and Machine In-telligence, vol. 32, no. 5, pp. 770–787, 2010.

[17] R. Raina, A. Y. Ng, and D. Koller, “Constructing informativepriors using transfer learning,” in International Conference onMachine Learning, 2006, pp. 713–720.

[18] P. Germain, A. Lacasse, F. Laviolette, and M. Marchand, “APAC-Bayes risk bound for general loss functions,” in Advancesin Neural Information Processing Systems, 2007, pp. 449–456.

[19] Y. Mansour, M. Mohri, and A. Rostamizadeh, “Domain adap-tation with multiple sources,” in Advances in Neural InformationProcessing Systems, 2009, pp. 1041–1048.

[20] ——, “Multiple source adaptation and the Renyi divergence,” inConference on Uncertainty in Artificial Intelligence, 2009, pp. 367–374.

[21] S. Sun, H. Shi, and Y. Wu, “A survey of multi-source domainadaptation,” Information Fusion, vol. 24, pp. 84–92, 2015.

[22] R. K. Ando and T. Zhang, “A framework for learning predictivestructures from multiple tasks and unlabeled data,” Journal ofMachine Learning Research, vol. 6, no. Nov, pp. 1817–1853, 2005.

[23] J. Donahue, J. Hoffman, E. Rodner, K. Saenko, and T. Darrell,“Semi-supervised domain adaptation with instance constraints,”in Conference on Computer Vision and Pattern Recognition, 2013, pp.668–675.

[24] V. M. Patel, R. Gopalan, R. Li, and R. Chellappa, “Visual domainadaptation: a survey of recent advances,” IEEE Signal ProcessingMagazine, vol. 32, no. 3, pp. 53–69, 2015.

[25] G. Csurka, “A comprehensive survey on domain adaptation forvisual applications,” in Domain Adaptation in Computer VisionApplications. Springer, 2017, pp. 1–35.

[26] M. Wang and W. Deng, “Deep visual domain adaptation: Asurvey,” Neurocomputing, vol. 312, pp. 135–153, 2018.

[27] G. Schweikert, G. Ratsch, C. Widmer, and B. Scholkopf, “Anempirical analysis of domain adaptation algorithms for genomicsequence analysis,” in Advances in Neural Information ProcessingSystems, 2009, pp. 1433–1440.

[28] V. Cheplygina, M. de Bruijne, and J. P. Pluim, “Not-so-supervised:a survey of semi-supervised, multi-instance, and transfer learn-ing in medical image analysis,” Medical Image Analysis, 2019.

[29] J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D.Lawrence, Dataset shift in machine learning. MIT Press, 2009.

[30] J. G. Moreno-Torres, T. Raeder, R. Alaiz-RodrıGuez, N. V. Chawla,and F. Herrera, “A unifying view on dataset shift in classifica-tion,” Pattern Recognition, vol. 45, no. 1, pp. 521–530, 2012.

[31] A. Margolis, “A literature review of domain adaptation withunlabeled data,” Rapport Technique, University of Washington, p. 35,2011.

[32] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEETransactions on Knowledge and Data Engineering, vol. 22, no. 10,pp. 1345–1359, 2010.

[33] A. Arnold, R. Nallapati, and W. W. Cohen, “A comparative studyof methods for transductive transfer learning,” in InternationalConference on Data Mining Workshops, 2007, pp. 77–82.

[34] J. W. Tucker, “Selection bias and econometric remedies in ac-counting and finance research,” Journal of Accounting Literature,vol. 29, pp. 31–57, 2010.

[35] R. Gronau, “Wage comparisons–a selectivity bias,” Journal ofPolitical Economy, vol. 82, no. 6, pp. 1119–1143, 1974.

[36] F. Vella, “Estimating models with sample selection bias: a sur-vey,” Journal of Human Resources, pp. 127–169, 1998.

[37] M. A. Hernan and J. M. Robins, “Estimating causal effectsfrom epidemiological data,” Journal of Epidemiology & CommunityHealth, vol. 60, no. 7, pp. 578–586, 2006.

[38] C. M. Stonnington, G. Tan, S. Kloppel, C. Chu, B. Draganski,C. R. Jack Jr, K. Chen, J. Ashburner, and R. S. Frackowiak,“Interpreting scan data acquired from multiple scanners: a studywith Alzheimer’s disease,” Neuroimage, vol. 39, no. 3, pp. 1180–1185, 2008.

[39] V. Cheplygina, I. P. Pena, J. H. Pedersen, D. A. Lynch, L. Sørensen,and M. de Bruijne, “Transfer learning for multicenter classi-fication of chronic obstructive pulmonary disease,” Journal ofBiomedical and Health Informatics, vol. 22, no. 5, pp. 1486–1496,2017.

[40] A. van Opbroek, M. Ikram, M. Vernooij, and M. De Bruijne,“Transfer learning improves supervised image segmentationacross imaging protocols,” IEEE Transactions on Medical Imaging,vol. 34, no. 5, pp. 1018–1030, 2015.

[41] K. Kamnitsas, C. Baumgartner, C. Ledig, V. Newcombe, J. Simp-son, A. Kane, D. Menon, A. Nori, A. Criminisi, D. Rueckert et al.,“Unsupervised domain adaptation in brain lesion segmentation

17

with adversarial networks,” in International Conference on Informa-tion Processing in Medical Imaging. Springer, 2017, pp. 597–609.

[42] W. M. Kouw, S. N. Ørting, J. Petersen, K. S. Pedersen, andM. de Bruijne, “A cross-center smoothness prior for variationalBayesian brain tissue segmentation,” in Conference on InformationProcessing in Medical Imaging, vol. 26, 2019, pp. 360 – 371.

[43] A. Torralba and A. A. Efros, “Unbiased look at dataset bias,” inConference on Computer Vision and Pattern Recognition, 2011, pp.1521–1528.

[44] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual cat-egory models to new domains,” European Conference on ComputerVision, pp. 213–226, 2010.

[45] L. Duan, D. Xu, I. W.-H. Tsang, and J. Luo, “Visual event recog-nition in videos by learning from web data,” IEEE Transactions onPattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1667–1680, 2011.

[46] A. Farhadi and M. K. Tabrizi, “Learning to recognize activitiesfrom the wrong view point,” in European Conference on ComputerVision, 2008, pp. 154–166.

[47] T. Van Kasteren, G. Englebienne, and B. J. Krose, “Transferringknowledge of activity recognition across sensor networks.” inPervasive Computing, vol. 10. Springer, 2010, pp. 283–300.

[48] H. Hachiya, M. Sugiyama, and N. Ueda, “Importance-weightedleast-squares probabilistic classifier for covariate shift adaptationwith application to human activity recognition,” Neurocomputing,vol. 80, pp. 93–101, 2012.

[49] E. Gedik and H. Hung, “Personalised models for speech detec-tion from body movements using transductive parameter trans-fer,” Personal and Ubiquitous Computing, pp. 1–15, 2017.

[50] L. Tai, G. Paolo, and M. Liu, “Virtual-to-real deep reinforcementlearning: Continuous control of mobile robots for mapless navi-gation,” in IEEE International Conference on Intelligent Robots andSystems, 2017, pp. 31–36.

[51] A. Marco, F. Berkenkamp, P. Hennig, A. P. Schoellig, A. Krause,S. Schaal, and S. Trimpe, “Virtual vs. real: Trading off simula-tions and physical experiments in reinforcement learning withBayesian optimization,” in International Conference on Robotics andAutomation, 2017, pp. 1557–1563.

[52] J. Hoffman, D. Wang, F. Yu, and T. Darrell, “FCNs in thewild: Pixel-level adversarial and constraint-based adaptation,”arXiv:1612.02649, 2016.

[53] M. Wulfmeier, A. Bewley, and I. Posner, “Addressing appearancechange in outdoor robotics with adversarial domain adaptation,”in International Conference on Intelligent Robots and Systems, 2017,pp. 1551–1558.

[54] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakr-ishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige et al., “Usingsimulation and domain adaptation to improve efficiency of deeprobotic grasping,” in International Conference on Robotics and Au-tomation, 2018, pp. 4243–4250.

[55] J. Deng, Z. Zhang, F. Eyben, and B. Schuller, “Autoencoder-basedunsupervised domain adaptation for speech emotion recogni-tion,” IEEE Signal Processing Letters, vol. 21, no. 9, pp. 1068–1072,2014.

[56] Q. Wang, W. Rao, S. Sun, L. Xie, E. S. Chng, and H. Li, “Un-supervised domain adaptation via domain adversarial trainingfor speaker recognition,” in International Conference on Acoustics,Speech and Signal Processing, 2018, pp. 4889–4893.

[57] V. Peddinti and P. Chintalapoodi, “Domain adaptation in senti-ment analysis of Twitter,” in AAAI Conference on Artificial Intelli-gence, 2011.

[58] J. Blitzer, M. Dredze, F. Pereira et al., “Biographies, bollywood,boom-boxes and blenders: Domain adaptation for sentimentclassification,” in Association for Computational Linguistics, vol. 7,2007, pp. 440–447.

[59] C. Widmer, N. C. Toussaint, Y. Altun, O. Kohlbacher, andG. Ratsch, “Novel machine learning methods for MHC Class Ibinding prediction,” in International Conference on Pattern Recog-nition in Bioinformatics. Springer, 2010, pp. 98–109.

[60] S. Mei, W. Fei, and S. Zhou, “Gene ontology based transferlearning for protein subcellular localization,” BMC Bioinformatics,vol. 12, no. 1, p. 1, 2011.

[61] T. Kato, K. Okada, H. Kashima, and M. Sugiyama, “A transferlearning approach and selective integration of multiple typesof assays for biological network inference,” in ComputationalKnowledge Discovery for Bioinformatics Research, 2012, pp. 188–202.

[62] Q. Xu and Q. Yang, “A survey of transfer and multitask learningin bioinformatics,” Journal of Computing Science and Engineering,vol. 5, no. 3, pp. 257–268, 2011.

[63] F. Gieseke, K. L. Polsterer, A. Thom, P. Zinn, D. Bomanns, R.-J.Dettmar, O. Kramer, and J. Vahrenhold, “Detecting quasars inlarge-scale astronomical surveys,” in International Conference onMachine Learning and Applications, 2010, pp. 352–357.

[64] J. Kremer, F. Gieseke, K. S. Pedersen, and C. Igel, “Nearestneighbor density ratio estimation for large-scale applications inastronomy,” Astronomy and Computing, vol. 12, pp. 67–72, 2015.

[65] R. Vilalta, K. D. Gupta, and L. Macri, “A machine learningapproach to Cepheid variable star classification using data align-ment and maximum likelihood,” Astronomy and Computing, vol. 2,pp. 46–53, 2013.

[66] R. Izbicki, A. B. Lee, P. E. Freeman et al., “Photo-z estimation: Anexample of nonparametric conditional density estimation underselection bias,” The Annals of Applied Statistics, vol. 11, no. 2, pp.698–724, 2017.

[67] M. Hardt, E. Price, N. Srebro et al., “Equality of opportunity insupervised learning,” in Advances in Neural Information ProcessingSystems, 2016, pp. 3315–3323.

[68] S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choud-hary, E. P. Hamilton, and D. Roth, “A comparative study offairness-enhancing interventions in machine learning,” in Con-ference on Fairness, Accountability, and Transparency, 2019, pp. 329–338.

[69] N. Quadrianto and V. Sharmanska, “Recycling privileged learn-ing and distribution matching for fairness,” in Advances in NeuralInformation Processing Systems, 2017, pp. 677–688.

[70] D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning adver-sarially fair and transferable representations,” arXiv:1802.06309,2018.

[71] M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations ofMachine Learning. MIT Press, 2012.

[72] M. Loog, “Supervised classification: Quite a brief overview,” inMachine Learning Techniques for Space Weather, 2018, pp. 113–145.

[73] D. Dua and C. Graff, “UCI machine learning repository,” 2017.[Online]. Available: http://archive.ics.uci.edu/ml

[74] M. H. Mahmud, “On universal transfer learning,” TheoreticalComputer Science, vol. 410, no. 19, pp. 1826–1846, 2009.

[75] T. M. Cover and J. A. Thomas, Elements of information theory. JohnWiley & Sons, 2012.

[76] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, andJ. W. Vaughan, “A theory of learning from different domains,”Machine Learning, vol. 79, no. 1-2, pp. 151–175, 2010.

[77] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, “Analysisof representations for domain adaptation,” in Advances in NeuralInformation Processing Systems, 2007, pp. 137–144.

[78] T. Van Erven and P. Harremos, “Renyi divergence and Kullback-Leibler divergence,” IEEE Transactions on Information Theory,vol. 60, no. 7, pp. 3797–3820, 2014.

[79] C. Cortes, Y. Mansour, and M. Mohri, “Learning bounds for im-portance weighting,” in Advances in Neural Information ProcessingSystems, 2010, pp. 442–450.

[80] J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman,“Learning bounds for domain adaptation,” in Advances in NeuralInformation Processing Systems, 2008, pp. 129–136.

[81] Y. Mansour, M. Mohri, and A. Rostamizadeh, “Domain adapta-tion: Learning bounds and algorithms,” in Conference on LearningTheory, 2009.

[82] K. Azizzadenesheli, A. Liu, F. Yang, and A. Anandkumar, “Reg-ularized learning for domain adaptation under label shifts,” inInternational Conference on Learning Representations, 2019.

[83] M. Gong, K. Zhang, T. Liu, D. Tao, C. Glymour, and B. Scholkopf,“Domain adaptation with conditional transferable components,”in International Conference on Machine Learning, 2016, pp. 2839–2848.

[84] P. Germain, A. Habrard, F. Laviolette, and E. Morvant, “A newPAC-Bayesian perspective on domain adaptation,” in Interna-tional Conference on Machine Learning, 2016, pp. 859–868.

[85] A. J. Storkey, “When training and test sets are different: charac-terising learning transfer,” in In Dataset Shift in Machine Learning,2009, pp. 3–28.

[86] G. Widmer and M. Kubat, “Learning in the presence of conceptdrift and hidden contexts,” Machine Learning, vol. 23, no. 1, pp.69–101, 1996.

18

[87] B. Scholkopf, D. Janzing, J. Peters, E. Sgouritsa, K. Zhang, andJ. Mooij, “On causal and anti-causal learning,” in InternationalConference on Machine Learning, 2012, pp. 1255–1262.

[88] J. Jiang and C. Zhai, “Instance weighting for domain adaptationin NLP,” in Annual Meeting of the Association of ComputationalLinguistics, 2007, pp. 264–271.

[89] M. Vidyasagar, A theory of learning and generalization. Springer-Verlag New York, Inc., 2002.

[90] H. White, “Consequences and detection of misspecified nonlinearregression models,” Journal of the American Statistical Association,vol. 76, no. 374, pp. 419–433, 1981.

[91] H. Shimodaira, “Improving predictive inference under covariateshift by weighting the log-likelihood function,” Journal of Statisti-cal Planning and Inference, vol. 90, no. 2, pp. 227–244, 2000.

[92] C. Cortes, M. Mohri, M. Riley, and A. Rostamizadeh, “Sampleselection bias correction theory,” in Conference on AlgorithmicLearning Theory, 2008, pp. 38–53.

[93] M. Sugiyama and K.-R. Muller, “Input-dependent estimation ofgeneralization error under covariate shift,” Statistics & Decisions,vol. 23, no. 4/2005, pp. 249–279, 2005.

[94] Y.-L. Yu and C. Szepesvari, “Analysis of kernel mean matchingunder covariate shift,” in International Conference on MachineLearning, 2012, pp. 1147–1154.

[95] M. Baktashmotlagh, M. Harandi, B. Lovell, and M. Salzmann,“Domain adaptation on the statistical manifold,” in Conference onComputer Vision and Pattern Recognition, 2014, pp. 2481–2488.

[96] M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, andM. Kawanabe, “Direct importance estimation with model selec-tion and its application to covariate shift adaptation,” in Advancesin Neural Information Processing Systems, 2008, pp. 1433–1440.

[97] M. Sugiyama, T. Suzuki, and T. Kanamori, Density ratio estimationin machine learning. Cambridge University Press, 2012.

[98] Y. Tsuboi, H. Kashima, S. Hido, S. Bickel, and M. Sugiyama,“Direct density ratio estimation for large-scale covariate shiftadaptation,” Journal of Information Processing, vol. 17, pp. 138–155,2009.

[99] T. Kanamori, S. Hido, and M. Sugiyama, “A least-squares ap-proach to direct importance estimation,” Journal of Machine Learn-ing Research, vol. 10, pp. 1391–1445, 2009.

[100] K. M. Borgwardt, A. Gretton, M. J. Rasch, H.-P. Kriegel,B. Scholkopf, and A. J. Smola, “Integrating structured biologi-cal data by kernel maximum mean discrepancy,” Bioinformatics,vol. 22, no. 14, pp. e49–e57, 2006.

[101] A. Gretton, K. M. Borgwardt, M. Rasch, B. Scholkopf, and A. J.Smola, “A kernel method for the two-sample-problem,” in Ad-vances in Neural Information Processing Systems, 2007, pp. 513–520.

[102] J. Huang, A. Gretton, K. M. Borgwardt, B. Scholkopf, and A. J.Smola, “Correcting sample selection bias by unlabeled data,” inAdvances in Neural Information Processing Systems, 2007, pp. 601–608.

[103] A. Gretton, A. Smola, J. Huang, M. Schmittfull, K. Borgwardt,and B. Scholkopf, “Covariate shift by kernel mean matching,”in Dataset Shift in Machine Learning, Q. Candela, M. Sugiyama,A. Schwaighofer, and N. Lawrence, Eds. MIT Press, 2009, pp.131–160.

[104] M. Sugiyama and K.-R. Muller, “Model selection under covariateshift,” in International Conference on Artificial Neural Networks,2005, pp. 235–240.

[105] M. Sugiyama, M. Krauledat, and K.-R. Muller, “Covariate shiftadaptation by importance weighted cross validation,” Journal ofMachine Learning Research, vol. 8, pp. 985–1005, 2007.

[106] T. Kanamori, T. Suzuki, and M. Sugiyama, “Statistical analysisof kernel-based least-squares density-ratio estimation,” MachineLearning, vol. 86, no. 3, pp. 335–367, 2012.

[107] M. Sugiyama, T. Suzuki, and T. Kanamori, “Density-ratio match-ing under the Bregman divergence: a unified framework ofdensity-ratio estimation,” Annals of the Institute of Statistical Math-ematics, vol. 64, no. 5, pp. 1009–1044, 2012.

[108] E. G. Miller, “A new class of entropy estimators for multi-dimensional densities,” in International Conference on Acoustics,Speech, and Signal Processing, vol. 3, 2003, pp. III–297.

[109] M. Loog, “Nearest neighbor-based importance weighting,” inInternational Workshop on Machine Learning for Signal Processing,2012, pp. 1–6.

[110] S. Bickel, M. Bruckner, and T. Scheffer, “Discriminative learningfor differing training and test distributions,” in International Con-ference on Machine Learning, 2007, pp. 81–88.

[111] ——, “Discriminative learning under covariate shift,” Journal ofMachine Learning Research, vol. 10, pp. 2137–2155, 2009.

[112] C. Elkan, “The foundations of cost-sensitive learning,” in Inter-national Joint Conference on Artificial Intelligence, vol. 17, 2001, pp.973–978.

[113] G. Jacobusse and C. Veenman, “On selection bias with imbal-anced classes,” in International Conference on Discovery Science.Springer, 2016, pp. 325–340.

[114] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer,“Smote: synthetic minority over-sampling technique,” Journal ofArtificial Intelligence Research, vol. 16, pp. 321–357, 2002.

[115] K. Zhang, B. Scholkopf, K. Muandet, and Z. Wang, “Domainadaptation under target and conditional shift,” in InternationalConference on Machine Learning, 2013, pp. 819–827.

[116] R. Gopalan, R. Li, V. M. Patel, R. Chellappa et al., “Domainadaptation for visual recognition,” Foundations and Trends R© inComputer Graphics and Vision, vol. 8, no. 4, pp. 285–378, 2015.

[117] M. Long, J. Wang, J. Sun, and S. Y. Philip, “Domain invarianttransfer kernel learning,” IEEE Transactions on Knowledge and DataEngineering, vol. 27, no. 6, pp. 1519–1532, 2015.

[118] E. Tzeng, J. Hoffman, T. Darrell, and K. Saenko, “Simultaneousdeep transfer across domains and tasks,” in International Confer-ence on Computer Vision, 2015, pp. 4068–4076.

[119] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle,F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” Journal of Machine Learn-ing Research, vol. 17, no. 59, pp. 1–35, 2016.

[120] R. Aljundi, R. Emonet, D. Muselet, and M. Sebban, “Landmarks-based kernelized subspace alignment for unsupervised domainadaptation,” in Conference on Computer Vision and Pattern Recogni-tion, 2015, pp. 56–63.

[121] B. Sun and K. Saenko, “Subspace distribution alignment forunsupervised domain adaptation,” in British Machine Vision Con-ference, 2015, pp. 24–1.

[122] B. Fernando, T. Tommasi, and T. Tuytelaars, “Joint cross-domainclassification and subspace learning for unsupervised adapta-tion,” Pattern Recognition Letters, vol. 65, pp. 60–66, 2015.

[123] D. Das and C. George Lee, “Unsupervised domain adaptationusing regularized hyper-graph matching,” in International Con-ference on Image Processing, 10 2018, pp. 3758–3762.

[124] D. Das and C. G. Lee, “Graph matching and pseudo-label guideddeep unsupervised domain adaptation,” in International Confer-ence on Artificial Neural Networks, 2018, pp. 342–352.

[125] ——, “Sample-to-sample correspondence for unsupervised do-main adaptation,” Engineering Applications of Artificial Intelligence,vol. 73, pp. 80–91, 2018.

[126] B. Gong, K. Grauman, and F. Sha, “Reshaping visual datasets fordomain adaptation,” in Advances in Neural Information ProcessingSystems, 2013, pp. 1286–1294.

[127] J. Hoffman, T. Darrell, and K. Saenko, “Continuous manifoldbased adaptation for evolving visual domains,” in Conference onComputer Vision and Pattern Recognition, 2014, pp. 867–874.

[128] R. Gopalan, R. Li, and R. Chellappa, “Domain adaptation forobject recognition: an unsupervised approach,” in InternationalConference on Computer Vision, 2011, pp. 999–1006.

[129] M. Baktashmotlagh, M. Harandi, B. Lovell, and M. Salzmann,“Unsupervised domain adaptation by domain invariant projec-tion,” in International Conference on Computer Vision, 2013, pp. 769–776.

[130] R. Caseiro, J. F. Henriques, P. Martins, and J. Batista, “Beyondthe shortest path: unsupervised domain adaptation by samplingsubspaces along the spline flow,” in Conference on Computer Visionand Pattern Recognition, 2015, pp. 3846–3854.

[131] B. Gong, K. Grauman, and F. Sha, “Connecting the dots withlandmarks: discriminatively learning domain-invariant featuresfor unsupervised domain adaptation,” in International Conferenceon Machine Learning, 2013, pp. 222–230.

[132] R. Gopalan, R. Li, and R. Chellappa, “Unsupervised adaptationacross domain shifts by generating intermediate data repre-sentations,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 36, no. 11, pp. 2288–2302, 2014.

[133] J. Zheng, M.-Y. Liu, R. Chellappa, and P. J. Phillips, “A Grass-mann manifold-based domain adaptation approach,” in Interna-tional Conference on Pattern Recognition, 2012, pp. 2095–2099.

[134] G. Ditzler and R. Polikar, “Hellinger distance based drift de-tection for nonstationary environments,” in IEEE Symposium on

19

Computational Intelligence in Dynamic and Uncertain Environments,2011, pp. 41–48.

[135] M. Baktashmotlagh, M. Harandi, and M. Salzmann,“Distribution-matching embedding for visual domainadaptation,” Journal of Machine Learning Research, vol. 17,no. 1, pp. 3760–3789, 2016.

[136] W. M. Kouw, L. J. Van Der Maaten, J. H. Krijthe, and M. Loog,“Feature-level domain adaptation,” Journal of Machine LearningResearch, vol. 17, no. 171, pp. 1–32, 2016.

[137] N. Courty, R. Flamary, A. Habrard, and A. Rakotomamonjy,“Joint distribution optimal transportation for domain adapta-tion,” in Advances in Neural Information Processing Systems, 2017,pp. 3730–3739.

[138] I. Redko, A. Habrard, and M. Sebban, “Theoretical analysisof domain adaptation with optimal transport,” in Joint Euro-pean Conference on Machine Learning and Knowledge Discovery inDatabases, 2017, pp. 737–753.

[139] B. Bhushan Damodaran, B. Kellenberger, R. Flamary, D. Tuia, andN. Courty, “DeepJDOT: Deep joint distribution optimal transportfor unsupervised domain adaptation,” in European Conference onComputer Vision, 2018, pp. 447–463.

[140] Q. Chen, Y. Liu, Z. Wang, I. Wassell, and K. Chetty, “Re-weightedadversarial adaptation network for unsupervised domain adap-tation,” in Conference on Computer Vision and Pattern Recognition,2018, pp. 7976–7985.

[141] T. Le, K. Nguyen, and D. Phung, “Theoretical perspective of deepdomain adaptation,” arXiv:1811.06199, 2018.

[142] S. J. Pan, X. Ni, J.-T. Sun, Q. Yang, and Z. Chen, “Cross-domainsentiment classification via spectral feature alignment,” in Inter-national Conference on World wide web, 2010, pp. 751–760.

[143] S. J. Pan, J. T. Kwok, and Q. Yang, “Transfer learning via dimen-sionality reduction.” in AAAI Conference on Artificial Intelligence,vol. 8, 2008, pp. 677–682.

[144] B. Scholkopf, A. Smola, and K.-R. Muller, “Kernel principal com-ponent analysis,” in International Conference on Artificial NeuralNetworks. Springer, 1997, pp. 583–588.

[145] K. Muandet, D. Balduzzi, and B. Scholkopf, “Domain gener-alization via invariant feature representation,” in InternationalConference on Machine Learning, 2013, pp. 10–18.

[146] Q. Gu and J. Zhou, “Learning the shared subspace for multi-task clustering and transductive transfer classification,” in Inter-national Conference on Data Mining, 2009, pp. 159–168.

[147] Y. Shi and F. Sha, “Information-theoretical learning of discrimi-native clusters for unsupervised domain adaptation,” in Interna-tional Conference on Machine Learning, 2012, pp. 1275–1282.

[148] S. Si, D. Tao, and B. Geng, “Bregman divergence-based regu-larization for transfer subspace learning,” IEEE Transactions onKnowledge and Data Engineering, vol. 22, no. 7, pp. 929–942, 2010.

[149] I.-H. Jhuo, D. Liu, D. Lee, and S.-F. Chang, “Robust visualdomain adaptation with low-rank reconstruction,” in Conferenceon Computer Vision and Pattern Recognition, 2012, pp. 2168–2175.

[150] M. Shao, D. Kit, and Y. Fu, “Generalized transfer subspacelearning through low-rank constraint,” International Journal ofComputer Vision, vol. 109, no. 1-2, pp. 74–93, 2014.

[151] G. E. Hinton and R. S. Zemel, “Autoencoders, minimum descrip-tion length and Helmholtz free energy,” in Advances in NeuralInformation Processing Systems, 1994, pp. 3–10.

[152] Y. Bengio, A. Courville, and P. Vincent, “Representation learning:a review and new perspectives,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828,2013.

[153] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol,“Stacked denoising autoencoders: learning useful representationsin a deep network with a local denoising criterion,” Journal ofMachine Learning Research, vol. 11, no. Dec, pp. 3371–3408, 2010.

[154] X. Glorot, A. Bordes, and Y. Bengio, “Domain adaptation forlarge-scale sentiment classification: a deep learning approach,”in International Conference on Machine Learning, 2011, pp. 513–520.

[155] M. Chen, Z. Xu, F. Sha, and K. Weinberger, “Marginalized de-noising autoencoders for domain adaptation,” in InternationalConference on Machine Learning, 2012, pp. 767–774.

[156] D. Kifer, S. Ben-David, and J. Gehrke, “Detecting change in datastreams,” in International Conference on Very Large Data Bases, 2004,pp. 180–191.

[157] W. Zellinger, B. A. Moser, T. Grubinger, E. Lughofer,T. Natschlager, and S. Saminger-Platz, “Robust unsupervised

domain adaptation for neural networks via moment alignment,”Information Sciences, 2019.

[158] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distanceguided representation learning for domain adaptation,” in AAAIConference on Artificial Intelligence, 2018.

[159] W. Zellinger, T. Grubinger, E. Lughofer, T. Natschlager, andS. Saminger-Platz, “Central moment discrepancy (CMD) fordomain-invariant representation learning,” arXiv:1702.08811,2017.

[160] J. Lee, N. Charoenphakdee, S. Kuroki, and M. Sugiyama, “Do-main discrepancy measure using complex models in unsuper-vised domain adaptation,” arXiv:1901.10654, 2019.

[161] R. Shu, H. H. Bui, H. Narui, and S. Ermon, “A DIRT-T approachto unsupervised domain adaptation,” in International Conferenceon Learning Representations, 2018.

[162] B. Usman, K. Saenko, and B. Kulis, “Stable distribution align-ment using the dual of the adversarial distance,” in InternationalConference on Learning Representations Workshop, 2018.

[163] M. Long, H. Zhu, J. Wang, and M. I. Jordan, “Unsuperviseddomain adaptation with residual transfer networks,” in Advancesin Neural Information Processing Systems, 2016, pp. 136–144.

[164] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krish-nan, “Unsupervised pixel-level domain adaptation with genera-tive adversarial networks,” in Conference on Computer Vision andPattern Recognition, vol. 1, no. 2, 2017, p. 7.

[165] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarialdiscriminative domain adaptation,” in Conference on ComputerVision and Pattern Recognition, vol. 1, no. 2, 2017, p. 4.

[166] M. Long, Y. Cao, J. Wang, and M. Jordan, “Learning transferablefeatures with deep adaptation networks,” in International Confer-ence on Machine Learning, 2015, pp. 97–105.

[167] K. Bousmalis, G. Trigeorgis, N. Silberman, D. Krishnan, andD. Erhan, “Domain separation networks,” in Advances in NeuralInformation Processing Systems, 2016, pp. 343–351.

[168] S. Sun, B. Zhang, L. Xie, and Y. Zhang, “An unsupervised deepdomain adaptation approach for robust speech recognition,”Neurocomputing, vol. 257, pp. 79–87, 2017.

[169] J.-T. Huang, J. Li, D. Yu, L. Deng, and Y. Gong, “Cross-languageknowledge transfer using multilingual deep neural network withshared hidden layers,” in International Conference on Acoustics,Speech and Signal Processing, 2013, pp. 7304–7308.

[170] G. Louppe, M. Kagan, and K. Cranmer, “Learning to pivotwith adversarial networks,” in Advances in Neural InformationProcessing Systems, 2017, pp. 981–990.

[171] J. Blitzer, S. Kakade, and D. Foster, “Domain adaptation with cou-pled subspaces,” in International Conference on Artificial Intelligenceand Statistics, 2011, pp. 173–181.

[172] W. Li, L. Duan, D. Xu, and I. Tsang, “Learning with augmentedfeatures for supervised and semi-supervised heterogeneous do-main adaptation,” IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, vol. 36, no. 6, pp. 1134–1148, 2014.

[173] H. Xu and S. Mannor, “Robustness and generalization,” MachineLearning, vol. 86, no. 3, pp. 391–423, 2012.

[174] A. Habrard, J.-P. Peyrache, and M. Sebban, “A new boosting algo-rithm for provably accurate unsupervised domain adaptation,”Knowledge and Information Systems, vol. 47, no. 1, pp. 45–73, 2016.

[175] T. van Laarhoven and E. Marchiori, “Unsupervised domain adap-tation with random walks on target labelings,” arXiv:1706.05335,2017.

[176] A. Liu and B. Ziebart, “Robust classification under sample selec-tion bias,” in Advances in Neural Information Processing Systems,2014, pp. 37–45.

[177] X. Chen, M. Monfort, A. Liu, and B. D. Ziebart, “Robust covariateshift regression,” in Conference on Artificial Intelligence and Statis-tics, 2016, pp. 1270–1279.

[178] W. M. Kouw and M. Loog, “Target contrastive pessimistic dis-criminant analysis,” arXiv:1806.09463, 2018.

[179] M. T. Rosenstein, Z. Marx, L. P. Kaelbling, and T. G. Dietterich,“To transfer or not to transfer,” in NIPS 2005 Workshop on TransferLearning, vol. 898, 2005, pp. 1–4.

[180] Z. Wang, Z. Dai, B. Poczos, and J. Carbonell, “Characterizing andavoiding negative transfer,” in Conference on Computer Vision andPattern Recognition, 2019, pp. 11 293–11 302.

[181] W. Hu, G. Niu, I. Sato, and M. Sugiyama, “Does distributionallyrobust supervised learning give robust classifiers?” in Interna-tional Conference on Machine Learning, 2018, pp. 2034–2042.

20

[182] K. Saito, K. Watanabe, Y. Ushiku, and T. Harada, “Maximumclassifier discrepancy for unsupervised domain adaptation,” inConference on Computer Vision and Pattern Recognition, 2018, pp.3723–3732.

[183] Z. Sun, C. Wang, H. Wang, and J. Li, “Learn multiple-kernelSVMs for domain adaptation in hyperspectral data,” IEEE Geo-science and Remote Sensing Letters, vol. 10, no. 5, pp. 1224–1228,2013.

[184] J. Lee and M. Raginsky, “Minimax statistical learning and domainadaptation with Wasserstein distances,” arXiv:1705.07815, 2017.

[185] G. J. McLachlan, “Iterative reclassification procedure for con-structing an asymptotically optimal rule of allocation in dis-criminant analysis,” Journal of the American Statistical Association,vol. 70, no. 350, pp. 365–369, 1975.

[186] O. Perez and M. Sanchez-Montanes, “A new learning strategyfor classification problems with different training and test distri-butions,” in International Conference on Artificial Neural Networks.Springer, 2007, pp. 178–185.

[187] M. Chen, K. Q. Weinberger, and J. Blitzer, “Co-training fordomain adaptation,” in Advances in Neural Information ProcessingSystems, 2011, pp. 2456–2464.

[188] A. Habrard, J.-P. Peyrache, and M. Sebban, “Iterative self-labelingdomain adaptation for linear structured image classification,”International Journal on Artificial Intelligence Tools, vol. 22, no. 05,p. 1360005, 2013.

[189] C. M. Bishop, Pattern recognition and machine learning. Springer,2006.

[190] T. van Laarhoven and E. Marchiori, “Domain adaptation withrandomized expectation maximization,” arXiv:1803.07634, 2018.

[191] J. Wang, Y. Chen, S. Hao, W. Feng, and Z. Shen, “Balanceddistribution adaptation for transfer learning,” in InternationalConference on Data Mining, 2017, pp. 1129–1134.

[192] E. Morvant, “Domain adaptation of weighted majority votesvia perturbed variation-based self-labeling,” Pattern RecognitionLetters, vol. 51, pp. 37–43, 2015.

[193] P. Germain, A. Habrard, F. Laviolette, and E. Morvant, “A PAC-Bayesian approach for domain adaptation with specialization tolinear classifiers,” in International Conference on Machine Learning,2013, pp. 738–746.

[194] M. Long, J. Wang, G. Ding, J. Sun, and P. S. Yu, “Transfer fea-ture learning with joint distribution adaptation,” in InternationalConference on Computer Vision, 2013, pp. 2200–2207.

[195] A. Gelman, H. S. Stern, J. B. Carlin, D. B. Dunson, A. Vehtari, andD. B. Rubin, Bayesian data analysis. Chapman and Hall/CRC,2013.

[196] J. R. Finkel and C. D. Manning, “Hierarchical Bayesian domainadaptation,” in Conference of the North American Chapter of theAssociation for Computational Linguistics, 2009, pp. 602–610.

[197] B. P. Carlin and T. A. Louis, Bayes and empirical Bayes methods fordata analysis. Chapman and Hall/CRC, 2010.

[198] X. Li and J. Bilmes, “A Bayesian divergence prior for classifieradaptation,” in Conference on Artificial Intelligence and Statistics,2007, pp. 275–282.

[199] S. E. Shepstone, K.-A. Lee, H. Li, Z.-H. Tan, and S. H. Jensen,“Total variability modeling using source-specific priors,” IEEETransactions on Audio, Speech, and Language Processing, vol. 24, pp.504–517, 2016.

[200] L. Frermann, I. Titov, and M. Pinkal, “A hierarchical Bayesianmodel for unsupervised induction of script knowledge,” in Con-ference of the European Chapter of the Association for ComputationalLinguistics, 2014, pp. 49–57.

[201] W. Bian, D. Tao, and Y. Rui, “Cross-domain human action recog-nition,” IEEE Transactions on Systems, Man, and Cybernetics, PartB, vol. 42, no. 2, pp. 298–307, 2012.

[202] N. Srivastava and R. R. Salakhutdinov, “Discriminative transferlearning with tree-based priors,” in Advances in Neural InformationProcessing Systems, 2013, pp. 2094–2102.

[203] D. Wolpert, “The lack of a priori distinctions between learningalgorithms,” Neural Computation, vol. 8, no. 7, pp. 1341–1390,1996.

[204] A. Melino, “Testing for sample selection bias,” The Review ofEconomic Studies, vol. 49, no. 1, pp. 151–153, 1982.

[205] S. Ben-David and R. Urner, “On the hardness of domain adapta-tion and the utility of unlabeled target samples,” in Conference onAlgorithmic Learning Theory, 2012, pp. 139–153.

[206] D. McNamara and M.-F. Balcan, “Risk bounds for transferringrepresentations with and without fine-tuning,” in InternationalConference on Machine Learning, 2017, pp. 2373–2381.

[207] S. Kanazawa, “General intelligence as a domain-specific adapta-tion.” Psychological Review, vol. 111, no. 2, p. 512, 2004.

[208] J. T. Leek, R. B. Scharpf, H. C. Bravo, D. Simcha, B. Langmead,W. E. Johnson, D. Geman, K. Baggerly, and R. A. Irizarry, “Tack-ling the widespread and critical impact of batch effects in high-throughput data,” Nature Reviews: Genetics, vol. 11, no. 10, 2010.

[209] W. E. Johnson, C. Li, and A. Rabinovic, “Adjusting batch effectsin microarray expression data using empirical Bayes methods,”Biostatistics, vol. 8, no. 1, pp. 118–127, 2007.

[210] T. L. Fare, E. M. Coffey, H. Dai, Y. D. He, D. A. Kessler, K. A.Kilian, J. E. Koch, E. LeProust, M. J. Marton, M. R. Meyeret al., “Effects of atmospheric ozone on microarray data quality,”Analytical Chemistry, vol. 75, no. 17, pp. 4672–4675, 2003.

[211] M. Rojas-Carulla, B. Schoelkopf, R. Turner, and J. Peters, “In-variant models for causal transfer learning,” Journal of MachineLearning Research, vol. 19, no. 36, 2018.

[212] S. Magliacane, T. van Ommen, T. Claassen, S. Bongers, P. Ver-steeg, and J. M. Mooij, “Domain adaptation by using causal infer-ence to predict invariant conditional distributions,” in Advancesin Neural Information Processing Systems, 2018, pp. 10 846–10 856.

[213] K. Zhang, M. Gong, and B. Scholkopf, “Multi-source domainadaptation: a causal view,” in AAAI Conference on Artificial In-telligence, 2015, pp. 3150–3157.

This figure "biophoto_marco.png" is available in "png" format from:

http://arxiv.org/ps/1901.05335v2

This figure "biophoto_wouter.jpg" is available in "jpg" format from:

http://arxiv.org/ps/1901.05335v2