[kd08]

EG UK Theory and Practice of Computer Graphics (2008)Ik Soo Lim Wen Tang (Editors)

Image Abstraction by Structure Adaptive Filtering

Jan Eric Kyprianidis and Juumlrgen Doumlllner

Hasso-Plattner-Institut University of Potsdam Germany

Abstract

In this work we present a framework of automatic non-photorealistic image processing techniques that createsimplified stylistic illustrations from color images videos and 3D renderings To smooth low-contrast regions whilepreserving edges we present a new fast separated implementation of the bilateral filter Our approach works byfiltering in direction of the gradient and then filtering the intermediate result in perpendicular direction Whenapplied iteratively our approach does not suffer from horizontal or vertical artifacts and creates smooth output atcurved boundaries To extract salient important edges we first apply a one-dimensional difference-of-Gaussiansfilter in direction of the gradient and then apply smoothing along a flow field which we derive from the smoothedstructure tensor Our method creates smooth coherent output for line and curve segments

Categories and Subject Descriptors (according to ACM CCS) I33 [Computer Graphics] Line and Curve GenerationI33 [Computer Graphics] Display algorithms

Figure 1 Examples of abstracted images created using our framework

1 Introduction

Photorealistic visualizations used in 3D applications designedfor typical desktops usually show high visual informationdensity In this paper we present non-photorealistic imageprocessing techniques to distill the perceptually importantinformation and optimize the content for the limited screenspace of small displays Our method (Figure 2) extends the

kyprianidis|doellnerhpiuni-potsdamde

approach of Winnemoumlller et al [WOG06] to use iterated bi-lateral filtering for abstraction and difference-of-Gaussians(DoG) for edge extraction We present enhancements to thesetechniques to improve the quality of the output by adaptingthem to the local orientation of the input Our method issimple and efficient to implement on CPU and GPU Imple-mented on the GPU our approach enables real-time process-ing of video and application as a post-processing effect forreal-time 3D renderings

copy The Eurographics Association 2008

Jan Eric Kyprianidis and Juumlrgen Doumlllner Image Abstraction by Structure Adaptive Filtering

Separated Flow-based DoG Filter

Separated Orientation-aligned Bilateral Filter

Bilateral Filterin GradientDirection

Bilateral Filterin TangentDirection

ColorQuantization

Input Output

LocalOrientationEstimation

DoG Filterin GradientDirection

Smooting along Flow Field and Thresholding

ne-iterations

na-iterations

Figure 2 Overview of our abstraction framework

To represent local orientation we construct a smooth tensorfield From the eigenvectors of this tensor field we derive avector field that has similar characteristics as the edge tangentflow (ETF) of Kang et al [KLC07] but its computation ismuch less expensive Besides gradient calculation only linearseparable smoothing with a box or Gaussian filter is neces-sary In contrast to that ETF construction requires severaliterations of a nonlinear filter with a large filter kernel

The xy-separated bilateral filter [PvV05] used by Win-nemoumlller et al suffers from horizontal and vertical artifactsThese artifacts appear in particular when the filter is appliediteratively (Figure 6(c)) Pham presents in [Pha06] a uv-separated bilateral filter implementation that is aligned alongthe flow curves of the structure tensor It gives good resultseven under severe noise conditions but is not optimal whenused with color quantization (Figure 6(d)) Our approachwhich filters in direction perpendicular to the gradient insteadof filtering along the flow curve creates smooth output atcurved boundaries (Figure 6(b)) and works well with colorquantization

DoG edges look frayed and donrsquot reassemble straight lineand curve segments very well (Figure 9) To work around thislimitation Kang et al [KLC07] recently introduced the con-

cept of flow-based difference-of-Gaussians (FDoG) whichcompared to DoG edges create more coherent lines Theyreplaced the DoG filter by a flow-guided anisotropic filterkernel whose shape is defined by the ETF In this work weshow that comparable high-quality results can be achievedby a separated implementation with corresponding reducedcomputational complexity As first pass we apply a one-dimensional DoG filter in gradient direction followed by asecond pass that applies smoothing along the flow curves ofthe vector field induced by the smoothed structure tensor Tointegrate the flow curves we present a straightforward-to-implement method which compared to Euler integration andline integral convolution [CL93] produces more uniformlydistributed sampling positions

2 Related Work

The structure tensor appears in the corner detector of Har-ris and Stephens [HS88] Kass and Witkin constructed in[KW87] a vector field to estimate local orientation whichcan be interpreted as a specialization of the structure tensorBiguumln et al show in [BGW91] that the structure tensor canbe seen as an approximation to fitting a plane to the Fouriertransform of an n-dimensional structure Van Vliet and Ver-



beek discuss in [vVV95] the connection between the structuretensor and double angle representation

The bilateral filter is a nonlinear operation that smoothsimages while preserving edges [AW95TM98PKTD07] It isa powerful tool but computationally very expensive A bruteforce implementation can calculate an output pixel in Or2time where r denotes the filter radius A faster calculationscheme is presented in [Wei06] but is limited to box spatialkernels and targeted toward vectorization on multiple CPUsAnother fast acceleration technique is presented in [PD06]They express the bilateral filter as linear convolutions fol-lowed by two nonlinearities in a higher-dimensional spacewhere the signal intensity is added to the original domaindimensions Applications of this technique are presentedin [CPD07] Barash and Comaniciu show in [BC04] thatthere is a close connection between anisotropic nonlineardiffusion filters and iterated bilateral filters This motivatedWinnemoumlller et al [WOG06] to apply iterated bilateral fil-tering for abstraction For performance reasons they wereforced to use the xy-separated bilateral filter [PvV05]

DeCarlo and Santella [DS02 SD04] use eye-tracking datato guide image abstraction based on mean-shift segmenta-tion [CM02] at different scales Edge extraction is basedon the Canny edge detector [Can86] Other authors egWang et al [WXSC04] extended these ideas to video Hayset al [HE04] present techniques for transforming images andvideos into painterly renderings using brush strokes Theyuse radial basis functions to interpolate brush orientation Fis-cher et al [FBS05] applied stylization techniques to reducethe visual realism of both the camera image and the virtualgraphical objects of an augmented reality system For edge de-tection the Canny edge detector is used Image abstraction isarchived by bilateral filtering To attain real-time frame ratesthe bilateral filter is applied to a down sampled version of theinput image A big drawback of Canny edges is that they canbecome disconnected Winnemoumlller et al [WOG06] there-fore use difference-of-Gaussians (DoG) edges for their videoabstraction framework motivated by the results of [GRG04]where DoG edges are successfully used for human facialillustrations For the abstraction part they use iterated bi-lateral filtering DoG edges are also used in [RD07] butabstraction is based on the Kuwahara [KHEK76] filter Animage abstraction tool based on gradient domain image pro-cessing techniques was recently presented by Orzan et alin [OBBT07]

3 Method

A schematic overview of our framework is shown in Figure 2Input is typically an image a frame of a video or the outputof a 3D rendering We start by estimating the local orientationby calculating the smoothed structure tensor Then the inputis iteratively abstracted by using the separated orientation-aligned bilateral filter We perform a total of na-iterationsTo the result we apply color quantization as described in[WOG06] After ne lt na iterations we extract edges from

(a)

(b)

Figure 3 Tangent field induced by the structure tensor before(a) and after (b) smoothing The right side shows the vectorfield visualized with line integral convolution Gradients withhigh magnitude are colored red

the intermediate result using the separated FDoG filter Inour examples we use ne D 1 and na D 4 Finally theextracted edges are superimposed on the output of the colorquantization

31 Local Orientation Estimation

Our estimation of local orientation is based on the eigen-values of the structure tensor To make full use of RGBcolor information during computation we combine it withDi Zenzorsquos multi-image gradient method [DZ86 Cum89]Suppose f W R2 R3 denotes our input image We firstcalculate an approximation to the directional derivatives in x-and y-direction

f

xD

R

x

G

x

B

x

t

f

yD

R

y

G

y

B

y

tusing the Sobel filter Let J D f=x f=y be the Jaco-bian matrix then the structure tensor of f is given by

gij D JTJ D

0BBBf

xf

x

f

xf

y

f

xf

y

f

yf

y

1CCCA DWE F

F G

The structure tensor induces for a given point a quadraticform which measures the squared rate of change in directionof a vector n D nx ny

Sn D En2x C 2Fnxny CGn2y



(a) (b) (c) (d)

Figure 4 Smoothing the structure tensor illustrated for two gradients of a grayscale image (a) More weight is given to closeraligned vectors (b) Vectors pointing in opposite directions do not cancel out (c) More weight is given to vectors with highermagnitude (d) Problematic case For orthogonal vectors of same magnitude the eigenvalues are equal

The extremal values of Sn on the unit circle correspondto the eigenvalues of gij Since gij is a symmetric ma-trix the eigenvalues are real numbers and the eigenvectors areorthogonal All principal minor determinants of gij are non-negative and therefore gij is positive semidefinite [Swa73]that is both eigenvalues are non-negative The major eigen-value 1 is zero if and only if both gradients are zero Theminor eigenvalue 2 is zero eg in the case of grayscaleimages By solving det

gij i I

D 0 the eigenvalues can

be found to be

12 DE CG ˙

pE G2 C 4F 2

2

Corresponding eigenvectors are

v1 D

F

1 E

v2 D

2 G

F

Locally the maximum rate of change is attained along Rv1and the minimum rate of change along Rv2 The rate ofchange is given by

p1 resp

p2 Note that v1 and v2

only represent orientation and do not point into a particulardirection because J andJ map to the same structure tensorgij

By assigning the eigenvector v2 of the minor eigenvalue toevery point of an image we get a vector field that is alignedto local edge orientation (Figure 3(a)) Unfortunately thisvector field has discontinuities and accordingly tracing itsflow curves does not give good results This can be seen onthe right side of Figure 3(b) which shows the vector fieldvisualized with line integral convolution [CL93] In order tosmooth the vector field we apply smoothing to the structuretensor The result of applying a 5x5 box filter is shown inFigure 3(b) Even better results are achieved using a 7x7 or9x9 Gaussian filter The smoothing of the structure tensoris a linear operation whereas the effect on the eigenvectorfield is highly nonlinear which corresponds geometricallyto principal component analysis What is important is thatwe in contrast to [BvVV99 Knu89] do not normalize thestructure tensor This has the effect that more weight is givento vectors of higher magnitude Figure 4 illustrates the ef-fect of smoothing the structure tensor for two gradients of a

Figure 5 Multiple iterations of the separated orientation-aligned bilateral filter Left Original image Right Resultafter four iterations (d D 30 r D 425)

grayscale image For a given pixel we will refer in the follow-ing to v1 as the gradient direction and to v2 as the tangentdirection The vector field induced by the tangents is calledthe tangent field

32 Separated Orientation-aligned Bilateral Filter

Before filtering we convert the input to CIE-Lab color space[WS82] to avoid color bleeding artifacts [TM98] We usefor the closeness and similarity function Gaussian functionsof the Euclidean distance between their arguments For animage f the bilateral filter is then defined byR

f xGdkx OxkGr kf x f Oxk dxR

Gdkx OxkGr kf x f Oxk dx

with Ox being the center of the filter neigborhood and Gbeing the one-dimensional Gaussian function of variance

G t D1

p2

exp

t2

22

We are interested in applying the bilateral filter recursively(Figure 5) to achieve a nonlinear diffusion effect [BC04]Unfortunately the bilateral filter is not separable Thereforeit is computationally expensive especially when applied re-cursively We propose to approximate the bilateral filter bya separated implementation that first filters in gradient andthen in tangent direction (Figure 7) A comparison of our



(a) Full kernel (b) Orientation-aligned (c) xy-separated [PvV05] (d) uv-separated [Pha06]

Figure 6 Comparison of different bilateral filter implementations (d D 30 r D 425) Top row shows the ouput of thefilter after 4 iterations Bottom row shows the filter result with color quantization [WOG06] applied

Figure 7 Separated orientation-aligned bilateral filter Yel-low points indicate sampling positions which are linearly in-terpolated Left First pass filters in gradient direction RightSecond pass filters in tangent direction

approach with other methods to separate the bilateral filteris shown in Figure 6 Sampling in the required direction isachieved by using a constant step direction which has unitsize along either x or y axis

ıd D

dagger1dy

dx

jdx j jdy j

dx

dy 1

jdx j lt jdy j

Using unit step size for one axis has the advantage that inter-polation at a sampling position can be achieved by linear in-terpolation between two neighboring pixels (Figure 7) Let c0be the sampling origin and either t0 D ı

v1c0

to sample

in gradient direction or t0 D ıv2c0

to sample in tangent

direction Then the sampling positions are c˙i D c0 ˙ i t0and the response of the bilateral filter in direction t0 is givenby

1

k

f c0C

NXiD1

Gd

ikt0k

hGr

1313f cCi f01313 f cCi C Gr

1313f ci f01313 f ci iHere k is the normalization factor eg the sum of all filterweights and N can for example be chosen to be b2d=kt0kc

33 Separated Flow-based Difference-of-Gaussians

We extract edges after ne bilateral filter iterations from theluminance channel By construction the gradient points indirection where the rate of change is maximum Thus if apoint lies on an edge it is most likely found in gradient direc-tion and applying a one-dimensional DoG filter in gradientdirection Z

13f xDe kx Oxk dx

gives the most filter response (Figure 8(a)) Here f denotesthe luminance channel of the input image and De is theDoG operator which is defined by

De t D Ge t Ge0t

The parameter e0 is set to 16 e to approximate theLaplacian-of-Gaussian [MH80] controls the sensitivityof the edge detection For smaller values detects less noisebut important edges may be missed We use D 099 and



(a) (b) (c) (d)

Figure 8 Separated flow-based DoG Yellow points indicate sampling positions which are linearly interpolated (a) First passfilters in direction of the gradient (b) Second pass filters along the flow curves induced by the tangent field (c) Flow curveintegration by line integral convolution [CL93] (d) Flow curve integration by Eulerrsquos method

e D 10 in our examples To evaluate the one-dimensionalDoG filter in gradient direction we use the same approach asdescribed in the previous section

To average the calculated filter response we apply a one-dimensional Gaussian filter along the flow curves inducedby the tangent field (Figure 8(b)) To calculate the samplingpositions we adapt the approach for sampling along a line Atevery sampling position ci1 a new step direction ti and anew step length li is calculated The sampling positions arethen given by

ci D ci1 C li ti

As step direction the orientation of the tangent v2ci1 istaken Since the tangent does not define a particular directionthe direction with the smaller change in orientation is chosen

ti D sgn˝v2ci1 ti1

˛ v2ci1

The step length is chosen such that the new sampling positionis aligned to the center of the pixel in either x or y axis

liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ

ci1y bci1yc 12 sgntiy

tiy

ˇjtix j lt jtiy j

In general the sampling points are not uniformly distributedalong the flow curve For this reason the trapezium ruleis used for integration Let c0 be the starting point tC0 Dv2c0 and t0 D t

C0 be the initial positive and negative step

directions and Li DPijD1lj then the result of smoothing

along the flow curve is given by

1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci

We choose NC and N such that LNC LNC lt 2m Inour examples we use a variance of m D 30 By construc-

tion only linear interpolation between two neighboring pixelsis necessary The tangent at a sampling position can be cal-culated from the interpolated structure tensors of the twoneighboring pixels or by using nearest-neighbor interpolationIn the latter case the tangents can be precomputed For thepurpose of comparison Figure 8(c) illustrates flow integrationbased on line integral convolution [CL93] and Figure 8(d)shows Eulerrsquos method

As final step thresholding is applied We use the smoothedstep function approach described in [WOG06]

Te h D1 h gt 0

1C tanhe h h 0

The parameter e controls the sharpness of the edge outputTo smooth aliasing artifacts caused by thresholding we op-tionally process the thresholded output using a 3x3 Gaussianfilter Kang et al [KLC07] use multiple iterations of theirflow-based DoG filter to further enhance the filter responseWith the exception of Figure 10 all examples shown werecreated using a single iteration with e D 2 Figure 9 com-pares our approach to the isotropic DoG filter and Figure 10to the FDoG filter Note the smooth and coherent line andcurve segments created by the FDoG variants

4 Results

We have implemented our framework in C++ using theOpenGL Shading Language (GLSL) and Apple Quicktimefor video decoding Figure 11 shows an assortment of exam-ples created with our method The following table shows theaverage frame rate for videos at different resolutions

Resolution 8800 GTX 8800 GTS FX 570M

640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215

As test systems two desktop computers (Core2 Duo E660024GHz) with Nvidia GeForce 8800 GTXGTS graphics cardand a notebook (Core2 Duo 7700 24GHz) with NvidiaQuadro FX 570M graphics card were used



(a) Separated FDoG (b) DoG

Figure 9 Separated FDoG vs isotropic DoG

(a) Separated FDoG (b) FDoG [KLC07]

Figure 10 Multiple FDoG iterations The images showthe result of applying 3 iterations of the filter as describedin [KLC07]

5 Conclusions

In this work we presented a framework of image processingtechniques that abstracts visual contents in HD resolutionin real-time We showed that the quality of the xy-separatedbilateral filter can be increased by aligning the filter to thelocal structure Moreover we have shown that flow-basedDoG filtering can be implemented as a separated algorithm Itoutperforms the classical DoG filter in quality although it isnevertheless efficient to compute Both presented techniquesrely on a smooth vector field whose flow curves follow salientimage features We showed that such a vector field can beconstructed from the eigenvalues of the smoothed structuretensor in a highly efficient way

6 Acknowledgments

The authors would like to thank the anonymous reviewersfor their comments This work was supported by the GermanResearch Foundation (DFG) grant DO 6975-1

References

[AW95] Aurich V Weule J Non-linear gaussian filters per-forming edge preserving diffusion In Proc DAGM-Symposium(1995) pp 538ndash545

[BC04] Barash D Comaniciu D A common framework fornonlinear diffusion adaptive smoothing bilateral filtering andmean shift Image and Vision Computing 22 1 (2004) 73ndash81

[BGW91] Biguumln J Granlund G Wiklund J Multidimen-sional orientation estimation with applications to texture analysisand optical flow IEEE Transactions on Pattern Analysis andMachine Intelligence 13 8 (1991) 775ndash790

[BvVV99] Bakker P van Vliet L Verbeek P Edge preservingorientation adaptive filtering In Proc CVPR IEEE Conference onComputer Vision and Pattern Recognition (1999) vol 1 pp 1535ndash1540

[Can86] Canny J F A computational approach to edge detectionIEEE Transactions on Pattern Analysis and Machine Intelligence8 (1986) 769ndash798

[CL93] Cabral B Leedom L C Imaging vector fields usingline integral convolution In Proc ACM SIGGRAPH (1993)pp 263ndash270

[CM02] Comaniciu D Meer P Mean shift A robust approachtoward feature space analysis IEEE Transactions on PatternAnalysis and Machine Intelligence 24 5 (2002) 603ndash619

[CPD07] Chen J Paris S Durand F Real-time edge-awareimage processing with the bilateral grid ACM Transactions onGraphics 26 3 (2007) 103

[Cum89] Cumani A Edge Detection in Multispectral ImagesTech rep Istituto Elettrotecnico Nazionale Galileo Ferraris1989

[DS02] DeCarlo D Santella A Stylization and abstractionof photographs ACM Transactions on Graphics 21 3 (2002)769ndash776

[DZ86] Di Zenzo S A note on the gradient of a multi-imageComputer Vision Graphics and Image Processing 33 1 (1986)116ndash125

[FBS05] Fischer J Bartz D Straber W Stylized augmentedreality for improved immersion In Proc VR IEEE Virtual Reality(2005) pp 195ndash202

[GRG04] Gooch B Reinhard E Gooch A Human facialillustrations Creation and psychophysical evaluation ACM Trans-actions on Graphics 23 1 (2004) 27ndash44

[HE04] Hays J Essa I A Image and video based painterlyanimation In Proc ACM NPAR (2004) pp 113ndash120

[HS88] Harris C Stephens M A combined corner and edgedetector In Proceedings of The Fourth Alvey Vision Conference(1988) pp 147ndash151

[KHEK76] Kuwahara M Hachimura K Eiho S KinoshitaM Digital processing of biomedical images Plenum Press 1976pp 187ndash203

[KLC07] Kang H Lee S Chui C K Coherent line drawingIn Proc ACM NPAR (2007) pp 43ndash50

[Knu89] Knutsson H Representing local structure using tensorsIn The 6th Scandinavian Conference on Image Analysis (OuluFinland June 1989) pp 244ndash251

[KW87] Kass M Witkin A Analyzing oriented patterns Com-puter Vision Graphics and Image Processing 37 3 (March 1987)362ndash385

[MH80] Marr D Hildreth R C Theory of edge detectionProc Royal Society London 207 (1980) 187ndash217



(a) (b) (c)

(d) (e) (f)

Figure 11 Examples Separated orientation-aligned bilateral filter na D 4 d D 30 r D 425 Separated flow-basedDoG ne D 1 D 099 e D 10 m D 30 D 20 Color quantization q D 8 q D 34

[OBBT07] Orzan A Bousseau A Barla P Thollot JStructure-preserving manipulation of photographs In Proc ACMNPAR (2007)

[PD06] Paris S Durand F A fast approximation of the bilateralfilter using a signal processing approach In European Conferenceon Computer Vision (2006) vol 4 pp 568ndash580

[Pha06] Pham T Spatiotonal adaptivity in Super-Resolution ofUndersampled Image Sequences PhD thesis Quantitative Imag-ing Group Delft University of Technology 2006

[PKTD07] Paris S Kornprobst P Tumblin J Durand F Agentle introduction to bilateral filtering and its applications InACM SIGGRAPH courses (2007)

[PvV05] Pham T van Vliet L Separable bilateral filtering forfast video preprocessing In IEEE International Conference onMultimedia and Expo (2005)

[RD07] Redmond N Dingliana J Adaptive abstraction of 3dscenes in real-time In EUROGRAPHICS Short Presentations(2007)

[SD02] Santella A DeCarlo D Abstracted painterly ren-derings using eye-tracking data In Proc ACM NPAR (2002)pp 75ndash82

[SD04] Santella A DeCarlo D Visual interest and npr Anevaluation and manifesto In Proc ACM NPAR (2004) pp 71ndash150

[Swa73] Swamy K On sylvesterrsquos criterion for positive-semidefinite matrices IEEE Transactions on Automatic Control18 3 (1973) 306ndash306

[TM98] Tomasi C Manduchi R Bilateral filtering for grayand color images In Proc ICCV International Conference onComputer Vision (1998) pp 839ndash846

[vVV95] van Vliet L Verbeek P Estimators for orientationand anisotropy in digitized images In Proc ASCI Conference ofthe Advanced School for Computing and Imaging Heijen (1995)vol 95 pp 16ndash18

[Wei06] Weiss B Fast median and bilateral filtering ACM Trans-actions on Graphics 25 3 (2006) 519ndash526

[WOG06] Winnemoumlller H Olsen S C Gooch B Real-timevideo abstraction ACM Transactions on Graphics 25 3 (2006)1221ndash1226

[WS82] Wyszecki G Stiles W S Color Science Concepts andMethods Quantitative Data and Formulae Wiley IntersciencePublishers New York 1982

[WXSC04] Wang J Xu Y Shum H-Y Cohen M F Videotooning ACM Transactions on Graphics 23 3 (2004) 574ndash583

Figure 1 2 11(a) 11(b) 11(d) 11(e) original photos courtesy httpphilipgreenspuncomFigure 11(c) original from the movie rsquoremixedrsquo by The Procrastinators at MITFigure 5 6 11(f) original photo copyright by Anthony Santella [DS02 SD02]



Separated Flow-based DoG Filter

Separated Orientation-aligned Bilateral Filter

Bilateral Filterin GradientDirection

Bilateral Filterin TangentDirection

ColorQuantization

Input Output

LocalOrientationEstimation

DoG Filterin GradientDirection

Smooting along Flow Field and Thresholding

ne-iterations

na-iterations

Figure 2 Overview of our abstraction framework

To represent local orientation we construct a smooth tensorfield From the eigenvectors of this tensor field we derive avector field that has similar characteristics as the edge tangentflow (ETF) of Kang et al [KLC07] but its computation ismuch less expensive Besides gradient calculation only linearseparable smoothing with a box or Gaussian filter is neces-sary In contrast to that ETF construction requires severaliterations of a nonlinear filter with a large filter kernel

The xy-separated bilateral filter [PvV05] used by Win-nemoumlller et al suffers from horizontal and vertical artifactsThese artifacts appear in particular when the filter is appliediteratively (Figure 6(c)) Pham presents in [Pha06] a uv-separated bilateral filter implementation that is aligned alongthe flow curves of the structure tensor It gives good resultseven under severe noise conditions but is not optimal whenused with color quantization (Figure 6(d)) Our approachwhich filters in direction perpendicular to the gradient insteadof filtering along the flow curve creates smooth output atcurved boundaries (Figure 6(b)) and works well with colorquantization

DoG edges look frayed and donrsquot reassemble straight lineand curve segments very well (Figure 9) To work around thislimitation Kang et al [KLC07] recently introduced the con-

cept of flow-based difference-of-Gaussians (FDoG) whichcompared to DoG edges create more coherent lines Theyreplaced the DoG filter by a flow-guided anisotropic filterkernel whose shape is defined by the ETF In this work weshow that comparable high-quality results can be achievedby a separated implementation with corresponding reducedcomputational complexity As first pass we apply a one-dimensional DoG filter in gradient direction followed by asecond pass that applies smoothing along the flow curves ofthe vector field induced by the smoothed structure tensor Tointegrate the flow curves we present a straightforward-to-implement method which compared to Euler integration andline integral convolution [CL93] produces more uniformlydistributed sampling positions

2 Related Work

The structure tensor appears in the corner detector of Har-ris and Stephens [HS88] Kass and Witkin constructed in[KW87] a vector field to estimate local orientation whichcan be interpreted as a specialization of the structure tensorBiguumln et al show in [BGW91] that the structure tensor canbe seen as an approximation to fitting a plane to the Fouriertransform of an n-dimensional structure Van Vliet and Ver-






3 Method


(a)

(b)





f

xD

R

x

G

x

B

x

t

f

yD

R

y

G

y

B

y


gij D JTJ D

0BBBf

xf

x

f

xf

y

f

xf

y

f

yf

y

1CCCA DWE F

F G





(a) (b) (c) (d)



gij i I


be found to be

12 DE CG ˙

pE G2 C 4F 2

2


v1 D

F

1 E

v2 D

2 G

F


p1 resp











G t D1

p2

exp

t2

22








ıd D

dagger1dy

dx

jdx j jdy j

dx

dy 1

jdx j lt jdy j


v1c0

to sample




1

k

f c0C

NXiD1

Gd

ikt0k

hGr





13f xDe kx Oxk dx


De t D Ge t Ge0t




(a) (b) (c) (d)




ci D ci1 C li ti


ti D sgn˝v2ci1 ti1

˛ v2ci1


liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ


tiy

ˇjtix j lt jtiy j





1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci




Te h D1 h gt 0

1C tanhe h h 0


4 Results



640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215








5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)























3 Method


(a)

(b)





f

xD

R

x

G

x

B

x

t

f

yD

R

y

G

y

B

y


gij D JTJ D

0BBBf

xf

x

f

xf

y

f

xf

y

f

yf

y

1CCCA DWE F

F G





(a) (b) (c) (d)



gij i I


be found to be

12 DE CG ˙

pE G2 C 4F 2

2


v1 D

F

1 E

v2 D

2 G

F


p1 resp











G t D1

p2

exp

t2

22








ıd D

dagger1dy

dx

jdx j jdy j

dx

dy 1

jdx j lt jdy j


v1c0

to sample




1

k

f c0C

NXiD1

Gd

ikt0k

hGr





13f xDe kx Oxk dx


De t D Ge t Ge0t




(a) (b) (c) (d)




ci D ci1 C li ti


ti D sgn˝v2ci1 ti1

˛ v2ci1


liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ


tiy

ˇjtix j lt jtiy j





1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci




Te h D1 h gt 0

1C tanhe h h 0


4 Results



640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215








5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)




















(a) (b) (c) (d)



gij i I


be found to be

12 DE CG ˙

pE G2 C 4F 2

2


v1 D

F

1 E

v2 D

2 G

F


p1 resp











G t D1

p2

exp

t2

22








ıd D

dagger1dy

dx

jdx j jdy j

dx

dy 1

jdx j lt jdy j


v1c0

to sample




1

k

f c0C

NXiD1

Gd

ikt0k

hGr





13f xDe kx Oxk dx


De t D Ge t Ge0t




(a) (b) (c) (d)




ci D ci1 C li ti


ti D sgn˝v2ci1 ti1

˛ v2ci1


liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ


tiy

ˇjtix j lt jtiy j





1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci




Te h D1 h gt 0

1C tanhe h h 0


4 Results



640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215








5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)
























ıd D

dagger1dy

dx

jdx j jdy j

dx

dy 1

jdx j lt jdy j


v1c0

to sample




1

k

f c0C

NXiD1

Gd

ikt0k

hGr





13f xDe kx Oxk dx


De t D Ge t Ge0t




(a) (b) (c) (d)




ci D ci1 C li ti


ti D sgn˝v2ci1 ti1

˛ v2ci1


liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ


tiy

ˇjtix j lt jtiy j





1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci




Te h D1 h gt 0

1C tanhe h h 0


4 Results



640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215








5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)




















(a) (b) (c) (d)




ci D ci1 C li ti


ti D sgn˝v2ci1 ti1

˛ v2ci1


liD

bdquo ˇci1x bci1xc

12 sgntix

tix

ˇjtix j jtiy jˇ


tiy

ˇjtix j lt jtiy j





1

k

1313tC0 13132

f c0 C

NCXiD1

1313tCi 1313C 1313tCi113132

Gm

LCi

fcCi

C

1313t0 13132

f c0 C

NXiD1

1313ti 1313C 1313ti113132

Gm

Li

fci




Te h D1 h gt 0

1C tanhe h h 0


4 Results



640 x 480 878 531 181768 x 576 615 375 124

1280 x 720 299 175 601920 x 1080 127 54 215








5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)
























5 Conclusions


6 Acknowledgments


References























(a) (b) (c)

(d) (e) (f)




















(a) (b) (c)

(d) (e) (f)



















[kd08]

Documents

abstraction framework

flow field

nonlinear filter

bilateral filter pv

gaussian filter

tensor field

smooth coherent output

local orientation