ping tan et al. proceedings of acm siggraph asia...

Ping Tan et al.

Proceedings of ACM SIGGRAPH Asia 2008

Ja Min Jeong

2010/09/21

Korea University

Computer Graphics Lab.

Korea University Computer Graphics Lab.

Ja Min Jeong | 2011/09/21 | # 2 KUCG |

Abstract

Single Image User Input

3D Branch Reconstruction

Leaves Reconstruction


Ja Min Jeong | 2011/09/21 | # 3 KUCG |

1. Introduction

• Trees

Ubiquitous

Hard to model • Large varieties

• Natural complexity in geometry

• Useful

Movie

Architectural designs

Games

Web Applications


Ja Min Jeong | 2011/09/21 | # 4 KUCG |

1. Introduction

• Previous Method

Amount of efforts -> good results • Many Parameters, Many images, 3D reconstruction

• Remarkably simple & Convincing results

Single image

Few user drawn strokes

Non-parametric synthesis


Ja Min Jeong | 2011/09/21 | # 5 KUCG |

Overveiw

Single Tree Image User Input

Foliage (red) GrabCut Foliage extraction

Branch (blue) Gaussian mixture

model Branch Tracing

Leaf Texture

synthesis

SubTree Example New branch

synthesis

Branch

Pruning Branch Complete Generation leaves

Gaussian mixture

model Gibbs energy GrabCut


Ja Min Jeong | 2011/09/21 | # 6 KUCG |

1.1 Related work

• Gaussian model


Ja Min Jeong | 2011/09/21 | # 7 KUCG |

1.1 Related work

• Gaussian mixture model


Ja Min Jeong | 2011/09/21 | # 8 KUCG |

1.1 Related work

• Gaussian mixture model


Ja Min Jeong | 2011/09/21 | # 9 KUCG |

1.1 Related work

• GrabCut


Ja Min Jeong | 2011/09/21 | # 10 KUCG |

1.2 Our approach

• Image based Tree Modeling [Ping Tan et al.]


Ja Min Jeong | 2011/09/21 | # 11 KUCG |

1.2 Our approach

• Sketch based method

Interactive Design of Botanical Trees using Freehand Sketches and Example-based Editing [Makoto Okabe et al.]


Ja Min Jeong | 2011/09/21 | # 12 KUCG |

2 Image Plane Sketching

• 2.1 User interface

Foliage region(red)

Branch stroke(blue)

• Not satisfied with this result

Adding or Deleting strokes for correction


Ja Min Jeong | 2011/09/21 | # 13 KUCG |

2.2 Foliage extraction

• Foliage extraction

GrabCut

Gaussian mixture model • Mixture of 10 Gaussians

• Four components (Foliage) : green or red Gaussian

• Six components (background)


Ja Min Jeong | 2011/09/21 | # 14 KUCG |

• Weight : Gibbs energy

• Foliage

• Background : indicates the RGB values at pixel x

: GMM parameters


S T


Ja Min Jeong | 2011/09/21 | # 15 KUCG |

• Gibbs energy

= 1 : Background pixels

= 0 : Leaf pixels

: set of all 4-neighbor pixel pairs

• constant (strength of smoothness) : 60 in our implementation



Ja Min Jeong | 2011/09/21 | # 16 KUCG |

2.3 Visible branch tracing

• Lazy Snapping[Yin Li et al.]


Ja Min Jeong | 2011/09/21 | # 17 KUCG |


• Lazy Snapping[Yin Li et al.]

• Circle radius > branch radius

•

: 0 (branch pixel) 1 (others)

: branch Gaussian parameter

: non-branch Gaussian parameter


Ja Min Jeong | 2011/09/21 | # 18 KUCG |


• Not satisfied with this result (Add and Del)

• Discarding redundant joints (simplied)

• Circle radius is fixed (50 pixels) all examples

• Branch radius to 75% of its parent


Ja Min Jeong | 2011/09/21 | # 19 KUCG |

3 Tree Growing


Ja Min Jeong | 2011/09/21 | # 20 KUCG |

3.1 Growth engine

• Initialization

Conversion of 2D branches into 3D


Ja Min Jeong | 2011/09/21 | # 21 KUCG |

3.1.1 Initialization

• Creation of the library

Elementary subtrees


Ja Min Jeong | 2011/09/21 | # 22 KUCG |

3.1.2 Non-parametric synthesis

New branches grow at the end of its `supporting branch‘

New branches can grow along the `supporting branch'


Ja Min Jeong | 2011/09/21 | # 23 KUCG |


Selection of a

branch to be

replaced

Selection of a

replacing library

subtree

Branch pruning

Check Threshold

in Section 3.2

Iteratively about 100 times


Ja Min Jeong | 2011/09/21 | # 24 KUCG |


• Selection of a branch to be replaced Larger radius and Older generation

• Selection of a replacing library subtree Add predefined subtrees: type I, type II or both

The rotation angle (12 levels of 360 degrees)

The scaling factor (same length as the replaced branch)

• Branch pruning Pruned by the extracted foliage Silhouette

removed, new branches if they are too close to some existing branches


Ja Min Jeong | 2011/09/21 | # 25 KUCG |

3.2 Data-driven attractors

• Desired tree volume

2D (weakly controls) 3D (better)

• Image attractors

Growth is then driven by minimizing a cost


Ja Min Jeong | 2011/09/21 | # 26 KUCG |


• Extrapolated 3D attractors

How to 2D -> 3D ?

Foliage region • Tree is rotated 90 degrees around its main trunk

• Merged with the original one

Branch • Take the set of branch joints of the current tree

• Rotate these joints 90 degree around the main trunk


Ja Min Jeong | 2011/09/21 | # 27 KUCG |



Ja Min Jeong | 2011/09/21 | # 28 KUCG |


• 2D

• 3D

Top Veiw Side Veiw


Ja Min Jeong | 2011/09/21 | # 29 KUCG |


• Speedup

Computation of the distances • Distance computation in previous iteration can be reused

: newly created branches (red)


Ja Min Jeong | 2011/09/21 | # 30 KUCG |

4 Completing the Tree

• Leaves of the tree

Synthesized from the recovered branch structure and textured

a at rectangle leaf size • (1/10 of main trunk radius)

Proportional to Branch length • 50 ~ 200 leaves

Arrangement of the leaves • Randomized

Leaves texture <- input image

leaf shape, leaf size, density and arrangement of leaves -> parameterized


Ja Min Jeong | 2011/09/21 | # 31 KUCG |

Result

• 20 minutes

• PC with 2.4G CPU


Ja Min Jeong | 2011/09/21 | # 32 KUCG |

Result


Ja Min Jeong | 2011/09/21 | # 33 KUCG |

Result


Ja Min Jeong | 2011/09/21 | # 34 KUCG |

Limitations

• Less similar to the original tree at different veiws

A database of real 3D branches might be helpful

• Less leaf detail

Cannot model large leaves

Cannot model flowers

• Interaction to mark out tree

Cannot handle very large scale data


Ja Min Jeong | 2011/09/21 | # 35 KUCG |

Conclusion & Future Work

• Simple and effective system Single image

Minimize user interaction

Amateur user : Sketch few strokes

• Previous multi-views methods -> Single image

• Modeling procedure fully automatic Automatic tree detection

Visible branch tracing algorithm

ping tan et al. proceedings of acm siggraph asia...

Documents