![Page 1: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/1.jpg)
Approximate Nearest Line Search in High Dimensions
Sepideh Mahabadi
![Page 2: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/2.jpg)
The NLS Problem
• Given: a set of 𝑁 lines 𝐿 in ℝ𝑑
![Page 3: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/3.jpg)
The NLS Problem
• Given: a set of 𝑁 lines 𝐿 in ℝ𝑑 • Goal: build a data structure s.t.
– given a query 𝑞, find the closest line ℓ∗ to 𝑞
![Page 4: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/4.jpg)
The NLS Problem
• Given: a set of 𝑁 lines 𝐿 in ℝ𝑑 • Goal: build a data structure s.t.
– given a query 𝑞, find the closest line ℓ∗ to 𝑞
– polynomial space – sub-linear query time
![Page 5: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/5.jpg)
The NLS Problem
• Given: a set of 𝑁 lines 𝐿 in ℝ𝑑 • Goal: build a data structure s.t.
– given a query 𝑞, find the closest line ℓ∗ to 𝑞
– polynomial space – sub-linear query time
Approximation • Finds an approximate closest line ℓ 𝑑𝑑𝑑𝑑 𝑞,ℓ ≤ 𝑑𝑑𝑑𝑑(𝑞, ℓ∗)(1 + 𝜖)
![Page 6: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/6.jpg)
BACKGROUND
Nearest Neighbor Problems Motivation Previous Work Our result Notation
![Page 7: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/7.jpg)
Nearest Neighbor Problem NN: Given a set of 𝑁 points 𝑃, build a data structure s.t. given a query point 𝑞, finds the closest point 𝑝∗ to 𝑞.
![Page 8: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/8.jpg)
Nearest Neighbor Problem NN: Given a set of 𝑁 points 𝑃, build a data structure s.t. given a query point 𝑞, finds the closest point 𝑝∗ to 𝑞. • Applications: database, information retrieval,
pattern recognition, computer vision – Features: dimensions – Objects: points – Similarity: distance between points
![Page 9: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/9.jpg)
Nearest Neighbor Problem NN: Given a set of 𝑁 points 𝑃, build a data structure s.t. given a query point 𝑞, finds the closest point 𝑝∗ to 𝑞. • Applications: database, information retrieval,
pattern recognition, computer vision – Features: dimensions – Objects: points – Similarity: distance between points
• Current solutions suffer from “curse of dimensionality”: – Either space or query time is exponential in 𝑑 – Little improvement over linear search
![Page 10: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/10.jpg)
Approximate Nearest Neighbor(ANN)
• ANN: Given a set of 𝑁 points 𝑃, build a data structure s.t. given a query point 𝑞, finds an approximate closest point 𝑝 to 𝑞, i.e.,
𝑑𝑑𝑑𝑑 𝑞,𝑝 ≤ 𝑑𝑑𝑑𝑑 𝑞, 𝑝∗ 1 + 𝜖
![Page 11: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/11.jpg)
Approximate Nearest Neighbor(ANN)
• ANN: Given a set of 𝑁 points 𝑃, build a data structure s.t. given a query point 𝑞, finds an approximate closest point 𝑝 to 𝑞, i.e.,
𝑑𝑑𝑑𝑑 𝑞,𝑝 ≤ 𝑑𝑑𝑑𝑑 𝑞, 𝑝∗ 1 + 𝜖 • There exist data structures with different
tradeoffs. Example: – Space: 𝑑𝑁 𝑂 1
𝜖2
– Query time: 𝑑 log 𝑁𝜖
𝑂 1
![Page 12: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/12.jpg)
Motivation for NLS
One of the simplest generalizations of ANN: data items are represented by 𝑘-flats (affine subspace) instead of points
![Page 13: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/13.jpg)
Motivation for NLS
One of the simplest generalizations of ANN: data items are represented by 𝑘-flats (affine subspace) instead of points • Model data under linear variations • Unknown or unimportant parameters in
database
![Page 14: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/14.jpg)
Motivation for NLS
One of the simplest generalizations of ANN: data items are represented by 𝑘-flats (affine subspace) instead of points • Model data under linear variations • Unknown or unimportant parameters in
database • Example:
– Varying light gain parameter of images – Each image/point becomes a line – Search for the closest line to the query image
![Page 15: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/15.jpg)
Previous and Related Work • Magen[02]: Nearest Subspace Search for constant 𝑘
– Query time is fast : 𝑑 + log𝑁 + 1𝜖
𝑂 1
– Space is super-polynomial : 2 log 𝑁 𝑂 1
![Page 16: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/16.jpg)
Previous and Related Work • Magen[02]: Nearest Subspace Search for constant 𝑘
– Query time is fast : 𝑑 + log𝑁 + 1𝜖
𝑂 1
– Space is super-polynomial : 2 log 𝑁 𝑂 1
Dual Problem: Database is a set of points, query is a 𝑘-flat • [AIKN] for 1-flat: for any 𝑑 > 0
– Query time: 𝑂 𝑑3𝑁0.5+𝑡
– Space: 𝑑2𝑁𝑂 1𝜖2+
1𝑡2
![Page 17: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/17.jpg)
Previous and Related Work • Magen[02]: Nearest Subspace Search for constant 𝑘
– Query time is fast : 𝑑 + log𝑁 + 1𝜖
𝑂 1
– Space is super-polynomial : 2 log 𝑁 𝑂 1
Dual Problem: Database is a set of points, query is a 𝑘-flat • [AIKN] for 1-flat: for any 𝑑 > 0
– Query time: 𝑂 𝑑3𝑁0.5+𝑡
– Space: 𝑑2𝑁𝑂 1𝜖2+
1𝑡2
• Very recently [MNSS] extended it for 𝑘-flats
– Query time 𝑂 𝑛𝑘
𝑘+1−𝜌+𝑡
– Space: 𝑂(𝑛1+𝜎𝑘
𝑘+1−𝜌 + 𝑛 log𝑂1𝑡 𝑛)
![Page 18: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/18.jpg)
Our Result We give a randomized algorithm that for any sufficiently small 𝜖 reports a 1 + 𝜖 -approximate solution with high probability
• Space: 𝑁 + 𝑑 𝑂 1𝜖2
• Time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
![Page 19: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/19.jpg)
Our Result We give a randomized algorithm that for any sufficiently small 𝜖 reports a 1 + 𝜖 -approximate solution with high probability
• Space: 𝑁 + 𝑑 𝑂 1𝜖2
• Time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
• Matches up to polynomials, the performance of best algorithm for ANN. No exponential dependence on 𝑑
![Page 20: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/20.jpg)
Our Result We give a randomized algorithm that for any sufficiently small 𝜖 reports a 1 + 𝜖 -approximate solution with high probability
• Space: 𝑁 + 𝑑 𝑂 1𝜖2
• Time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
• Matches up to polynomials, the performance of best algorithm for ANN. No exponential dependence on 𝑑
• The first algorithm with poly log query time and polynomial space for objects other than points
![Page 21: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/21.jpg)
Our Result We give a randomized algorithm that for any sufficiently small 𝜖 reports a 1 + 𝜖 -approximate solution with high probability
• Space: 𝑁 + 𝑑 𝑂 1𝜖2
• Time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
• Matches up to polynomials, the performance of best algorithm for ANN. No exponential dependence on 𝑑
• The first algorithm with poly log query time and polynomial space for objects other than points
• Only uses reductions to ANN
![Page 22: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/22.jpg)
Notation
• 𝐿 : the set of lines with size 𝑁 • q : the query point
![Page 23: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/23.jpg)
Notation
• 𝐿 : the set of lines with size 𝑁 • q : the query point • 𝐵(𝑐, 𝑟): ball of radius 𝑟 around 𝑐
![Page 24: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/24.jpg)
Notation
• 𝐿 : the set of lines with size 𝑁 • q : the query point • 𝐵(𝑐, 𝑟): ball of radius 𝑟 around 𝑐 • 𝑑𝑑𝑑𝑑: the Euclidean distance
between objects
![Page 25: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/25.jpg)
Notation
• 𝐿 : the set of lines with size 𝑁 • q : the query point • 𝐵(𝑐, 𝑟): ball of radius 𝑟 around 𝑐 • 𝑑𝑑𝑑𝑑: the Euclidean distance
between objects • 𝑎𝑛𝑎𝑎𝑎: defined between lines
![Page 26: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/26.jpg)
Notation
• 𝐿 : the set of lines with size 𝑁 • q : the query point • 𝐵(𝑐, 𝑟): ball of radius 𝑟 around 𝑐 • 𝑑𝑑𝑑𝑑: the Euclidean distance
between objects • 𝑎𝑛𝑎𝑎𝑎: defined between lines • 𝛿-close: two lines ℓ , ℓ′ are 𝛿-close
if sin(𝑎𝑛𝑎𝑎𝑎 ℓ, ℓ′ ) ≤ 𝛿
![Page 27: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/27.jpg)
MODULES
Net Module Unbounded Module Parallel Module
![Page 28: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/28.jpg)
Net Module • Intuition: sampling points from each line
finely enough to get a set of points 𝑃, and building an 𝐴𝑁𝑁(𝑃, 𝜖) should suffice to find the approximate closest line.
![Page 29: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/29.jpg)
Net Module • Intuition: sampling points from each line
finely enough to get a set of points 𝑃, and building an 𝐴𝑁𝑁(𝑃, 𝜖) should suffice to find the approximate closest line.
Lemma: • Let 𝑥 be the separation parameter:
distance between two adjacent samples on a line, Then – Either the returned line ℓ𝑝 is an
approximate closest line – Or 𝑑𝑑𝑑𝑑 𝑞, ℓ𝑝 ≤ 𝑥/𝜖
![Page 30: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/30.jpg)
Net Module • Intuition: sampling points from each line
finely enough to get a set of points 𝑃, and building an 𝐴𝑁𝑁(𝑃, 𝜖) should suffice to find the approximate closest line.
Lemma: • Let 𝑥 be the separation parameter:
distance between two adjacent samples on a line, Then – Either the returned line ℓ𝑝 is an
approximate closest line – Or 𝑑𝑑𝑑𝑑 𝑞, ℓ𝑝 ≤ 𝑥/𝜖
Issue: It should be used inside a bounded region
![Page 31: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/31.jpg)
Unbounded Module - Intuition • All lines in 𝐿 pass through the origin 𝑜
![Page 32: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/32.jpg)
Unbounded Module - Intuition • All lines in 𝐿 pass through the origin 𝑜
• Data structure: – Project all lines onto any sphere 𝑆 𝑜,𝑟 to get point set 𝑃
– Build ANN data structure 𝐴𝑁𝑁(𝑃, 𝜖)
![Page 33: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/33.jpg)
Unbounded Module - Intuition • All lines in 𝐿 pass through the origin 𝑜
• Data structure: – Project all lines onto any sphere 𝑆 𝑜,𝑟 to get point set 𝑃
– Build ANN data structure 𝐴𝑁𝑁(𝑃, 𝜖) • Query Algorithm:
– Project the query on 𝑆(𝑜, 𝑟) to get 𝑞′ – Find the approximate closest point to 𝑞′, i.e., 𝑝 = 𝐴𝑁𝑁𝑃 𝑞′
– Return the corresponding line of 𝑝
![Page 34: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/34.jpg)
Unbounded Module • All lines in 𝐿 pass through a small ball 𝐵 𝑜, 𝑟
• Query is far enough, outside of 𝐵(𝑜,𝑅) • Use the same data structure and query algorithm
![Page 35: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/35.jpg)
Unbounded Module • All lines in 𝐿 pass through a small ball 𝐵 𝑜, 𝑟
• Query is far enough, outside of 𝐵(𝑜,𝑅) • Use the same data structure and query algorithm Lemma: if 𝑅 ≥ 𝑟
𝜖𝜖 , the returned line ℓ𝑝 is
• Either an approximate closest line • Or is 𝛿-close to the closest line ℓ∗
![Page 36: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/36.jpg)
Unbounded Module • All lines in 𝐿 pass through a small ball 𝐵 𝑜, 𝑟
• Query is far enough, outside of 𝐵(𝑜,𝑅) • Use the same data structure and query algorithm Lemma: if 𝑅 ≥ 𝑟
𝜖𝜖 , the returned line ℓ𝑝 is
• Either an approximate closest line • Or is 𝛿-close to the closest line ℓ∗
This helps us in two ways • Bound the region for the net module • Restrict search to almost parallel lines
![Page 37: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/37.jpg)
Parallel Module - Intuition • All lines in 𝐿 are parallel
![Page 38: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/38.jpg)
Parallel Module - Intuition • All lines in 𝐿 are parallel • Data structure:
– Project all lines onto any hyper-plane 𝑎 which is perpendicular to all the lines to get point set 𝑃
– Build ANN data structure 𝐴𝑁𝑁(𝑃, 𝜖)
![Page 39: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/39.jpg)
Parallel Module - Intuition • All lines in 𝐿 are parallel • Data structure:
– Project all lines onto any hyper-plane 𝑎 which is perpendicular to all the lines to get point set 𝑃
– Build ANN data structure 𝐴𝑁𝑁(𝑃, 𝜖) • Query algorithm:
– Project the query on 𝑎 to get 𝑞′ – Find the approximate closest point to 𝑞′, i.e., 𝑝 = 𝐴𝑁𝑁𝑃 𝑞′
– Return the corresponding line to 𝑝
![Page 40: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/40.jpg)
Parallel Module • All lines in 𝐿 are 𝛿-close to a base line ℓ𝑏 • Project the lines onto a hyper-plane 𝑎 which is
perpendicular to ℓ𝑏 • Query is close enough to 𝑎 • Use the same data structure and query algorithm
![Page 41: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/41.jpg)
Parallel Module • All lines in 𝐿 are 𝛿-close to a base line ℓ𝑏 • Project the lines onto a hyper-plane 𝑎 which is
perpendicular to ℓ𝑏 • Query is close enough to 𝑎 • Use the same data structure and query algorithm Lemma: if 𝑑𝑑𝑑𝑑 𝑞,𝑎 ≤ 𝐷𝜖
𝜖 , then
• Either the returned line ℓ𝑝 is an approximate closest line
• Or 𝑑𝑑𝑑𝑑 𝑞, ℓ𝑝 ≤ 𝐷
![Page 42: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/42.jpg)
Parallel Module • All lines in 𝐿 are 𝛿-close to a base line ℓ𝑏 • Project the lines onto a hyper-plane 𝑎 which is
perpendicular to ℓ𝑏 • Query is close enough to 𝑎 • Use the same data structure and query algorithm Lemma: if 𝑑𝑑𝑑𝑑 𝑞,𝑎 ≤ 𝐷𝜖
𝜖 , then
• Either the returned line ℓ𝑝 is an approximate closest line
• Or 𝑑𝑑𝑑𝑑 𝑞, ℓ𝑝 ≤ 𝐷 Thus, for a set of almost parallel lines, we can use a set of parallel modules to cover a bounded region.
![Page 43: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/43.jpg)
How the Modules Work Together
Given a set of lines, we come up with a polynomial number of balls.
![Page 44: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/44.jpg)
How the Modules Work Together
Given a set of lines, we come up with a polynomial number of balls. • If 𝑞 is inside the ball
– Use net module q
![Page 45: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/45.jpg)
How the Modules Work Together
Given a set of lines, we come up with a polynomial number of balls. • If 𝑞 is inside the ball
– Use net module
• If 𝑞 is outside the ball – First use unbounded module to
find a line ℓ
q
![Page 46: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/46.jpg)
How the Modules Work Together
Given a set of lines, we come up with a polynomial number of balls. • If 𝑞 is inside the ball
– Use net module
• If 𝑞 is outside the ball – First use unbounded module to
find a line ℓ
q ℓ
![Page 47: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/47.jpg)
How the Modules Work Together
Given a set of lines, we come up with a polynomial number of balls. • If 𝑞 is inside the ball
– Use net module
• If 𝑞 is outside the ball – First use unbounded module to
find a line ℓ – Then use parallel module to
search among parallel lines to ℓ
q ℓ
![Page 48: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/48.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆
![Page 49: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/49.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇
![Page 50: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/50.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝
![Page 51: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/51.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝 • For log𝑛 iterations
– Use ℓ𝑝 to find a much closer line ℓ𝑝′ – Update ℓ𝑝 with ℓ𝑝′
Improvement step
![Page 52: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/52.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝 • For log𝑛 iterations
– Use ℓ𝑝 to find a much closer line ℓ𝑝′ – Update ℓ𝑝 with ℓ𝑝′
Improvement step
![Page 53: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/53.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝 • For log𝑛 iterations
– Use ℓ𝑝 to find a much closer line ℓ𝑝′ – Update ℓ𝑝 with ℓ𝑝′
Why?
Improvement step
![Page 54: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/54.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝 • For log𝑛 iterations
– Use ℓ𝑝 to find a much closer line ℓ𝑝′ – Update ℓ𝑝 with ℓ𝑝′
Let 𝑑1 , … , 𝑑log 𝑛 be the log 𝑛 closest lines to 𝑞 in the set 𝑆
Improvement step
![Page 55: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/55.jpg)
Outline of the Algorithms • Input: a set of 𝑛 lines 𝑆 • Randomly choose a subset of 𝑛/2 lines 𝑇 • Solve the problem over 𝑇 to get a line ℓ𝑝 • For log𝑛 iterations
– Use ℓ𝑝 to find a much closer line ℓ𝑝′ – Update ℓ𝑝 with ℓ𝑝′
Let 𝑑1 , … , 𝑑log 𝑛 be the log 𝑛 closest lines to 𝑞 in the set 𝑆 With high probability at least one of {𝑑1 , … , 𝑑log 𝑛} is sampled in 𝑇
– 𝑑𝑑𝑑𝑑 𝑞, ℓ𝑝 ≤ 𝑑𝑑𝑑𝑑 𝑞, 𝑑log 𝑛 (1 + 𝜖) – log 𝑛 improvement steps suffices to find an approximate closest
line
Improvement step
![Page 56: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/56.jpg)
Improvement Step
Given a line ℓ, how to improve it, i.e., find a closer line?
![Page 57: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/57.jpg)
Improvement Step
Given a line ℓ, how to improve it, i.e., find a closer line? • Data structure • Query Processing Algorithm
![Page 58: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/58.jpg)
Improvement Step
Given a line ℓ, how to improve it, i.e., find a closer line? • Data structure • Query Processing Algorithm Use the three modules here
![Page 59: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/59.jpg)
Conclusion
Bounds we get for NLS problem
– Polynomial Space: 𝑂 𝑁 + 𝑑 𝑂 1𝜖2
– Poly-logarithmic query time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
![Page 60: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/60.jpg)
Conclusion
Bounds we get for NLS problem
– Polynomial Space: 𝑂 𝑁 + 𝑑 𝑂 1𝜖2
– Poly-logarithmic query time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
Future Work • The current result is not efficient in practice
– Large exponents – Algorithm is complicated
![Page 61: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/61.jpg)
Conclusion
Bounds we get for NLS problem
– Polynomial Space: 𝑂 𝑁 + 𝑑 𝑂 1𝜖2
– Poly-logarithmic query time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
Future Work • The current result is not efficient in practice
– Large exponents – Algorithm is complicated
• Can we get a simpler algorithms?
![Page 62: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/62.jpg)
Conclusion
Bounds we get for NLS problem
– Polynomial Space: 𝑂 𝑁 + 𝑑 𝑂 1𝜖2
– Poly-logarithmic query time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
Future Work • The current result is not efficient in practice
– Large exponents – Algorithm is complicated
• Can we get a simpler algorithms? • Generalization to higher dimensional flats
![Page 63: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/63.jpg)
Conclusion
Bounds we get for NLS problem
– Polynomial Space: 𝑂 𝑁 + 𝑑 𝑂 1𝜖2
– Poly-logarithmic query time : 𝑑 + log𝑁 + 1𝜖
𝑂 1
Future Work • The current result is not efficient in practice
– Large exponents – Algorithm is complicated
• Can we get a simpler algorithm? • Generalization to higher dimensional flats • Generalization to other objects, e.g. balls
![Page 64: Approximate Nearest Line Search in High Dimensionsmahabadi/slides/lnn-short.pdf · • Applications: database, information retrieval, pattern recognition, computer vision – Features:](https://reader036.vdocuments.us/reader036/viewer/2022081615/5fd3c309137dbf22e67175c4/html5/thumbnails/64.jpg)
THANK YOU!