adaptive cleaning for rfid data streams
DESCRIPTION
Shawn Jeffery Minos Garofalakis Michael Franklin UC Berkeley Intel Research Berkeley UC Berkeley . Adaptive Cleaning for RFID Data Streams. Presented by Willie and Abhishek. Disclaimer: The slides are taken from Jeffrey’s talk at VLDB ‘06. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/1.jpg)
Adaptive Cleaning for Adaptive Cleaning for RFID Data StreamsRFID Data Streams
Presented by Willie and Abhishek
Shawn Jeffery Minos Garofalakis Michael Franklin UC Berkeley Intel Research Berkeley UC Berkeley
Disclaimer: The slides are taken from Jeffrey’s talk at VLDB ‘06
![Page 2: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/2.jpg)
04/22/23
RFID: Radio Frequency RFID: Radio Frequency IDentificationIDentification
![Page 3: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/3.jpg)
04/22/23
RFID data is dirtyRFID data is dirtyShelf 0 Shelf 1
RFIDReaders
StaticTags
Mobile Tags
15ft1.5ft
3ft9ft
3ft
3ft
3ft
A simple experiment:•2 RFID-enabled shelves•10 static tags•5 mobile tags
![Page 4: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/4.jpg)
04/22/23
RFID Data CleaningRFID Data Cleaning
Time
Raw readings
Smoothed output
• RFID data has many dropped readings• Typically, use a smoothing filter to
interpolateSELECT distinct tag_idFROM RFID_stream [RANGE ‘5 sec’]GROUP BY tag_idBut, how to set the size
of the window?
Smoothing Filter
![Page 5: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/5.jpg)
04/22/23
Window Size for RFID Window Size for RFID SmoothingSmoothing
Fido moving Fido resting
Small windowRealityRaw readings
Large window
Need to balance completeness vs. capturing tag movement
![Page 6: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/6.jpg)
04/22/23
Truly Declarative Truly Declarative SmoothingSmoothing
• Problem: window size non-declarative• Application wants a clean stream
of data• Window size is how to get it
• Solution: adapt the window size in response to data
![Page 7: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/7.jpg)
04/22/23
ItineraryItinerary
• Introduction: RFID data cleaning• A statistical sampling perspective• SMURF
• Per-tag cleaning• Multi-tag cleaning
• Ongoing work• Conclusions
![Page 8: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/8.jpg)
04/22/23
A Statistical Sampling A Statistical Sampling PerspectivePerspective
• Key Insight: RFID data random sample of present tags
• Map RFID smoothing to a sampling experiment
![Page 9: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/9.jpg)
04/22/23
RFID’s Gory DetailsRFID’s Gory Details
Epoch TagID ReadRate0 1 .90 2 .60 3 .3
Tag 1
Tag 2
Tag 3
Tag 4
Antenna & readerTags
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
Read Cycle (Epoch)
(For Alien readers)
Tag List
![Page 10: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/10.jpg)
04/22/23
RFID Smoothing to SamplingRFID Smoothing to Sampling
RFID SamplingRead cycle (epoch) Sample trialReading Single sampleSmoothing window Repeated trialsRead rate Probability of inclusion
(pi)
Now use sampling theory to drive adaptation!
![Page 11: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/11.jpg)
04/22/23
SMURFSMURF• Statistical Smoothing for Unreliable RFID
Data• Adapts window based on statistical
properties• Mechanisms for:
• Per-tag and multi-tag cleaning
Multi-tagCleaning
SMURFPer-tag
Cleaning
raw RFID streams
cleanedcount readings
cleanedper-tag readings
Application(s) Application(s)
![Page 12: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/12.jpg)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: Model and BackgroundModel and Background
• Use a binomial sampling model
Time (epochs)
pi
1
0
Smoothing Window
wi Bernoulli trials
piavg
Si
(Read rate of tag i)
E1 E2 E3 E4 E5 E6 E7 E8 E9E0
![Page 13: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/13.jpg)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
• If the tag is there, read it with high probability
Want a large window
pi
1
0
Reading with a low pi
Expand the window
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
![Page 14: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/14.jpg)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: CompletenessCompleteness
Expected epochs needed to read
With probability 1-
Desired window size for tag i
1ln*1
avgi
ip
w
![Page 15: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/15.jpg)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions• Detect transitions as statistically
significant changes in the data
pi
1
0
Statistically significant difference Flag a transition and
shrink the window
The tag has likely left by this point
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
![Page 16: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/16.jpg)
04/22/23
Per-Tag Smoothing: Per-Tag Smoothing: TransitionsTransitions
# expected readings Is the difference
“statistically significant”?# observed
readings
)1(**2|*||| avgi
avgii
avgiii ppwpwS
![Page 17: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/17.jpg)
04/22/23
SMURF in ActionSMURF in ActionFido moving Fido resting
SMURF
Experiments with real and simulated data show similar results
![Page 18: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/18.jpg)
04/22/23
Multi-tag CleaningMulti-tag Cleaning
• Some applications only need aggregates• E.g., count of items on each shelf Don’t need to track each tag!
• Use statistical mechanisms for both:• Aggregate computation • Window adaptation
![Page 19: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/19.jpg)
04/22/23
Aggregate Aggregate ComputationComputation
• –estimators (Horvitz-Thompson) • Count:
• P[tag i seen in a window of size w]:
Use small windows to capture movementUse the estimator to compensate for lost
readings
wSiwN
1
wavgii p )1(1
![Page 20: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/20.jpg)
04/22/23
Window AdaptationWindow Adaptation
• Upper bound window similar to per-tag
• “Transition” based on variance within subwindows
1ln*1
avgpw
CountNw
Nw’
Time (epochs)E1 E2 E3 E4 E5 E6 E7 E8 E9E0
'VarVar2ww NN
![Page 21: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/21.jpg)
04/22/23
Multi-tag ScenarioMulti-tag Scenario
![Page 22: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/22.jpg)
04/22/23
Ongoing Work: Spatial Ongoing Work: Spatial SmoothingSmoothing
• With multiple readers, more complicated
Reinforcement
A? B? A U B? A B?Arbitration
A? C? All are addressed by statistical framework!
U
A
B
C
D
Two rooms, two readers per room
![Page 23: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/23.jpg)
04/22/23
Beyond RFIDBeyond RFID
• -estimator for other aggregates Use SMURF for sensor networks
• Use SMURF in general streaming systems (e.g., TelegraphCQ)
Remove RANGE clause from CQL
Other sensor data
Other streaming data
![Page 24: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/24.jpg)
04/22/23
Related WorkRelated Work
• Commercial RFID middleware• Smoothing filters: need to set smoothing
window• RFID-related work
• Rao et al., StreamClean: complementary• Intel Seattle, HiFi, ESP: static window size
• BBQ, MauveDB• Heavyweight, model-based• SMURF is non-parametric, sampling-based
• Statistical filters (digital signal processing)• Non-linear digital filters inspired SMURF design
![Page 25: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/25.jpg)
04/22/23
ConclusionsConclusions
• Current smoothing filters not adequate• Not declarative!
• SMURF: Declarative smoothing filter• Uses statistical sampling to adapt window size
![Page 26: Adaptive Cleaning for RFID Data Streams](https://reader035.vdocuments.us/reader035/viewer/2022070423/568167f0550346895ddd6165/html5/thumbnails/26.jpg)
04/22/23
Thanks!Thanks!
Questions?