Download - Data Acquisition Reference Architecture
![Page 1: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/1.jpg)
![Page 2: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/2.jpg)
Data Acquisition
Reference Architecture
Kitt Farrar
Cell: (513) 894-2321
Developer Education Day 2011
![Page 3: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/3.jpg)
3
Why is Software Important?
Total Measurement Application Cost =
Price of HW + Price of SW + Cost of Development
Visible Price Hidden Costs
Source: National Instruments Survey of Data Acquisition Customers and Prospects 2005
36%,
Price of
HW & SW64%,
Development
Time Costs
![Page 4: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/4.jpg)
4
Oil Well Fracture Pump Monitoring and Analysis Using NI LabVIEW
Application: A system that monitors the operating parameters of a reciprocating pump used in well servicing applications.
Challenge: Building an advanced monitoring system that can survive being mounted directly to an oil well servicing pump in a rugged environment while performing advanced analysis on sensor data.
Products: NI CompactRIO, LabVIEW, LabVIEW Real-Time and LabVIEW FPGA modules, and NI Single-Board RIO
Key Benefit: The high-speed I/O and analysis catch the momentary pressure spikes and vibration indications of the pumps.
“LabVIEW has made the software development side much quicker than our past experiences in C-based
programming. What most C programmers take two years to do, we can accomplish in a couple of months.
We can use that time savings to get to market quicker and capitalize on our competitors‟ lag time.”
– Robert Stewart, Supreme Electrical Services Inc.
![Page 5: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/5.jpg)
5
Agenda• Framework Definition
• Framework Outline
• Demo - Overview
• Framework Modules
• Demo – Adding Your Own Display Windows
• Additional Systems Engineering Components
• Networking Your Data Display
• Substitute Other Options for Logging To Disk
• References
![Page 6: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/6.jpg)
6
Data Acquisition Reference Design
• Open framework: use as-is or custom tailor
• Core functionality:
Configuring Data Acquisition
Acquiring Data
Logging Data to Disk
Displaying Data
• Leverages several SE software components
![Page 7: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/7.jpg)
7
1. Handle all GUI events from the main user interface.
2. Process messages from user interaction or other external processes. Additional
display windows or „plug-ins‟ can be launched from this process.
3. Acquire data continuously. Data is sent via a queue to the data logging process and is
also published via a notifier for any display windows.
4. Log data.
Four Processes
![Page 8: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/8.jpg)
8
Inter-Loop Communication
• Two types of data passed between processes:
Commands
Data
• Commands are sent using the Asynchronous
Message Communication (AMC) library
• Time Domain Data is sent using two methods:
Queues (lossless)
Notifiers (potentially lossy)
![Page 9: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/9.jpg)
9
![Page 10: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/10.jpg)
10
Demonstration
![Page 11: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/11.jpg)
11
Source Code Organization
• Multiple Project Library
(.lvlib) files
Protects VI namespaces
Defines VI scope (i.e.
public or private)
Organizes the VIs
![Page 12: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/12.jpg)
12
Configuration Module
• Provides the user a way to configure the system
via…
Front Panel Interaction
Loading saved configurations from file
Manipulating configurations currently in memory
![Page 13: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/13.jpg)
13
Data Acquisition Module
• Queue-driven State Machine with 4 states
Idle
Start
Acquire
Stop
• Acquires data from the configured source:
Hardware
Simulated Hardware
File
![Page 14: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/14.jpg)
14
![Page 15: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/15.jpg)
15
Data Logging Module
• Also a queue-driven state machine
• Saves time domain data to disk using a lossless
queue buffering mechanism
![Page 16: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/16.jpg)
16
Display Windows Module
• Launches windows as reentrant VIs using VI
server from the Main Message Processor
• Display windows are all Queued State Machines
• Add your own display windows using the Display
Template.vit
![Page 17: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/17.jpg)
17
![Page 18: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/18.jpg)
18
Demonstration:
Adding Your Own Display Windows
![Page 19: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/19.jpg)
19
Additional SE Software Components
• Asynchronous Message Communication
Sending commands between processes
• Channel Monitor XControl
Intuitive front panel channel monitoring utility
• FlexSignal Reference Library
Acquiring from real H/W, simulated H/W, or a file
• GXML Reference Library
Stores system configuration information in XML format
![Page 20: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/20.jpg)
20
Asynchronous Message Communication
• Library for sending messages
Within a Process
Between Processes
![Page 21: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/21.jpg)
21
Creating the Message Queue and
Inserting Messages
![Page 22: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/22.jpg)
22
Processing Messages
![Page 23: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/23.jpg)
23
Networking Your Data Display
• One to Many
LabVIEW Remote Front Panels or
LabVIEW Web Services
• One to One
Network Streams
STM (Simple TCP Messaging)
![Page 24: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/24.jpg)
24
Network Streams (LabVIEW 2010)
• New API for high-throughput streaming over the
network
Open/Read/Write/Close API
Multi-element read/write supported
• Supported on LabVIEW Real-Time and Windows
• Performance is comparable to TCP
![Page 25: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/25.jpg)
25
Seamlessly Stream Data Over the Network
![Page 26: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/26.jpg)
26
Pitfalls of Streaming with Variables
Machine 1 Machine 2
Server
Client Writers
Client Readers
1 2 3 4 1 2 3 4
5 6 7
5 6 745 6 7
• Lack of flow control can result in data loss
• Data may be lost if the TCP/IP connection is dropped
• Data loss does not result in an error, only a warning
1 2 3 4
![Page 27: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/27.jpg)
27
Network Streams in Action
Machine 1 Machine 2
Writer Endpoint Reader Endpoint
123 3 1245 12345
Acknowledge (3)
123 45 45 45 45
Acknowledge (2)
12 345
Flow Control (2)
6789 6789
![Page 28: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/28.jpg)
28
Options for Logging to Disk
1. ASCII
2. Binary
3. TDMS
![Page 29: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/29.jpg)
29
ASCII: Characteristics
• Human readable text file
• Data represented as string
• Common for low speed DAQ
![Page 30: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/30.jpg)
30
ASCII: Write to File
Convert to String
![Page 31: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/31.jpg)
31
Binary: Characteristics
• Not human readable
• Common for high speed DAQ
• Common for multiple channel DAQ
![Page 32: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/32.jpg)
32
Binary: Write to File
![Page 33: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/33.jpg)
33
TDMS: Characteristics
• NI’s recommended file format
• Binary header
• Binary data storage
• Optimized for high-speed streaming
• Versatile data buffers
• Recommended for all DAQ
![Page 34: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/34.jpg)
34
TDMS: Write to File with Primitives
![Page 35: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/35.jpg)
35
TDMS: Create & Set Properties
![Page 36: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/36.jpg)
36
TDMS: Write Data & Close
![Page 37: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/37.jpg)
37
Benchmarks: Performance Comparison
20 Chunks of 10,000 Bytes Each
![Page 38: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/38.jpg)
38
Benchmarks: Size Comparison
20 Chunks of 10,000 Bytes Each
![Page 39: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/39.jpg)
39
Examples
1. Writing data to an ASCII file with timestamps
2. Writing waveform data to file with timestamps
3. Using the Microsoft Report Generation Toolkit
![Page 40: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/40.jpg)
40
For Further Reference
• Data Acquisition Reference Design for LabVIEW
http://zone.ni.com/devzone/cda/tut/p/id/11805
• Data Acquisition Reference Application (The Code)
http://zone.ni.com/devzone/cda/epd/p/id/6438
Refer to the Data Acquisition Reference Application link
for hyperlinks to the additional reference software
libraries that you will need.
![Page 41: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/41.jpg)
41
Additional References
• TDM Streaming (TDMS) VIs
http://zone.ni.com/devzone/cda/tut/p/id/3539
• TDMS File Format Internal Structure
http://zone.ni.com/devzone/cda/tut/p/id/5696
• TDMS Excel Add-In for Microsoft Excel Download
http://zone.ni.com/devzone/cda/epd/p/id/2944
• Creating a Report in Microsoft Excel Using the LabVIEW
Report Generation Toolkit
http://zone.ni.com/devzone/cda/tut/p/id/3178
![Page 42: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/42.jpg)
42
Review
• Framework Definition
• Framework Outline
• Demo - Overview
• Framework Modules
• Demo – Adding Your Own Display Windows
• Additional Systems Engineering Components
• Networking Your Data Display
• Other Options for Logging To Disk
• References
![Page 43: Data Acquisition Reference Architecture](https://reader031.vdocuments.us/reader031/viewer/2022013114/543d0e3eafaf9fbe618b4965/html5/thumbnails/43.jpg)
Data Acquisition
Reference Architecture
Kitt Farrar
Cell: (513) 894-2321
Developer Education Day 2011
Questions?