the dds benchmarking environment james edmondson [email protected] vanderbilt university...
TRANSCRIPT
2
Topics
• What is the DBE?• What technologies does the DBE use?• How does the DBE setup the environment? • What problems had to be dealt with? • What features will be added? • Demonstration• Conclusion and Wrap Up
3
What is the DBE?
• DDS Benchmarking Environment• Sets up the testing environment and framework• Configures and launches implementation specific services• Launches Publishers and Subscribers• Coordinates interaction, logging info, and stats collection• Allows for easily scaling the number of participants• Sets up the statistics for collection by Bagatel
Thro
ughp
ut Publi
shers
NetworkingSubscribers
4
What is the DBE?
• May be used to test other middleware or technologies• Focus is on documented, reusable resources
Thro
ughp
ut Publi
shers
NetworkingSubscribers
Versatility Dependability
5
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
6
Node 0
DBE
Node 1 Node 2 Node N
What does the DBE do?
7
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
What does the DBE do?
8
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
What does the DBE do?
9
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
ConfigureImplementation
10
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
ConfigureImplementation
Start theSubscribers
11
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
ConfigureImplementation
12
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
ConfigureImplementation
13
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Setup TestEnvironment
ConfigureImplementation
14
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Subscriber Subscriber Subscriber
Setup TestEnvironment
ConfigureImplementation
15
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Subscriber Subscriber Subscriber
Setup TestEnvironment
ConfigureImplementation
Start thePublisher
16
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Subscriber Subscriber Subscriber
Setup TestEnvironment
ConfigureImplementation
17
What does the DBE do?
Node 0
DBE
Node 1 Node 2 Node N
Publisher Subscriber Subscriber Subscriber
Setup TestEnvironment
ConfigureImplementation
18
What technologies does the DBE use?
• Scripts are written in Perl and use PerlAce module• Remote execution is accomplished via SSH• Logs, stats and test info are stored on a NFS partition• All tests are currently ran on Fedora Core Linux machines• Configured to test NDDS, TAO DDS, and Splice• Synchronization between tests is maintained by using
lockfilesDBE
Linux SSH NFS Perl lockfiles
19
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
20
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
21
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
Change $ENV
22
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
Change $ENV
23
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
Change $ENV Generate ID
24
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
Change $ENV Generate ID
25
How does the DBE setup the environment?
• Most path information is updated using the ~/.ssh/environment file
• Some custom environment variables are set by DBE scripts on individual clients
• Individual Test IDs are assigned to each test• Directories are created on the NFS partition for stats and logs
DBE
Change $ENV Generate ID Create Directories
26
How does the DBE setup the environment?
• Publishers and subscribers are notified of test settings– Result file locations
– QoS settings on a per subscriber basis
– Network settings
– Test specific settings (number of messages to send, etc.)
DBE
Repos Publishers Subscribers
Pass Test Info
27
What problems had to be dealt with?
• Lack of a terminal with remote command execution with SSH• TCP and UDP buffer sizes were too small
• Synchronization amongst publishers and subscribers
Subscriber
Subscriber
PublisherW
ait
PublisherBuffer
PublisherBuffer
Next Test
28
What problems had to be dealt with?
• Flock() doesn’t work as expected with stateless NFS
• Shared libraries are referenced locally• Block buffering of stdout
flock(“myfile”) touch(“myfile.lock”)
stdout
stderr
file
fileimmediate
block buffering
29
What problems had to be dealt with?
• More problems on request ;)
30
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
31
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Create Publisher
32
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Create Publisher
33
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher
Create Publisher
34
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher
35
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher
Create Subscriber
36
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher
Create Subscriber
37
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
Create Subscriber
38
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
39
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
Create Subscriber
40
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
Create Subscriber
41
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber Subscriber
Create Subscriber
42
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber Subscriber
43
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber Subscriber
KillSubscriber 1
44
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber Subscriber
KillSubscriber 1
45
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
KillSubscriber 1
46
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
KillSubscriber 1
47
What features will be added?
• New tests will require new features• One such feature is per node scripting of subscribers and
publishers• Feature will allow dynamic starting and stopping of
participantsDBE
Publisher Subscriber
48
What other additions are planned?
• The ability to launch logging services– Memory and CPU usage will need to be recorded
• The ability to test other middleware and applications
49
Demonstration
Primer messages = 100
Stats messages = 1000
Test type = throughput
Implementation = NDDS
50
Conclusion and Wrap Up
• The DBE is an excellent suite of tools for automating DDS testing on a networked environment
• New test types will be added to gain further insight into the DDS implementations
• Many new features will also be added to better reveal problems and benefits of the various DDS products
• We would like to thank our AFRL and Vanderbilt Sponsors for their continued support