siyapath - a p2p, gossip based volunteer computing framework
DESCRIPTION
Slide deck presented at NBQSA, Sri Lanka - 2012 This project was awarded Gold at this competitionTRANSCRIPT
P2P, Gossip Based Volunteer Computing Framework
SIYAPATH
RISING DEMAND FOR
HIGH COMPUTATIONAL POWER
Simulations
Analysis of large data from optical telescopes
Gene sequencers
Gravitational wave detectors
Edge-detection for high resolution medical image sequences
Biology-inspired algorithms
National Best Quality Software Awards, Sri Lanka - 20122
HOW TO ADDRESS?
Supercomputing
Distributed Computing Cluster Computing
Grid/Desktop-Grid Computing
Volunteer Computing
National Best Quality Software Awards, Sri Lanka - 20123
EXISTING VOLUNTEER
COMPUTING FRAMEWORKS
Client Server Architecture Server bottleneck
Single point of failure
Complexity of Configuring
National Best Quality Software Awards, Sri Lanka - 20124
INTRODUCING SIYAPATH
P2P ArchitectureAbsence of server bottleneck
Absence of single point of failure
Gossip Protocols Scalability
Robustness
Reliability
Simplicity in Configuring
National Best Quality Software Awards, Sri Lanka - 20125
FEATURES
Job Submission
Job Scheduling
Task Distribution
Task Processing
Result Validation
Result Delivery
Fault Tolerance
Security
Job Processor
Task
Processor
Task
Processor
Task
Processor
User
National Best Quality Software Awards, Sri Lanka - 20126
DEMONSTRATION
National Best Quality Software Awards, Sri Lanka - 20127
PERFORMANCE TESTINGTest 1 2 3 4 5
Users 5 10 15 20 25
Volunteers 30 30 30 30 30
Avg. Response Time (s) 52.16 89.63 136.29 211.02 249.25
Avg. Throughput (Tasks per
minute)
126.76 136.45 130.03 109.49 125.96
0
50
100
150
200
250
300
1 2 3 4 5
Tim
e (s
econ
ds)
Test
Average Response Time
0
20
40
60
80
100
120
140
160
1 2 3 4 5
Tas
ks p
er m
inut
e
Test
Average Throughput
National Best Quality Software Awards, Sri Lanka - 20128
BEST PRACTICES
Version control system
Build Automation
Unit / Integrated testing
Logging
Continuous Integration
User friendly Website
National Best Quality Software Awards, Sri Lanka - 20129
OPEN SOURCE COMPONENTS
Services Framework - Apache Thrift Communication via a service interface
Addresses implementation details of the network
P2P nature using thrift client/server
System Monitoring – SIGAR
Apache Maven
Apache Log4j
National Best Quality Software Awards, Sri Lanka - 201210
DOCUMENTATION
User manuals for the framework
User API Docs
Research publications
Project Website
National Best Quality Software Awards, Sri Lanka - 201211
TEAM
Members:Amila Manoj Silva
Nadeeshani Hewage
Mohamed Nufail
Malith Dhanushka
Supervisors:Dr. Srinath Perera
Mrs. Vishakha Nanayakkara
National Best Quality Software Awards, Sri Lanka - 201212
Thank You!
Questions?
National Best Quality Software Awards, Sri Lanka - 201213