social cloud: facilitating “trustworthy” compute & data resource sharing presented by daniel...
TRANSCRIPT
Social Cloud: Facilitating “Trustworthy” Compute
& Data Resource Sharing
Presented by Daniel S. Katz*
Most work performed by:Kyle Chard, Simon Caton, Omer Rana, Kris Bubendorfer,
Ioan Petri, Ivan Rodero, Magdalena Punceva, Manish Parasharhttp://www.facebook.com/SocialCloudComputing
4th GEOSS Science and Technology Stakeholder Workshop
*Work by Katz was supported by the National Science Foundation while working at the Foundation. Any opinion, finding, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
How does resource sharing happen today
Required:
Available:
Where to get these resources?
Buy servers
Grid
Cloud
Volunteer1,2
Scenario:A user wants to backup data, but doesn’t have the required amount of computational resources/storage capacity
1: Anderson 2002; 2: Stainforth 2002
Existing approaches all have certain shortcomings
Use the (spare) capacity of friends and extend the Volunteer Computing approach to bilateral and multilateral exchanges?
Buy servers
Grid
Cloud
Volunteer
Data control
Willingness to provide resources
Access to (heterogeneous) resources
Resources available on demand
Costs, inflexibility
Exchange not necessarily bilateral
QoS, user-friendliness, access
Trust assumptions
Many years of Cloud computing … and the same old hurdles
• Security (still a major limitation)• Lack of Customisability• Economics
– Small scale consumers have requirements that may not match providers offerings
– Providers have explicit incentives to lock in consumers
– Still no true open cloud market
• Trust– Always assumed at some level– Can have anonymity
(Market-based/broker allocation)• But has new levels of trust needed
– Many models fall apart when trust is not assumed
The vision of a social cloud
• Definition: (based on Chard et al. 2011)“Social Clouds are a scalable, dynamic and user-centric resource sharing framework in which computational resources, services and information are shared amongst members on the premise of the relationships encoded in a social network.”
Socially-oriented Sharing Platform
Social Clouds enable the sharing of (heterogeneous) resources in a framework where the social structures infer an implicit level of trust
Social Cloud – building on existing Social Network Platforms
Users contribute to “good” causes
Resources are idle 40-95%1,000,000,000
Users
On average 190 friends
Ubiquitous: Facebook ~ 1.3B users (Q2, 2014)Represent mostly pre-existing real world relationshipsHave notions of pre-existent trust fabric inherently interwoven into the network structureMany applications now use social networks as a platform for:
Authentication e.g. Facebook ConnectOnline Presence e.g. fb.com/your_page, Google Places (API)Application Portals e.g. progress thru processors, ASPEN and PolarGrid project
Social Clouds ... Via Edge Devices, not Data Centers
• Cloud computing is built around large datacenters– Distributed, but still built with
capacity limits
• Expanding the boundaries of clouds across edge devices
• Devices can vary in scope:– Set-top boxes(from 500 GB to 1 TB of storage)– Media Centres (Home PCs)– Can use content distribution
networks (Netflix, HTTP, YouTube)
21.04.23 7
Social Clouds: two aspects/research challenges
• Resource management– Edge Devices – with Peer-
2-Peer-like interaction– Means of content
distribution– Provides basis for a
“credits” and revenue model
– Could be via social group or facilitated by content provider (e.g. Virgin Media)
• Access– Use of Social Media (e.g.
Facebook) app. to utilise friends network
– Use trust relationships within an existing social network to discover partners for interaction
– Use of a “virtual currency” (credit tokens) that facilitates trading within a local community
Social Clouds: types of data(e.g. podcasts, blogs, photos, music, documents)
• User generated content may be:– Intermittent - often required at different time
intervals but not continuously– Temporary - required for a short period of time
(e.g. processing memory for running an experiment) and often only once
– Backup - required with the highest security implications and privacy
– Working - can be accessed in real time and continuously
Incentives
Family, close friends
Friends, Colleagues
Other People
Different groups have different levels of perceived trustChoice of interaction mechanism could depend on relationship type
Example:•Non-monetary incentives for close friends•Monetary incentives for other people
Volunteer
Trophy
Posted Price
Reciprocal
Auction
Relationship to GEOSS(http://www.earthobservations.org/geoss.php)
• Variety of data sets– at different locations (“a network of content providers”)
• Linking “system of systems”• Enable multiple types of specialist, application specific,
decision support – Community provision of algorithms + “shim” (translation
services)
• Trust issues seem to be key– Who: (i) generates, (ii) hosts; (iii) caches the data set– Provenance issues associated with data– Hosted on devices with varying capability (and reliability/
availability profiles)Similar requirements to Social Clouds
Investigations/Talk Structure
• Social Cloud platform– Data Storage: Facebook app.– Computational: Seattle-based VMs – Use of Social Clouds to support Content Distribution (user-
side version of the “Cloud Files” implementation)• Resource trading scenarios
– Eigentrust (direct interaction + recommendations and feedback)
– Complementary (“virtual”) currency• Trust relationships (incentive models)
– Game theoretic models (incentives to provide incorrect feedback) + number of malicious peers
The Platform
Data Storage: Facebook app.
Platform Architecture
Social MarketplaceMatching supply with demandProtocols for resource allocation, rules of exchange, information store/registry
Platform ManagerOverall system coordination
Socio-Technical AdapterIdentity verification (e.g. OAuth)Connect to various “types” of social networks (DBLP, Facebook, Foursquare, etc)
Resource FabricsVirtualisation & Sandboxing mechanismIntegration with Globus end points, Seattle VM, etc
Social Cloud: Platform for Data Storage (Demonstrator)
• Simple Storage Service Implemented as a Facebook application• Use Case: a back up facility
Agreement
Kyle Chard, Kris Bubendorfer, Simon Caton, Omer F. Rana, “Social Cloud Computing: A Vision for Socially Motivated Resource Sharing”. IEEE Transactions on Services Computing 5(4): 551-563 (2012)
Posted Price
Storage
Social CloudSocial Cloud
Storage
Storage
MDSMDS
User ID URL Capacity Price
User1 100 MB 5
User2 500 MB 10
User3 5 GB 7
– Enables interactions based upon active trading/collaborative decisions
– Intuitively facilitates reciprocal collaboration– Current “norm” in industry solutions
Dynamic Auctions
• Auction:– Enables dynamic participant pairing– Sealed bid second price reverse auction
• Could be extended to any other auction mechanism
Application
Posted Price Scalability
• Varying the size of the MDS and number of matches• With a size of 2000, 100 matches can be discovered
in ~ 2 seconds
The Platform
Computational: Seattle-based VMs
Processor Sharing – via extended Seattle VMs
• Matching between users & owners• Seattle (https://seattle.poly.edu/html/ ) – Open P2P platform
– Seattle “Clearing house” mechanism. 10 “vessels” (VMs) for each new install– Node Manager: gatekeeper for resources deployed on every contributed resource (credential checking for VM
interaction)– Host machine location (in a lookup service) + Public/Private keys generated– Repy (Reduced Python for sandboxed environments)
Processor Sharing – via extended Seattle VMs
• Identify list of donation nodes• Filter list based on “friends list” for a particular user • Match mechanism
– Select consumer preferences for each friend– Select preferences for each friend for requesting user
• Extends Seattle’s implementation of (pseudo) random allocation to reduce user/donation permutations
Simon Caton, Christian Haas, Kyle Chard, Kris Bubendorfer, Omer F. Rana, “A Social Compute Cloud: Allocating and Sharing Infrastructure Resources via Social Networks”. IEEE Transactions on Services Computing 7(3): 359-372 (2014)
The Platform
Use of Social Clouds to support Content Distribution (user-side version of the “Cloud Files” implementation)
Social CDN – Use Case
• Enable content distribution through user-supplied storage– Home users (DSL-based)– National Labs (dedicated networks) – often related to a
particular project (e.g. D0)• Variable availability profiles
– Bandwidth throttling with DSL-based set up– Establish a VPN between contributing sites (use of
SocialVPN)• Identify:
– Number of replicas needed to ensure availability• Served-based approach
– “CloudFiles” in RackSpace (OpenStack) – use of Akamai CDN
A Social Content Delivery Network for Scientific Cooperation
Replica Placement:•Random•Node Degree: highest no. of edges•Community Node Degree (highest degree within a community, i.e. no adjacent placement)•Clustering Coefficient (similar to highest betweenness scores)
Scenario and Community Representation
26
• Baseline Graph: DBLP publications graph (Kyle): 3 degrees (2009-10)– Nodes: authors, Edges: coauthorship of 1 or more papers
• Double co-authorship: at least 2 publications• No. of Authors: < 6 authors on the paper• Trust: captured through prior collaborative work• Having constructed a network, we assign replicas, and then test with publications
from 2011
Kai Kugler, Simon Caton, Kyle Chard, Daniel S. Katz, "On Replica Placement in a Social CDN for e-Science,” 10th IEEE International Conference on eScience, 2014.
Double Coauthorship No. of Coauthors
Results (at least 60 repetitions)
27
Kyle Chard, Simon Caton, Omer Rana and Daniel S. Katz, “A Social Content Delivery Network for Scientific Cooperation: Vision, Design, and Architecture”, 3rd Int. Workshop on Data Intensive Computation in the Cloud (DataCloud), in conjunction with ACM/IEEE SC12 conference, Salt Lake City, November 2012
Data “Followers” & “Survivability” dynamics(The “Data Wildfire”)
• Register interest in a data set– Equivalent to “Like” (Facebook) and “Favorite”
(Twitter) – Event generated on subsequent update on a data set
• Enable “interesting” data set to be propagated– Equivalent to a “Share” (Facebook) and “Retweet”
(Twitter)– Enables data sets with community interest to become
popular over time• Can be useful as a basis to support resource allocation
Finding Brokers
Broker Emergence
• Finding suitable providers– Centralized (registry/index)– Distributed (flooding, gossip protocol, federated registry
services)• Brokers can utilise both centralized and distributed
capability– Brokers influence interaction dynamics in the network– Predefine broker nodes at start up
• How can trading within a Social Cloud be enhanced by dynamic emergence of “brokers”– Which nodes could be more useful Brokers?
Take away message
• Dynamic selection of brokers based on their position in the network– Incentivise nodes to become broker based on increase in revenue– Role adaptation in the network – buyers/seller broker– Use of “influence” metrics based on a social score
• Dynamic network properties may lead to limited benefit with pre-defined brokers
• the social score algorithm generates a higher volume of trades than the dominating set algorithm.
• The performance differences of Social Score vs Dominating set are determined by two aspects:– Graph properties and– Choice of evaluation metrics.
Ioan Petri, Magdalena Punceva, Omer F. Rana, George Theodorakopoulos: Broker Emergence in Social Clouds. IEEE CLOUD 2013, San Jose, CA, USA, pp 669-676
Incentive Models
Incentives & Trading
In Conclusion
• Social Clouds provide an important user-driven alternative to data-center based Clouds– E.g., Wuala networks (NL), AmazingStore (China), etc
• Issues of Trust, Reputation and Economic incentives are key– Include other factors: availability, reliability, uptime, power
usage, etc.– Softer than traditional Service Level Agreement model
• Current focus: Broker “emergence” in Social Clouds– Identify dominating sets in a social graph– Implementation using CometCloud
Further Reading
D. Neumann, C. Bodenstein, O. F. Rana, R. Krishnaswamy, ”STACEE: Enhancing Storage Clouds using Edge Devices”.IEEE/ACM Workshop on Autonomic Computing for Economics (ACE 2011) alongside ICAC 2011, Karlsruhe, Germany, 14 June 2011. ACM Press.
Kyle Chard, Kris Bubendorfer, Simon Caton, Omer Rana, “Social Cloud Computing: A Vision for Socially Motivated Resource Sharing,” IEEE Transactions on Services Computing, 2011.
Ioan Petri, Omer Rana, Gheorghe Cosmin Silaghi: “SLA as a Complementary Currency in Peer-2-Peer Markets”, Proceedings of GECON 2010: 141-152, Springer Verlag.
“Trust Modelling and Analysis in Peer-to-Peer Clouds” Ioan Petri, Omer Rana, Yacine Rezgui, and Gheorghe Cosmin Silaghi, Journal of Cloud Computing, Inderscience, 2012
Omer Rana and Simon Caton, ”Business Models for On-line Social Networks: Challenges and Opportunities”, International Journal of Virtual Communities and Social Networking, July-September 2010, Vol.2, No.3, pp 31-42
Ioan Petri, Omer F. Rana, Gheorghe Cosmin Silaghi: Service level agreement as a complementary currency in peer-to-peer markets. Future Generation Comp. Syst. 28(8): 1316-1327 (2012)