AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

AI Buzzwords Explained: Distributed Constraint OptimizationProblemsFerdinando Fioretto (University of Michigan; fioretto@umich.edu)William Yeoh (Washington University in St. Louis; wyeoh@wustl.edu)DOI: 10.1145/3175502.3175506

The power network is the largest operat-ing machine on earth, generating more thanUS$400bn a year1 keeping the lights on forour homes, offices, and factories. A signifi-cant concern in power networks is for the en-ergy providers to be able to generate enoughpower to supply the demands at any point intime. Short terms demand peaks are how-ever hard to predict and, thus, in the modernsmart electricity grid, the energy providers canexploit the demand-side flexibility of the con-sumers to reduce the peaks in load demand.

This control mechanism is called Demand-side management (DSM). DSM can be ob-tained by scheduling shiftable loads (i.e., aportion of power consumption that can bemoved from a time slot to another) from peakto off-peak hours (Fioretto, Yeoh, & Pontelli,2017; Logenthiran, Srinivasan, & Shun, 2012;Voice, Vytelingum, Ramchurn, Rogers, & Jen-nings, 2011). In a simplified version of thisproblem, the energy provider has a desiredmaximal amount of power that it can gener-ate and, thus, use to serve its customers.When the predicted amount of customer loadsexceed such amount, the provider has toreschedule some of these loads in differenttime slots to satisfy the constraint on the max-imum power capacity.

Such an approach, however, requires theprovider to control a portion of the consumer’selectrical appliances, affecting privacy andusers’ autonomy. On the other hand, res-idential and commercial buildings are pro-gressively being partially automated, throughthe introduction of smart devices (e.g., smartthermostats, circulator heating, washing ma-chines). Household penetration is at 5.8% in2016 and is expected to hit 18.6% in 2020.2

Copyright c© 2018 by the author(s).1U.S. Energy Information Administration2https://www.statista.com/outlook/

279/109/smart-home/united-states#market-driver

Device scheduling can be executed by theusers, without the control of a centralized au-thority, and a coordinated device schedulingwithin a neighborhood of buildings can beused as a DSM strategy, preserving user dataprivacy. Figure 1 illustrates such scenario.

One possible way to solve this problem isthrough the use of Distributed Constraint Op-timization Problems (DCOPs) (Fioretto, Pon-telli, & Yeoh, 2016; Modi, Shen, Tambe, &Yokoo, 2005; Petcu & Faltings, 2005a). DCOPalgorithms are a class of distributed cooper-ative multi-agent algorithms in which severalautonomous agents coordinate their decisionsto achieve a shared goal while accounting forpersonal preferences. The agents can bethought of as software programs whose exe-cution does not depend on the execution ofother agents. Their actions are expressed us-ing the concept of variables, i.e., abstract en-tities that can take one out of several values(describing the possible set of actions for theagent). Each agent needs to decide whichvalue to assign to its variables. The outcomeof an action is expressed in terms of a cost (orreward) and typically depends on the joint ac-tion of multiple agents. The goal of a DCOP isexpressed in the form of an objective functionto be minimized (or maximized). To coordinatetheir actions, agents employ a message pass-ing mechanism realized through a networkedcommunication.

Mathematically, a DCOP is composed by thefollowing entities:

• A = {a1, . . . , ap}: The set of autonomousagents participating in the problem.• X = {x1, . . . , xn}: The set of variables in

the problem. Each variable is controlled byexactly one agent.• D = {D1, . . . , Dn}: The domains for the

variables in X , where Di represents the setof possible values that the variable xi maybe assigned.

8

AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

Figure 1: An illustration of a smart neighborhood with each home controlling a set of smart devices.

• C = {c1, . . . , ce}: The set of problem con-straints. Each constraint ci is a function thatinvolves (multiple) variable(s) from X andassociates a cost for each combination oftheir value assignments.• α : X → A: A mapping that associates

variables to agents, expressing which agentcontrols which variables.

The goal of the problem is to find an assign-ment for the agent variables that minimizes thesum of all costs over all constraints. Since theagents are physically distributed across a net-work, all communication take the form of mes-sages. Thus, agents coordinate the value as-signment for their variables following a givendistributed protocol. In addition, agents knowl-edge is limited to their resources: each agentknows exclusively the outcomes of the vari-ables it controls and the constraints it shareswith some other agents. This scheme is ef-fective to design algorithms that preserve dataprivacy.

a1a2

a3 a4

x1x4

x3x2

x4x1 x2 x3 < L+ + +

Figure 2: An illustration of a smart neighbor witheach home controlling a set of smart devices.

In the DSM smart device scheduling exam-

ple in Figure 2, we illustrate a smart neigh-borhood composed of 4 homes, each repre-sented by one agent: a1, . . . , a4. In this simpli-fied scenario, each agent controls one singlevariable x1, . . . , x4, which represents a simpli-fied electrical appliance that can be switchedoff (consuming 0 kWh) or on (consuming 2kWh). Assume the energy provider imposesa total consumption limit of 4 kWh. We repre-sent the domains of each variable with the set{0, 2}, indicating the consumption associatedwith the device’s actions. The only constraintof the problem is expressed with the formulax1 + x2 + x3 + x4 ≤ 4, meaning that the ag-gregated energy consumption cannot exceed4 kWh. A possible solution to the problem isthus having agents a1, and a2 switching theirappliances on, and thus consuming a totalof x1 = 2 + x2 = 2 = 4 kWh, and agentsa3 and a4 switching their appliance off, thusconsuming 0 kWh. A more detailed descrip-tion of the DSM scheduling application and itscorresponding DCOP model can be found in(Fioretto et al., 2017; Tabakhi, Le, Fioretto, &Yeoh, 2017).

The DCOP framework is general and offers aflexible tool to model a wide variety of prob-lems. Examples of use of DCOPs to solvedistributed problems include service-orientedcomputing, that relies on sharing resourcesover a network, focusing on maximizing theeffectiveness of the shared resources whichare used by multiple applications (Choudhury,Dey, Dutta, & Choudhury, 2014; Jin, Cao, &Li, 2011; Li, Wang, Ding, & Li, 2014), sen-sor network problems, which consist of coor-dinating a large number of inexpensive andautonomous sensor nodes, constrained bya limited communication range and battery

9

AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

life (Hosseini Semnani & Basir, 2013; Ota,Matsui, & Matsuo, 2009; Stranders, Farinelli,Rogers, & Jennings, 2009; Zhang, Wang,Xing, & Wittenberg, 2005), and many oth-ers (Brys, Pham, & Taylor, 2014; Gaudreault,Frayret, & Pesant, 2009; Junges & Bazzan,2008; Kumar, Faltings, & Petcu, 2009; Miller,Ramchurn, & Rogers, 2012; Rust, Picard, &Ramparany, 2016; Yeoh & Yokoo, 2012; Zi-van, Yedidsion, Okamoto, Glinton, & Sycara,2015). More examples can be found in a re-cent survey (Fioretto, Pontelli, & Yeoh, 2016).

It turns out that it is difficult (NP-hard) tooptimally solve this kind of problems, andthat there is a close relationship betweenthe amount of information that needs tobe encoded in a message (message size)vs. the number of messages exchanged bythe agents (network load). Thus, an exten-sive piece of the DCOP literature focuses onthe study of algorithms that trade off solutionquality for faster runtime and reduced use ofnetwork resources (Farinelli, Rogers, Petcu,& Jennings, 2008; Fioretto, Yeoh, & Pontelli,2016; Maheswaran, Pearce, & Tambe, 2004;Nguyen, Yeoh, & Lau, 2013; Ottens, Dimi-trakakis, & Faltings, 2017; Pearce & Tambe,2007; Petcu & Faltings, 2007a; Yeoh, Sun, &Koenig, 2009; Zhang et al., 2005).

As one of the motivations for the use ofDCOPs is the preservation of privacy, thereis also a large body of work on privacy-preserving algorithms (Grinshpoun & Tassa,2016; Leaute & Faltings, 2011a, 2013; Tassa,Grinshpoun, & Zivan, 2017; Tassa, Zivan, &Grinshpoun, 2016).

Additionally, the DCOP model has also beenextended to handle problems where agentshave multiple objectives (Delle Fave, Stran-ders, Rogers, & Jennings, 2011; Matsui,Silaghi, Hirayama, Yokoo, & Matsuo, 2012),problems of a dynamic nature (i.e., where theproblem changes over time) (Hoang et al.,2016, 2017; Nguyen, Yeoh, Lau, Zilberstein, &Zhang, 2014; Petcu & Faltings, 2005b, 2007b;Yeoh, Varakantham, Sun, & Koenig, 2015),and problems with uncertainty (i.e., whereconstraint costs depend from uncertain fac-tors, such as weather) (Atlas & Decker, 2010;Le, Fioretto, Yeoh, Son, & Pontelli, 2016;Leaute & Faltings, 2011b; Nguyen, Yeoh, &Lau, 2012; Stranders, Delle Fave, Rogers, &

Jennings, 2011).

A forum for discussion on DCOP algorithmsand applications has been held at the Op-timization in Multi-Agent Systems (OptMAS)workshop since 2010 at the International Con-ference on Autonomous Agents and Multia-gent Systems (AAMAS). Recent dissertationswithin the past 5 years include (Billiau, 2015;Delle Fave, 2012; Fioretto, 2016; Grubshtein,2012; Guttierez, 2012; Hanada, 2017; Hatano,2013; Kim, 2015; Miller, 2014; Netzer, 2015;Okimoto, 2012; Ottens, 2012; Ueda, 2014;Yedidsion, 2015).

ReferencesAtlas, J., & Decker, K. (2010). Coordina-

tion for uncertain outcomes using dis-tributed neighbor exchange. In Proceed-ings of the international conference onautonomous agents and multiagent sys-tems (pp. 1047–1054).

Billiau, G. (2015). An equitable approachof solving distributed constraint optimiza-tion problems (PhD thesis). University ofWollongong, Wollongong (Australia).

Brys, T., Pham, T. T., & Taylor, M. E. (2014).Distributed learning and multi-objectivityin traffic light control. Connection Sci-ence, 26(1), 65–83.

Choudhury, B., Dey, P., Dutta, A., & Choud-hury, S. (2014). A multi-agent based op-timised server selection scheme for SOCin pervasive environment. In Advances inpractical applications of heterogeneousmulti-agent systems (pp. 50–61).

Delle Fave, F. M. (2012). Theory and practiceof coordination algorithms exploiting thegeneralised distributive law (PhD thesis).University of Southampton, Southamp-ton (UK).

Delle Fave, F. M., Stranders, R., Rogers, A.,& Jennings, N. (2011). Bounded decen-tralised coordination over multiple objec-tives. In Proceedings of the internationalconference on autonomous agents andmultiagent systems (pp. 371–378).

Farinelli, A., Rogers, A., Petcu, A., & Jen-nings, N. (2008). Decentralised coordi-nation of low-power embedded devicesusing the Max-Sum algorithm. In Pro-ceedings of the international conferenceon autonomous agents and multiagentsystems (pp. 639–646).

10

AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

Fioretto, F. (2016). Exploiting the struc-ture of distributed constraint optimizationproblems with applications in smart grids(PhD thesis). New Mexico State Univer-sity, Las Cruces (United States).

Fioretto, F., Pontelli, E., & Yeoh, W. (2016).Distributed constraint optimization prob-lems and applications: A survey. CoRR,abs/1602.06347 .

Fioretto, F., Yeoh, W., & Pontelli, E. (2016).Multi-variable agent decomposition forDCOPs. In Proceedings of the AAAIconference on artificial intelligence (pp.2480–2486).

Fioretto, F., Yeoh, W., & Pontelli, E. (2017). Amultiagent system approach to schedul-ing devices in smart homes. In Proceed-ings of the international conference onautonomous agents and multiagent sys-tems (pp. 981–989).

Gaudreault, J., Frayret, J.-M., & Pesant, G.(2009). Distributed search for supplychain coordination. Computers in Indus-try , 60(6), 441–451.

Grinshpoun, T., & Tassa, T. (2016). P-SyncBB:A privacy preserving branch and boundDCOP algorithm. Journal of Artificial In-telligence Research, 57 , 621–660.

Grubshtein, A. (2012). Distribute search byagents with personal preferences (PhDthesis). Ben-Gurion University of theNegev, Beer-Sheva (Israel).

Guttierez, P. (2012). Distributed con-straint optimization related with soft arcconsistency (PhD thesis). Universi-tat Autonoma de Barcelona, Bellaterra(Spain).

Hanada, K. (2017). Distributed lagrangianrelaxation protocol and its applications(PhD thesis). Kobe University, Kobe(Japan).

Hatano, D. (2013). Solutions for constraintproblem under the dynamic/distributedenvironment (PhD thesis). Kobe Univer-sity, Kobe (Japan).

Hoang, K. D., Fioretto, F., Hou, P., Yokoo, M.,Yeoh, W., & Zivan, R. (2016). Proactivedynamic distributed constraint optimiza-tion. In Proceedings of the internationalconference on autonomous agents andmultiagent systems (pp. 597–605).

Hoang, K. D., Hou, P., Fioretto, F., Yeoh, W.,Zivan, R., & Yokoo, M. (2017). Infinite-horizon proactive dynamic DCOPs. In

Proceedings of the international confer-ence on autonomous agents and multia-gent systems (pp. 212–220).

Hosseini Semnani, S., & Basir, O. A. (2013).Target to sensor allocation: A hierarchi-cal dynamic distributed constraint opti-mization approach. Computer Commu-nications, 36(9), 1024–1038.

Jin, Z., Cao, J., & Li, M. (2011). Adistributed application component place-ment approach for cloud computing envi-ronment. In Proceedings of the interna-tional conference on dependable, auto-nomic and secure computing (pp. 488–495).

Junges, R., & Bazzan, A. L. (2008). Eval-uating the performance of DCOP algo-rithms in a real world, dynamic problem.In Proceedings of the international con-ference on autonomous agents and mul-tiagent systems (pp. 599–606).

Kim, Y. (2015). Application of techniquesfor MAP estimation to distributed con-straint optimization problem (PhD the-sis). University of Massachusetts atAmherst, Amherst (United States).

Kumar, A., Faltings, B., & Petcu, A. (2009).Distributed constraint optimization withstructured resource constraints. In Pro-ceedings of the international conferenceon autonomous agents and multiagentsystems (pp. 923–930).

Le, T., Fioretto, F., Yeoh, W., Son, T. C., & Pon-telli, E. (2016). ER-DCOPs: A frame-work for distributed constraint optimiza-tion with uncertainty in constraint utilities.In Proceedings of the international con-ference on autonomous agents and mul-tiagent systems (pp. 606–614).

Leaute, T., & Faltings, B. (2011a). Coordinat-ing logistics operations with privacy guar-antees. In Proceedings of the interna-tional joint conference on artificial intelli-gence (ijcai) (pp. 2482–2487).

Leaute, T., & Faltings, B. (2011b). Distributedconstraint optimization under stochasticuncertainty. In Proceedings of the aaaiconference on artificial intelligence (aaai)(pp. 68–73).

Leaute, T., & Faltings, B. (2013). Protect-ing privacy through distributed computa-tion in multi-agent decision making. Jour-nal of Artificial Intelligence Research, 47 ,649–695.

11

AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

Li, X., Wang, H., Ding, B., & Li, X.(2014). MABP: an optimal resource al-location approach in data center net-works. Science China Information Sci-ences, 57 (10), 1–16.

Logenthiran, T., Srinivasan, D., & Shun, T.(2012). Demand side management insmart grid using heuristic optimization.IEEE Transactions on Smart Grid , 3(3),1244–1252.

Maheswaran, R., Pearce, J., & Tambe, M.(2004). Distributed algorithms for DCOP:A graphical game-based approach. InProceedings of the international confer-ence on parallel and distributed comput-ing systems (pp. 432–439).

Matsui, T., Silaghi, M., Hirayama, K., Yokoo,M., & Matsuo, H. (2012). Distributedsearch method with bounded cost vec-tors on multiple objective DCOPs. In Pro-ceedings of the principles and practice ofmulti-agent systems (pp. 137–152).

Miller, S. (2014). Decentralised coordinationof smart distribution networks using mes-sage passing (PhD thesis). University ofSouthampton, Southampton (UK).

Miller, S., Ramchurn, S. D., & Rogers, A.(2012). Optimal Decentralised Dispatchof Embedded Generation in the SmartGrid. In Proceedings of the internationalconference on autonomous agents andmultiagent systems (pp. 281–288).

Modi, P., Shen, W.-M., Tambe, M., & Yokoo,M. (2005). ADOPT: Asynchronous dis-tributed constraint optimization with qual-ity guarantees. Artificial Intelligence,161(1–2), 149–180.

Netzer, A. (2015). Distributed constraint op-timization, from utilitarianism to fairness(PhD thesis). Ben-Gurion University ofthe Negev, Beer-Sheva (Israel).

Nguyen, D. T., Yeoh, W., & Lau, H. C. (2012).Stochastic Dominance in StochasticDCOPs for Risk-sensitive Applications.In Proceedings of the international con-ference on autonomous agents and mul-tiagent systems (pp. 257–264).

Nguyen, D. T., Yeoh, W., & Lau, H. C. (2013).Distributed Gibbs: A memory-boundedsampling-based DCOP algorithm. InProceedings of the international confer-ence on autonomous agents and multia-gent systems (pp. 167–174).

Nguyen, D. T., Yeoh, W., Lau, H. C., Zilber-

stein, S., & Zhang, C. (2014). Decen-tralized multi-agent reinforcement learn-ing in average-reward dynamic DCOPs.In Proceedings of the AAAI conferenceon artificial intelligence (pp. 1447–1455).

Okimoto, T. (2012). Graph-based algo-rithms for distributed constraint satisfac-tion/optimization problems (PhD thesis).Kyushu University, Fukuoka (Japan).

Ota, K., Matsui, T., & Matsuo, H. (2009). Lay-ered distributed constraint optimizationproblem for resource allocation problemin distributed sensor networks. In Pro-ceedings of the principles and practice ofmulti-agent systems (pp. 245–260).

Ottens, B. (2012). Coordination and sam-pling in distributed constraint optimiza-tion (PhD thesis). Ecole Polytech-nique Federale de Lausanne, Lausanne(Switzerland).

Ottens, B., Dimitrakakis, C., & Faltings, B.(2017). DUCT: An upper confidencebound approach to distributed constraintoptimization problems. ACM Transac-tions on Intelligent Systems and Technol-ogy , 8(5), 69:1–69:27.

Pearce, J., & Tambe, M. (2007). Qual-ity guarantees on k-optimal solutions fordistributed constraint optimization prob-lems. In Proceedings of the internationaljoint conference on artificial intelligence(pp. 1446–1451).

Petcu, A., & Faltings, B. (2005a). A scal-able method for multiagent constraint op-timization. In Proceedings of the interna-tional joint conference on artificial intelli-gence (pp. 1413–1420).

Petcu, A., & Faltings, B. (2005b). Superstabi-lizing, Fault-Containing Distributed Com-binatorial Optimization. In Proceedingsof the AAAI conference on artificial intel-ligence (pp. 449–454).

Petcu, A., & Faltings, B. (2007a). MB-DPOP:A new memory-bounded algorithm fordistributed optimization. In Proceedingsof the international joint conference onartificial intelligence (pp. 1452–1457).

Petcu, A., & Faltings, B. (2007b). Optimalsolution stability in dynamic, distributedconstraint optimization. In Proceedingsof the international conference on intelli-gent agent technology (pp. 321–327).

Rust, P., Picard, G., & Ramparany, F. (2016).Using message-passing DCOP algo-

12

AI MATTERS, VOLUME 3, ISSUE 4 WINTER 2018

rithms to solve energy-efficient smart en-vironment configuration problems. InProceedings of the international jointconference on artificial intelligence (pp.468–474).

Stranders, R., Delle Fave, F. M., Rogers, A.,& Jennings, N. (2011). U-GDL: A decen-tralised algorithm for dcops with uncer-tainty (Tech. Rep.). Department of Elec-tronics and Computer Science: Univer-sity of Southampton.

Stranders, R., Farinelli, A., Rogers, A., & Jen-nings, N. R. (2009). Decentralised co-ordination of continuously valued con-trol parameters using the max-sum algo-rithm. In Proceedings of the internationalconference on autonomous agents andmultiagent systems (pp. 601–608).

Tabakhi, A. M., Le, T., Fioretto, F., & Yeoh, W.(2017). Preference elicitation for DCOPs.In Proceedings of the international con-ference on principles and practice of con-straint programming (pp. 278–296).

Tassa, T., Grinshpoun, T., & Zivan, R. (2017).Privacy preserving implementation ofthe Max-Sum algorithm and its vari-ants. Journal of Artificial Intelligence Re-search, 59, 311–349.

Tassa, T., Zivan, R., & Grinshpoun, T. (2016).Preserving privacy in region optimalDCOP algorithms. In Proceedings of theinternational joint conference on artificialintelligence (pp. 496–502).

Ueda, S. (2014). Computational coali-tion formation: Compact representationand constrained matching (PhD thesis).Kyushu University, Fukuoka (Japan).

Voice, T., Vytelingum, P., Ramchurn, S.,Rogers, A., & Jennings, N. (2011). De-centralised control of micro-storage inthe smart grid. In Proceedings of theAAAI conference on artificial intelligence(pp. 1421–1427).

Yedidsion, H. (2015). Distributed constraintoptimization for teams of mobile agents(PhD thesis). Ben-Gurion University ofthe Negev, Beer-Sheva (Israel).

Yeoh, W., Sun, X., & Koenig, S. (2009). Trad-ing off solution quality for faster computa-tion in DCOP search algorithms. In Pro-ceedings of the international joint confer-ence on artificial intelligence (pp. 354–360).

Yeoh, W., Varakantham, P., Sun, X., & Koenig,

S. (2015). Incremental DCOP searchalgorithms for solving dynamic DCOPproblems. In Proceedings of the inter-national conference on intelligent agenttechnology (pp. 257–263).

Yeoh, W., & Yokoo, M. (2012). Distributedproblem solving. AI Magazine, 33(3),53–65.

Zhang, W., Wang, G., Xing, Z., & Witten-berg, L. (2005). Distributed stochasticsearch and distributed breakout: Proper-ties, comparison and applications to con-straint optimization problems in sensornetworks. Artificial Intelligence, 161(1–2), 55–87.

Zivan, R., Yedidsion, H., Okamoto, S., Glinton,R., & Sycara, K. (2015). Distributed con-straint optimization for teams of mobilesensing agents. Journal of AutonomousAgents and Multi-Agent Systems, 29(3),495–536.

Ferdinando Fioretto isa postdoctoral researcherat the Industrial and Op-erations Engineering De-partment of the Univer-sity of Michigan. His re-search focuses on multi-agent systems, data pri-vacy, and discrete opti-

mization. His dissertation was awarded thebest dissertation in Artificial Intelligence fromthe Italian Association of Artificial Intelligence,in 2017. Additional information can be foundat: http://www-personal.umich.edu/˜fioretto/.

William Yeoh is an assis-tant professor in the Com-puter Science and En-gineering Department atWashington University inSt. Louis. His re-search interests includemulti-agent systems, dis-tributed constraint rea-

soning, heuristic search, and planning with un-certainty. He is an NSF CAREER awardeeand was named in IEEE’s AI’s 10-to-Watch listin 2015. Additional information can be foundat: https://sites.wustl.edu/wyeoh/.

13