On Dynamic Consensus Processes in Group Decision Making

Problems

I.J. Pereza, F.J. Cabrerizob, S. Alonsoc, Y.C. Dongd, F. Chiclanae, E. Herrera-Viedmab

aDept. of Computer Sciences and Engineering, University of Cadiz, ignaciojavier.perez@uca.es, Cadiz,Spain

bDept. of Computer Science and Artificial Intelligence, University of Granada, cabrerizo@decsai.ugr.es,viedma@decsai.ugr.es, Granada, Spain

cDept. of Software Engineering, University of Granada, zerjioi@ugr.es, Granada, SpaindDept. of Business School, Sichuan University, ycdong@scu.edu.cn, Chengdu, China

eCentre for Computational Intelligence, Faculty of Technology, De Montfort University,chiclana@dmu.ac.uk, Leicester, UK

Abstract

Consensus in group decision making requires discussion and deliberation between thegroup members with the aim to reach a decision that reflects the opinions of every groupmember in order for it to be acceptable by everyone. Traditionally, the consensus reach-ing problem is theoretically modelled as a multi stage negotiation process, i.e. an iterativeprocess with a number of negotiation rounds, which ends when the consensus level achievedreaches a minimum required threshold value. In real world decision situations, both theconsensus process environment and specific parameters of the theoretical model can changeduring the negotiation period. Consequently, there is a need for developing dynamic consen-sus process models to represent effectively and realistically the dynamic nature of the groupdecision making problem. Indeed, over the past few years, static consensus models havegiven way to new dynamic approaches in order to manage parameter variability or to adaptto environment changes. This paper presents a systematic literature review on the recentevolution of consensus reaching models under dynamic environments and critically analysetheir advantages and limitations.

Keywords: Group decision making; dynamic decision support systems; consensus process;multi period decision making; adaptive consensus models.

1. Introduction

Group Decision Making (GDM) is usually described as a best alternative(s) selectionprocess of selection from a given set of feasible options based on the opinions of a groupof people, frequently referred to as experts. This process is of importance not only whenthe decision consequences affect a group of people, but also when the decision itself can beimproved, that is to make better decisions, by involving more people in the decision process[5, 9, 30].

Ideally, unanimous decisions, i.e. total agreement on the decision by all experts, are aimedat, although it is not really necessary. Indeed, the majority rule is frequently presented as acornerstone of any democratic decision [6, 12]. Nevertheless, different situations may require

Preprint submitted to Information Sciences May 1, 2018

different rules. For example, the minority rule or the unique person rule could be moreoperative in order to save time and reduce cost in companies. Nowadays, though, it is moreand more important to make decisions under the consensus rule, i.e. to make consensualdecisions by reaching a general or widespread agreement among the involved people. Assuch, consensus reaching process has become an important research area in GDM problems[6, 17, 23].

As mentioned above, full consensus is not necessary and even untenable in many realworld situations. This could be a result of important differences between experts, whichare inherent to their own knowledge and personal interests. Thus, consensus within GDMhas been alternatively modelled following a ‘softer’ methodology for its measurement thatallows a range possibilities from absence to total agreement. This soft consensus measures,which aims to be realistic and flexible, is based on the implementation of fuzzy set and logicconcepts and tools [4, 6, 12, 24].

In practice, consensus reaching processes proceed in a convergent multistage way, whereexperts present their individual preferences at the beginning of the process and, while con-sensus level is not considered enough, they discuss, negotiate and bring positions closer bymodifying their initial opinions [22]. It is, therefore, assumed that the individuals are pre-pared and willing to commit to those opinion changes. Sometimes there exists a particularperson (or automated system) who acts like a moderator, being responsible of the processmanagement until the experts reach agreement [1].

Decision problems require a careful analysis of the current environment conditions andcharacteristics of the problem to be solved. All decision problems do not take place in astatic environment. In fact, when the current problem conditions or parameters vary duringthe discussion phase, the problem final solution could vary as well. Indeed, it is clear thatmany external and subjective factors could affect decision processes. As a consequence, theconsensus process has been recently studied as a dynamic process and some researchers havefocused on the incorporation of diverse dynamic parameters or variables to static consensusmodels. The purpose of this paper is to provide a comprehensive description of the currentstate of the art of dynamic consensus approaches and to analyse the following different newtrends and challenges in this field:

• Classical Dynamic Consensus Approaches in which the consensus process, bydefinition, is an iterative and dynamic process and the experts’ preferences can bemodified until an agreement is reached by increasing the consensus level.

• Time Modelling Consensus Approaches present a new variable to model time(t). Time modelling approach aims at predicting opinion dynamics in time (from t tot+ 1) by using time series. Additionally, in multi-period decision making approaches,an expert’s different preferences at different time periods are aggregated using new op-erators, such as the Dynamic Weighted Averaging (DWA) operator and its extensions.

• Dynamic Environment in Consensus Approaches. The consensus process envi-ronment can also change over time. Consequently, in order to reflect these changes inthe consensus model model itself, the following diverse dynamic variables have beenconsidered/analysed in the literature: i) dynamic alternatives, ii) dynamic experts, iii)dynamic expert’s importance and iv) dynamic criteria.

2

• Adaptive Consensus Approaches. The consensus model behaviour can also bedynamic. Feedback mechanism to generate recommendations, when present can alsoadapt dynamically with respect to: i) the collective consensus level (CL), ii) the indi-vidual consensus indexes (ICI) and iii) the expert’s importance.

The rest of the paper is set out as follows. Section 2 presents preliminary concepts onconsensus reaching processes and the research methodology used to define the systematicliterature review. In Section 3, the different dynamic consensus reaching models, consideredas main contributions in the literature review, are described. In Section 4 a discussionamong the reviewed approaches is presented, and potential future research avenues proposedto improve the current methods. Finally, in Section 5 conclusions are drawn.

2. Preliminaries

The resolution of a GDM problem involves two main processes [22, 39]:

1. The consensus process that aims to support experts until a satisfactory level of agree-ment among them is reached.

2. The selection process based on the individual experts’ opinions, which are usually fusedinto a collective opinion, from which a group decision is derived, mainly via the rankingof the different alternative.

This section presents preliminary information on consensus reaching processes in GDMproblems. Then, the systematic literature review process carried out to complete the researchdescribed.

2.1. Consensus reaching processes in GDM problems

A consensus reaching process model can be described as a multi stage process (dynamicand iterative) that simulates real face to face discussion or negotiation processes between agroup of experts. It relies on the assumption of experts’ willingness to change their opinionsor preferences, in several consensus rounds, in order to reach a collective agreement ona decision. It is not unusual for a moderator or facilitator person to support experts inreaching such agreement, which can be automated within an appropriate consensus supportsystem.

To date, multiple consensus reaching models have been published, with quite a largenumber of them implementing soft consensus measures and fuzzy logic based rules to guideand control the process [1, 4, 6, 22, 33]. An analysis of these models reveals a workingoperation pattern or framework, which is graphically depicted in Figure 1 and describedbelow.

(1) Before the start of the discussion process, experts are informed about the particularparameters of the problem and the different alternatives to choose the best ones from.

(2) Experts express their individual opinions or preferences on the alternatives by meansof a preference representation format (ranking of alternatives, evaluations or pairwisecomparison matrix).

3

Figure 1: General Consensus Framework

(3) The moderator/system computes consensus measures based on how distant the individ-ual preferences are. These measures are used to decide when the group consensus levelreach is sufficient.

(4) If the group consensus level is satisfactory, the selection process is activated. Otherwise,a new discussion round is carried out. Most consensus models implement what is knownas a feedback mechanism, which is used to feedback experts on how to improve the groupconsensus level.

(5) Some or all experts might change their opinions after the feedback information received,which will subsequently modify the group consensus. At this point, step (3) is activatedagain until the minimum group consensus threshold values is achieved or a maximumnumber of consensus rounds is reached.

It is worth noticing that in the above consensus framework (classical consensus ap-proaches), decision making problem parameters (experts set, alternatives set, experts’ im-portances, set of criteria etc.) remain fixed through the discussion rounds. In other words,the model configuration is set prior the first stage of the consensus process and it is not mod-ified during the whole process. This static framework of consensus is too rigid in practiceas there could be external factor, such as weather conditions, price fluctuations, alternativesand/or experts availability, that could happen and that would make the above frameworkimpractical. To address this issue, recent approaches have been proposed that allow thevariation of certain parameters and consequently the implementation of a dynamic structurein the above consensus framework. Consequently, a new generation of dynamic and adaptiveconsensus reaching models have been developed over the past few years. This paper presenta review, analysis and classification of these new kind of dynamic consensus reaching models.

4

2.2. Consensus measures in GDM problems

As per Laughlin [26], social decision scheme and social choice theory address the samefundamental problem, which it is the way in which a group of people combine or aggregate adistribution of member preferences in a collective decision subject to previously establishedparliamentary procedures and voting rules. Different rule procedures imply different groupdecision making methods, which range from directive to participatory methods. Methodsare closer to the directive one when the decision is made by a limited number of decisionmakers in the group. For example, individual dominance method where a single person hasthe authority to make the final decision, or minority influence method that usually takes theform of decisions delegated from larger groups and made by sub committees [2]. On the otherhand, methods are closer to the participatory one when every member of the group is involvedin making the final decision. Examples of participatory methods are majority rule methodsinclude a range of voting systems where the number of votes received by each alternativeestablish the final ranking, while consensus methods where a consensual agreement has tobe reached by all group members. This paper focuses on this last group of participatoryprocedures, i.e. on procedures to reach consensus among a group of experts, and as suchwill present an analysis of different consensus measures of the current literature.

As mentioned before, consensus is usually defined as the total and unanimous agreementof all the experts in relation to the feasible alternatives [12]. This definition implies theexistence of only two states of consensus (absence and total agreement), which translateinto a hard consensus measurement ({0, 1}) in accordance to classical logic. In any case, fulland unanimous agreement might be non-realistic and unnecessary in practice. An alternativerealistic approach to consensus measurement that extends the above hard/crisp measurementis preferred. Kacprzyc et. al. in [23] proposed the use of soft consensus measures based onfuzzy logic to quantify the level of consensus level to reflect the range of partial agreementstates located between the mentioned absence and total agreement states. Soft consensusmeasures are based on the concept of coincidence among experts’ preferences, and a studyby Cabrerizo et al. [6] identified three different coincidence criteria:

1. Strict coincidence among preferences. This consensus approach is based on the con-cept of equality, and it assumes only two possible coincidence values {1, 0} for whenpreferences are equal or not, respectively. The advantage of this approach is that thecomputation of the consensus degrees is simple and easy. However, the drawback isthat the consensus degrees obtained do not reflect the real consensus situation, withsimilar preference values but different are treated equally to very dissimilar preferencevalues.

2. Soft coincidence among preferences. This approach is based on the concept of similarityrather than equality. Thus, a similarity function is define, based on a distance functionbetween preference values, which allows for a gradual computation of the coincidenceconcept in [0, 1], which resembles better the real consensus situation. This criteria iscomputationally is more expensive than the strict coincidence criterion.

3. Coincidence among solutions. The advantage of this approach is that the consensusstate is evaluated by comparing the individual solutions each expert would obtainwithout considering other experts’ preferences. This approach is even more expensivecomputationally than the soft coincidence criterion.

5

From the above, it is clear that consensus measures are based on the implementation ofa similarity (distance) function. As such, it s worth mentioning the study by Del Moral etal. [12] on five of the most commonly used distance functions in modelling soft consensusmeasures: Manhattan, Euclidean, Cosine, Dice and Jaccard.

2.3. Systematic literature review process

In order to carry out a systematic review of publications on dynamic consensus reachingapproaches currently available in public research repositories, the guidelines proposed byKitchenham et al. in [25] have been followed. The two main databases of bibliographicalreferences in the world have been used: Web of Sciences (Clarivate Analytics) and Scopus(Elsevier).

With the purpose of finding the relevant papers and excluding irrelevant ones on topicssuch as voting systems or approaches that do not use the concept of consensus to guide thedecision making process, the terms “dynamic”, “changeable”, “multi period” and “adaptive” were entered simultaneously with “group decision making” or “GDM” and “consensus”.In each case, papers not related to dynamic consensus models were excluded. The referencessection of a considered relevant paper was also inspected to include some of the referenceswhen appropriate.

3. Analysis and classification of current dynamic consensus reaching approaches

Dynamic consensus approaches can be classified in one of the following four main cate-gories or methodologies: Classical Dynamic Consensus Approaches; Time Modelling Consen-sus Approaches; Dynamic Environment in Consensus Approaches; and Adaptive ConsensusApproaches.

3.1. Classical Dynamic Consensus Approaches

Traditionally, the consensus reaching problem is theoretically modelled as a multi stagenegotiation process, i.e. an iterative process with a number of negotiation rounds, which endswhen the consensus level achieved reaches a minimum required threshold value. Classicalapproaches focused their attentions on the iterative feature of the consensus process, and inparticular on the modification of the experts’ preferences to increase their level of consensus.This is mainly achieved by the design and building of appropriate feedback mechanismsto provide recommendations to experts based on their current state within the consensusprocess, which in practice means replacing the physical moderator with a virtual moderator.Other contributions on this category aimed at speeding the process by proposing the use ofnew technologies such as web sites or smartphones applications to support the process. Inthis way, the experts can resume the process and change their preferences at anytime andanywhere.

3.1.1. Change of preferences

Since the late 70s, several approaches on how groups reach agreement via the changing ofexperts’s preferences when the current level of consensus is not sufficient have been carriedout [4, 6, 12, 38, 45, 50].

6

The first authors who appeared to use the term “dynamic” to refer to the changing ofpreferences at each consensus process iteration were Fedrizzi et al. in 1999 [17]. Indeed,in this work the authors modelled the consensus process as a dynamical network that com-bines the minimization of a soft measure of collective dissensus, V , and an individual inertialmechanism which tries to model opinion changing aversion, U . The full cost function, W ,was then computed as function of V and U . Finally, the current iterative dynamism of theconsensual network corresponds to a parallel gradient descent scheme for the full cost func-tion W and the dynamical variables (individual preferences). This consensual and dynamicnetwork, which can be seen as a kind of non supervised learning algorithm, is able to modifysome individual preference variables through the process time. In conclusion, this approachmodels a dynamic network that minimises the cost to reach agreement amongst the experts.Subsequently, Fedrizzi et al. [18] extended their dynamical approach to the case of havinguncertain preferences modelled as triangular fuzzy numbers. Recently, a new version of thisdynamic approach that implements randomness in the modelling of the individual processiterations leading to singular (different) consensus process paths, with the aim to better cap-ture the nature of some real world consensus reaching processes, was presented in [32]. Theimprecision of the different consensus results is taken as the input of other simple process toobtain a collective consensus result from the distribution of the singular ones. Specifically,they introduce the randomness into the resistance component of the cost function by usinga parameter as a standard normally distributed random component (with mean zero andvariance one) divided by the number of iterations. In such a way, the effect of changing thestep size is created and random experts are allowed to back-up in the discussion process.

A second point of view about improving the process convergence by modifying the ex-perts’ preferences at each iteration is given by Herrera-Viedma et al. [1, 2, 22, 39]. In thiscase, the authors do not minimise the cost function as described before, they focus on max-imizing the experts’ knowledge on the current discussion process state. In such a way, notonly the feedback mechanism is extended to produce the most complete recommendationsto the experts but also some algorithms to deal with ignorance or missing preference valuesbased on experts’ consistency are developed. These authors design simple but understand-able and easy to use rules to support experts in changing their opinions. Indeed, the rulesdevised by these authors establishing the direction of change that the experts should followto increase the group consensus. This is accomplished by computing consensus degrees tomeasure how far away are any two individual experts’ preferences, and proximity measuresto quantify how far each each individual expert’s preferences are from the group’s prefer-ences, this last obtained by fusing the individual experts’ preferences. These measures arecomputed at the three levels of a preference relation (pair of alternatives, alternative, andrelation levels), which is exploited to identify the experts of the group whose contribute toconsensus is insufficient, and in which alternatives and pair of alternatives preference valuessuch contribution to consensus is specifically insufficient (identification rules) and to establishthe direction of change of preference values (direction rules) to increase the group consensus.

A different approach to the two described above has been proposed by Parreiras et al. in[35, 36]. In [36], the authors combine the different points of view on how to proceed makinggroup decisions, with the moderator taking part in the discussion process by inviting anyexpert to update his/her preferences but also by adjusting the weight allocated to eachexpert’s opinion that maximise the group soft consensus index. In [35], they propose a

7

dynamical consensus scheme that, in addition to the soft consensus index, makes use ofa comparability (similarity or credibility) and a concordance indices to control the flow ofinformation during the negotiation process. The concordance index is used to choose anexpert (usually the most discordant one) to be requested to explain to the other expertsor to review his/her preferences, while the comparability index is used to select the subsetof experts that had some problems in expressing their opinions so that they are supportedby the most assertive experts in constructing their own preferences. By supporting anintense flow of information among the group, it is demonstrated that the process of sharingimportant information or knowledge is upgraded (at a risk of a longer negotiation process)and, therefore, the quality of the collective solution is also improved. Eventually, if one ofthe following circumstances is satisfied, the negotiation process is finished: (1) the consensuslevel has reached the established threshold; (2) the number of discussion round has reachedthe established threshold; (3) after a specific number of rounds, the same expert remainsas the most discordant one and the moderator cannot help him/her to change his/her ownopinion any more; (4) the previously established negotiation time has finished.

Finally, Cao et al. [8] presented a monitor based approach to deal with the follow-ing dynamic aspects of the consensus process: convergence rates, measurements delays andasynchronous events. To do so, they used properties of composition of directed graphs to-gether with results from the theory of non homogeneous Markov chains to derive the worstcase convergence rates for the headings of a group of experts performing a consensus pro-cess. This approach also uses graph theoretic construction to solve consensus processes withmeasurement delays, asynchronous events or a group leader. In [7], Cao et al. presenteda graphical approach to the same problem. In order to establish the graph of a stochasticmatrix, they define the concepts of rooted, strongly rooted and neighbour shared. Recently,following the monitoring idea, Palomares et al. [34] presented a new graphical monitor-ing tool of preference evolution. Furthermore, Wu et al. [44, 45] presented some similarapproaches based on visual information to improve the consensus reaching process.

3.1.2. Fast Convergence

Usually, in real time situations, the quality of the collective decision is as important asthe time it takes to make such decision. This is the case of decision making in geographiccontext, in business, in navigation applications or in natural resources management. Thus, aGDM process should be rapid so that the experts can yield and process information quickly.Furthermore, there are decision situations where some of the decision process data canbe modified over time. Therefore, some researchers have focused on implementing previoustheoretical consensus models via new information technologies such as web sites, smartphoneapps or even implementing the GDM model as a web service [1, 24, 39, 40, 50]. They claimthat if the experts have the opportunity to follow the negotiation process in real time, thedecisions will improve. In [39], the concept of Mobile Decision Support System (MDSS)was proposed with the incorporation of mobile technologies in classical Decision SupportSystems. It allows experts to have timely and updated information, anytime and anywhere,leading to an agile decision process by simulating real face to face negotiation meetings.Additionally, shortening the negotiation time will limit the the number of changes in theparameters of the problem.

Table 1 summarises the advantages and drawbacks of the most important approaches

8

analysed at this level.

Advantages DrawbacksChange ofPreferences

Models support multi-stage pro-cesses to modify experts’ prefer-ences if the consensus level is notsatisfactory.

Automatic feedback mechanismimproves the convergence of theconsensus process.

The rest of the parameters of theproblem are assumed static.

Models not able to address environ-mental changes.

Fast Con-vergence

Models speed up the informationflow by using new technologies.

Chances of changing the problemparameters is reduced.

Models not able to address be-havioural changes.

Table 1: Classical Dynamic Consensus Approaches.

3.2. Time Modelling Consensus Approaches

Some researchers have considered time as the most important element for modellingdynamism, and as such the following two research trends in consensus reaching processedhave emerged recently: (1) opinion dynamic modelling and (2) multi-period decision making.

3.2.1. Opinion dynamic modelling

Hegselmann et al. [21] proposed a time dependent approach to deal with changes duringthe negotiation process of previous classical models and social influence networks and opinionchange. This new approached is based on the bounded confidence concept, where an expertis able to form his/her opinion by sharing other agents’ opinions. Specifically, the opinionchange evolution at time (t+ 1) is modelled as the weighted arithmetic mean of opinions inthe previous time stages (t), (t − 1), ..., (t − n). The crucial point here is that each expertassociates weights to the opinion of others in order to establish the bounded confidence.Only in the case of constant weights and enough confidence among experts the consensuscan be modelled with a classical approach. Therefore, the dynamism is in the computing ofthe opinion evolution taking into account the changes of others’ opinion weights. In this way,experts can adjust their confidence in other opinions and their preferences are automaticallyupdated by the system. This model is also known as the Hegselmann-Krause (HK) modelmodel. On the other hand Deffuant et al. [11] presented the Deffuant-Weisbuch (DW) modelto mix beliefs among interaction agents that has also become a classical bounded confidencemodel. Both models use the concept of bounded confidences, but differ in how many otherexperts each expert can interact with at a certain time.

Based on the DW model and the HK model, scholars have developed some interestingextensions, such as the linguistic version of the HK model developed by Dong et al. [14].Another worth mentioned approach was proposed by Li et al. [28], where a new automatedmethod based on a basic unit-interval monotonic (BUM) function was proposed to computethe time weight.

9

3.2.2. Multi-period group decision making

In 2008, Xu [46] proposed a multi-period decision making concept by which experts maygive different views about the same set of alternatives over a time period due to changes inthe decision context. Therefore, it was considered necessary to incorporate in the consensuscomputation not just the latest preferences of each expert but also their previous expressedones during the whole negotiation time period. This was modelled via a fusion of all thepreferences provided by the an expert, the output of which was considered by this scholarthe associated ‘timeless preference of each expert’. The fusion operator used was termed thethe dynamic weighted averaging (DWA) operator, which incorporates the variable time (t),with three variants: arithmetic series, geometric series, and normal distribution based DWAoperators. Also, the Uncertain based DWA operator (UDWA) was proposed to aggregateinterval valued arguments. This strategy has been taken forward to deal with intuitionisticfuzzy information, linguistic information [47–49], and hesitant preferences [37], respectively.

Recently, Gupta et al. [20] proposed a different approach to the multi-period decisionmaking problem, which is based the concept of expert’s net preferences (NP ) of alternativesin pairwise form after accounting his/her judgements in different time intervals over a period,which are subsequently transformed a set of sequences. These sequences are called maximumsequences and represent the group’s final alternatives rankings. Figure 2 shows this approach.

Figure 2: Ranked list algorithm process flow [20]

Table 2 summarises the advantages and drawbacks of the most important approachesanalysed at this level.

3.3. Dynamic Environment in Consensus Approaches

In this section we analyse some consensus approaches that model the problem environ-ment as dynamic. These are classed into four different groups: 1) Dynamic alternativesmanagement, 2) Dynamic experts management, 3) Dynamic experts’ importance manage-ment, and 4) Dynamic criteria management

3.3.1. Dynamic set of alternatives

As mentioned before, the set of alternatives can change during the decision makingprocess, with some of the alternatives’ feasibility or availability being affected (changed)

10

Advantages DrawbacksOpinionDynamics

Models are able to compute asound prediction of experts’ pref-erences evolution.

Decrease time of waiting for thereal opinion changes.

Prediction of other problem pa-rameters evolution is not possible.

The computed prediction couldfail in some cases.

Multi-PeriodGDM

These approaches consider a prob-lem snapshot in time.

Different opinions of the same ex-pert at different moments are ag-gregated to obtain an acceptedoverall opinion.

The opinions are the only change-able parameter.

Sometimes, the only acceptedoverall opinion should be the lat-est one instead of the aggregationof all the intermediate ones.

Table 2: Time Modelling Consensus Approaches.

during the process time. For example, in medical diagnosis scenarios where the treatmentsto recommend to the patient are the alternatives to select from. In such a case, medicaldoctors (experts) should take into account any variation of the patient state in the previousfew hours, as the patient health could have improved or deteriorated (presenting new healthsymptoms) due to the prescribed medication. Another typical example of this situation ise-commerce or e-business decision scenarios, with the different products or services availableto purchase form the set of alternatives to consider. In this particular case, it is easy to seethat, while the negotiation details (prices, quantity, etc...) is underway, the product/servicesavailability could change. This issue of dynamic alternatives in decision making has beenaddressed by Perez et al. in [39–41], where a method to remove the ‘worst’ alternativesand insert new ones into the discussion process was proposed. The proposed proceduredifferentiates the two cases: (1) remove old worst alternatives, and (2) insert new goodalternatives.

(1) The first case deals with situations where some alternatives are not available any longerbecause they have been discarded by the experts of due to external factors to the groupof experts. From the set of alternatives, candidates alternatives for removing are thoseranked below a threshold value in terms of their associated dominance degree (QGDD)(see Figure 3a).

(2) The second case is managed by the system, with a corresponding module informing whena new ‘good’ alternative appears in the decision context to replace the one with lowestQGDD (see Figure 3b).

Finally, in both cases, experts have to express their agreement or disagreement with thechanges proposed by the system. If most of the experts approves the changes, the systemmakes the necessary adjustments in the set of alternatives ready for the next consensus stage.In order to correctly integrate this new tool into the consensus reaching process, the feedback

11

(a) Case 1 (b) Case 2

Figure 3: Dynamic alternatives management [39]

mechanism is also updated to disregard as unnecessary any recommendation that refers toany of the discarded alternatives.

3.3.2. Dynamic set of experts

Commonly, the number of people involved in a group decisions process can change withsome people leaving the process with others replacing them or not. Moreover, when a largeamount of individuals is involved in a negotiation process, the decision task becomes harder,although this is possible now with new (mobile, internet enabled) technologies viable forthe decision making process management. For example, in social media or web communitiesenvironments new expert could be incorporated to the ongoing discussion process at anytime,and some of the current experts leaving the process as well. Alternatively, a big group ofpeople could be reduced in order to simplify communications and to facilitate the consensusreaching process. Some scholars have focused their research efforts on modelling these aspectsof the negotiation processes.

Ballester et al. in [3] presented a recursive procedure to select competent subgroups ofexperts just taking into account their own opinions about the whole group. However, themost important dynamic approach to deal with these decision problems is by Alonso etal. in [2], where they designed a sophisticated consensus reaching model that incorporatesa delegation scheme to manage large and dynamic groups of experts in which they maychoose to continue the discussion process or delegate other experts, normally with similarpreferences (see Figure 4). The introduction of a delegation mechanism can solve the problemof sporadic contributions that appears in web communities and social networks and reducethe amount of individual opinions related to the decision problem. Some details on theproposed delegation scheme are presented in the following:

1. At each discussion round an expert eh can freely delegated to other one th and leave thedecision making process or revoke a delegation previously made. In that case, he/shewill not be requested to update his/her preferences anymore.

2. The delegation mechanism can be interpreted as a trust network and, consequently, bemodelled as a directed graph. In such a way, transitivity conditions can be applied.Moreover, in order to avoid cycles, the system will produce alerts and prevent delegateto an expert that had previously (directly or indirectly) delegated to that expert.

12

Figure 4: Delegation scheme in the consensus model of [2]

3. After the delegation stipulated time, the system will summarise all the proposed del-egations by using trust weights vector τh, which are related to how many times anexpert has been delegated.

4. A trust checking mechanism is incorporated to avoid malicious experts. In such away, if an expert with a trust weight τh > 0 changes his/her preferences drastically inthe next round of the consensus process, those experts that delegated to him/her areinformed and allowed to cancel these delegations if they consider that their preferencesare no longer appropriately represented.

3.3.3. Dynamic experts’ importance

A key issue in GDM is the importance or weight associated to experts [13]. Saaty in [43]propose a process to determine experts’ weights by taking into account prior information onthe experts using a hierarchy of criteria such as experience, expertise, persuasive abilities,previous performance and effort on the problem to solve; while Ramanathan et al. [42]proposed an eigenvector based method to compute the weight vector by using interpersonalcomparison among experts. However, Forman et al. [19] noted that sometimes it is not easyto find a knowledgeable person to provide assessments based on Saaty’s hierarchy approachor he/she may not have enough knowledge to rank the experts’ importance. Therefore,without prior information there us a need to have an objective way to determine the weightsassociated to the experts. Once, experts’ weights are determined, a weighted average of

13

Figure 5: Non-cooperative behaviours in GDM [16]

their opinions or preferences is computed to derive the collective/group opinion or prefer-ence. Experts’ weights are considered static, constant during the whole the decision process.However, it is obvious that in a dynamic environment, experts’ importance may vary duringthe decision process. In recent years, some dynamic consensus models that incorporate adynamic way of determine experts’ weights have been developed.

• Dong et al. [13] proposed to determine the weights based on experts’ opinion transitionprobabilities. They state that the differences in the proximity between experts revealthe possibilities of opinion transitions. This opinion transition process can be mod-elled by using a finite state space Markov chain. Considering the limiting distributionof the chain, the authors calculated the power or weight distribution of experts in agroup. Eventually, Dong et al. [16] proposed another approach to integrate experts’weights dynamically generated into the consensus process. In this proposal, the ex-perts provide not only mutual evaluation information for the other experts, but alsopreference information about alternatives. This mutual evaluation is represented byusing multi-attribute mutual evaluation matrices (MMEMs). The authors proposed anoptimization-based way to obtain the experts’ weights from the MMEMs (see Figure 5).

• Li et al. [29] proposed a method for adjusting weights by using an analytic method;which was extended in [27] via the application of a Borda function and linguisticquantifier to construct the variable weight state vector.

14

Figure 6: Experts profile in GDM situations [10]

• D’Aniello et al. [10] proposed a consensus approach with variable importance of expertsvia the incorporation of the concept of expert’s profile, which contains useful informa-tion about the experts that is subsequently used compute dynamically the experts’weights depending on each situation (see Figure 6).

3.3.4. Dynamic set of criteria

Frequently, in GDM problems, it is necessary to take into account different criteria toassess the alternatives. For example, an alternative could be the best one based on a qualitycriterion, but the worst one based on a price criterion. Thus, the evaluation by using morethan one criterion is necessary to derive usually the overall best alternative. Multi-criteriadecision making (MCDM) refers to a process to derive the best alternative from a feasible setwhich are assessed against multiple and sometimes conflictive criteria. Most of the currentmulti criteria consensus models consider the criteria set as a fixed set or static parametersof the consensus process; although recently, new approaches have been developed that allowthe set of criteria to be modelled as dynamic parameters of the consensus process.

• Lourenzutti et al. in [31] proposed a generalized TOPSIS (Technique for Order Pref-erence by Similarity to Ideal Solution) in a dynamic environment. They introducedthe GMo-RTOPSIS (the Modular TOPSIS in random environments for Group decisionmaking) that treats each expert independently and with complete freedom to expresstheir opinions. Additionally, the experts are not forced to agree with an specific crite-ria set, and as such experts can consider different criteria sets and different underlyingfactors, weight vectors or mappings. The proposed method considers each criterion ofeach expert as a separate module, and random variables are used to deal with under-lying factors. Summarising, this approach is able to deal with real decision situationswhere the unpredictable environment can affect the performance of the alternatives.

15

Figure 7: The resolution framework for complex and dynamic MCGDM [15]

• Dong et al. [15] have also proposed a consensus reaching model for dynamic and com-plex MCGDM (Multi Criteria Group Decision Making) problem. In this approach,each expert has an individual set of attributes or criteria and an individual set ofalternatives, which undergo some dynamic changes through the consensus reachingprocess. The main feature of this proposal is that it is not necessary to reach an agree-ment among the experts; its aim or challenge being to be able to find one alternativeauthorised by all, or at least, most of the experts. This method is graphically depictedin Figure 7.

Tables 3 summarises the advantages and drawbacks of the most important approaches anal-ysed at this level.

3.4. Adaptive Consensus Approaches

The basis of adaptive consensus models is a refinement or sophistication of the processto allow the agreement increasing at the same time that the quality and amount of adviceproduced by the system are improved and reduced, respectively [6]. In other words, the gen-erated recommendations are adaptively selected by taking into account some of the dynamicparameters of the problem. In the following, the main adaptive consensus approaches andtheir results are described.

3.4.1. Feedback mechanism adaptation to the collective consensus level

One of the first adaptive consensus models was presented by Mata et al. [33], where theamount of generated recommendations at each discussion iteration was adapted to the cur-rent consensus degree. Indeed, the level of consensus affects the amount of changes requiredby a group of experts to facilitate the reaching of agreement. When the consensus levellow a higher amount of advice to support experts to close their preferences is required than

16

Advantages DrawbacksDynamic Set ofAlternatives

The set of alternatives is dynamicand can vary through the discus-sion process time.

Large sets of alternatives could betaken into account.

The four different approachesare applied independently.Thus, their combined appli-cation should be preferred inorder to deal with real worlddecision problems. They justsolve isolated parts of the wholeproblem.

Dynamic Set ofExperts

The set of experts is dynamic andexperts involved in the consensusprocess could vary through thediscussion process.

Large sets of experts (e.g. webcommunities) could help to takethe best decision.

The proposed delegation schemeand the trust network allow toput in place group representa-tives.

The four different approaches’behaviour is always the same,regardless of the environmentchanges

Dynamic Ex-perts’ Impor-tance

Experts’ importance is dynamicand the weight of each expert atthe preferences aggregation couldvary through the discussion pro-cess.

Different ways to determine ex-perts’ weight dynamically havebeen proposed.

Non-cooperative behaviours havebeen taken into account.

Dynamic Set ofCriteria

The criteria set is dynamic andthe criteria to use in assessingthe set of alternatives could varythrough the discussion process.

Each expert is allowed to havehis/her own criteria set.

Table 3: Dynamic Environment in Consensus Approaches

17

Figure 8: Reduction of the amount of advice into the discussion process [33]

when the consensus level is high . Specifically, these scholars propose a dynamic procedureof searching preferences, which they term PSp, to change for improving the agreement levelbased on the current consensus level achieved at the specific consensus round. As mentionedbefore, as the consensus round increases there will be an increase of the consensus leveland therefore this will lead to a decrease of the amount of recommendations of preferencechanges the experts will receive. The PSp dynamic strategy implemented is based on theclassification of the agreement in four categories: very low; low; medium; and high. Figure 8illustrates the described PSp dynamic strategy.

3.4.2. Feedback mechanism adaptation to the experts’ importance

A second adaptive consensus model was presented by Perez et al. [38]. This model ischaracterised by a novel feedback mechanism that adapts the amount of recommendations tothe experts’ weight or importance level. Experts with high expertise level normally need lessadvice compared with others with lower expertise. Thus, experts are classified into threedifferent classes: (1) high-important experts, Ehigh, (2) medium-important experts, Emed,and (3) low-important experts, Elow. It is worth noting that one of these categories couldbe empty, and therefore there could be situations where there are three subclasses of expertsbased on their level of expertise, or two subclasses to just one subclass. These three possiblesubclasses of expertise lead to three different strategies for selecting how many preferencesvalues to change in a consensus round to increase the group consensus level: (i) advising low-important experts, (ii) advising medium-important experts, and (iii) advising high-importantexperts (see Figure 9). The higher the expertise level, the less changes will be recommendedby the feedback process. Furthermore, as covered in a previous section, the experts’ weightcan be changed dynamically, which makes this consensus reaching model to be an adaptiveand dynamic consensus reaching model.

3.4.3. Feedback mechanism adaptation to the individual consensus indeces

Recently, a novel peer to peer dynamic adaptive consensus reaching model has been pro-posed by Dong et al. [13]. In comparison with centricity oriented methods, which use the

18

Figure 9: Consensus reaching process with non-homogeneous experts [38]

aggregated group opinion as the reference point in the consensus reaching process [33, 38],this new adaptive proposal compares the information from one expert to another one. Thispeer to peer comparison strategy reduces the computation cost and biases in the aggrega-tion process. Experts’ importance degrees are determined using a Markov chain allocatingmethod based on the opinion transition probability. The Markov chain models the prob-abilities when it is expected that differences in the proximity among experts reveal thepossibilities of changes in their preference.The model assumes it is more probable for anexpert to change his/her opinion closer to those of similar experts than to those of dissimilarexperts. An issue of this approach resides in the slow or even ineffective convergence of theconsensus process when small changes of preferences happen. The challenge then becomeson how to assist experts in making bigger changes on their preferences. However, the au-thors noticed that if non compatible experts, i.e. experts whose individual opinions are faraway, are selected and their preferences are changed to be closer then a strong increase ofthe consensus level will be achieved. This principle is used by the authors to proposed anew convergent and effective algorithm that implements a feedback mechanism that in eachround provides recommendations to just only those pairs of experts with maximum Individ-ual Consensus Index (ICI) in contrast to previous approaches based on a set of rules thatapply to all experts with ICI below the acceptable threshold [33, 38]. Therefore, as the ICIchanges dynamically, in response to the experts updated preferences, and the recommenda-tions’ recipients are selected depending on this index, the authors defined the approach asdynamic and adaptive (see Figure 10).

Table 4 summarises the advantages and drawbacks of the most important approachesanalysed at this level.

4. Discussion and future trends

Many dynamic consensus methodological approaches have appeared in recent years,which have been reviewed and analysed in the present paper in terms of their associatedadvantages and drawbacks. The main findings of this present review are:

19

Figure 10: Adapting the feedback mechanism to the individual consensus indexes [13]

Advantages DrawbacksCollective CL The behaviour of the model is dif-

ferent depending on the currentcollective consensus level.

If the consensus level is low, theamount of recommendations isbigger than otherwise.

The behaviour of the model ismodified but it does not allowchanges of the problem parame-ters.

Experts’Importance

The behaviour of the model is dif-ferent depending on the expertimportance.

The higher the importance of anexpert the less recommendationsreceives.

Experts’ importances remainfixed through the discussiontime.

Individual CI The behaviour of the model is dif-ferent depending on the currentindividual consensus index.

The behaviour of the model ismodified but it does not allowschanges of the problem parame-ters.

Table 4: Adaptive Consensus Approaches.

20

1. The main approaches modelling dynamic change of preferences approaches lack dy-namism in the rest of the problem elements. In these approaches, expert’s preferenceschange and the consensus process converges but the other parameters remain fixedthrough the whole process.

2. There are dynamic approaches able to to speed up the information flow of the process,shortening the negotiation time and consequently, reducing the probability of changein the problem parameters. These approaches make possible to achieve a rapid con-vergence of the consensus process. However, if changes in the problem parametershappen, these models are not able to address them appropriately.

3. Opinion dynamic approaches are able to predict the evolution of the experts’ prefer-ences. Nevertheless, the possible changes or evolution of the others problem parametersare not taken into account; while multi-period GDM approaches focus on the opinionsas the only changeable parameter.

4. The group of approaches that model the problem environment as a group of dynamicparameters (alternatives, experts, weights and criteria) focus on one environment pa-rameter at a time, thus to completely address the dynamism of the consensus processall the problem parameters should be treated as dynamic. The main issue associatedwith the analysed approaches is that their behaviour is always the same, regardless ofthe environment changes.

5. Adaptive consensus approaches address the issue of the fixed/static behaviour men-tioned above, as the corresponding models adapt their behaviour to the current problemsituation in an efficient way.

6. Each one of the analysed approaches focused on a single dynamic aspect of the con-sensus problem. Thus, a combination of different models would be desirable in orderto solve a real world consensus reaching problem.

The following future important research challenges in this field have been identified:

• Adaptive consensus modelling is still a hot topic that presents some good approachesbut it could be improved in some aspects to address real world situations that have notbeen modelled yet. For example, it could be interesting to study and develop modelsthat adapt the computation of the feedback advice to other dynamic parameters suchas the experts’ consistency or the expert’s changing aversion.

• One key issue of the current consensus model is that experts acceptance of the advicereceived from the feedback mechanism to increase the consensus level and to achieve aconvergent process is taken for granted. However, experts can decide not to implementthe feedback recommendations and keep their original opinions/preferences. Therefore,there is a need to provide a framework that adds legitimacy to the feedback process toconvince the experts to accept the provided advices. This can be achieved by imple-menting psychology concepts or persuasion principles in order to model the influenceas an important part of the consensus reaching processes [9]. Persuasion principles arebased on psychological studies about human behaviors. Therefore, if the consensusreaching processes incorporate some of these principles as a mean of influence, modelwith a higher power of persuasion to convince experts to be more flexible and reachthe collective agreement will be possible. Several persuasion principles of interest are:

21

– Returning a favour is usually a tendency among people.

– When people see other individuals doing something, they commonly do the samething, even if they do not understand the reasons why.

– Authority figures normally impose some rules that others comply with no ques-tions.

– Supply and demand law causes effect on people when they have to make decisions.

– When some people like an individual, they can be easily influenced by him/her.

• In any case, it is quite important to add some arguments to the provided opin-ions/preferences to support their correctness and help persuade others to follow you.Thus, dynamic and argumentative consensus models implementing argumentation mech-anisms to the current consensus approaches are desirable. Credibility measures seemsto be required for such approaches, which could also be modelled as an element ofdynamic nature in the consensus process.

Summarising, we have identified two main research gaps that could be considered as newchallenges to be addressed by the research community. The first one is the constructionof new adaptive consensus models in order to make more realistic and efficient feedbackmechanisms; while the second one refers to the need of new methods to deal with noncooperative experts where persuasion models and/or argumentation models might play akey role in modelling the consensus process.

5. Conclusions

In this paper, we have analysed in a comprehensive way some consensus approachesin which the model’s parameters or properties have been specifically designed to changedynamically over the discussion time. To do that, we have proposed a taxonomy to classifythe current dynamic consensus reaching models.

Classical dynamic consensus approaches are those that only admit changes on the experts’preferences, with the remaining parameter of the decision problem remaining fixed or static.The dynamisms can be introduced in the consensus process by incorporating a variableof time. Two different time modelling strategies have been identified: opinion dynamicapproaches that aim at predicting the experts’ preferences on a time state based on a setof previous time states; and multi period approaches that propose the aggregation of thedifferent preferences at different time periods. A third group of reviewed approaches goa step further by modelling the decision environment as a dynamic one by allowing otherproblem parameters –set of alternatives; set experts set; experts’ importance; or set ofcriteria– to change and not remain fixed throughout the consensus process. Finally, thelast group of dynamic consensus approaches analysed have been the adaptive consensusapproaches that incorporate implement feedback mechanisms that adapt at each consensusstage to the current collective consensus level, experts’ importance, or individual consensusindexes. The most productive research avenues with their strengths and weaknesses havebeen analysed, and future research challenges have been pointed out. Table 5 summarisesthe most important analysed techniques, with corresponding their references, as per thesefour different methodologies

22

Methodologies Techniques References

Classical CA Change of Preferences [8, 17, 18, 22, 32,

34–36, 44, 45]

Fast Convergence [1, 24, 39, 40, 50]

Time Modelling CA Opinion Dynamics [11, 14, 21, 28]

Multi-Period GDM [20, 37, 46–49]

Dynamic Environment Dynamic Set of Alternatives [39–41]

Dynamic Set of Experts [2, 3]

Dynamic Experts’ Importance [10, 13, 16, 27,

29]

Dynamic Set of Criteria [15, 31]

Adaptive Consensus Approaches Collective Consensus Level [33]

Experts’ Importance [38]

Individual Consensus Indeces [13]

Table 5: Taxonomy of methodologies and techniques for dynamic consensus approaches.

Acknowledgments

The authors would like to acknowledge the financial support from FEDER funds receivedfrom the Spanish Department for Economy and Competitiveness project (TIN2016-75850-R);the Chinese NSF grant (No. 71571124); and the University of Cadiz project (PR2016-026).

References

[1] S. Alonso, E. Herrera-Viedma, F. Chiclana, and F. Herrera. A web based consensussupport system for group decision making problems and incomplete preferences. Infor-mation Sciences, 180(23):4477–4495, 2010.

[2] S. Alonso, I.J. Perez, F.J. Cabrerizo, and E. Herrera-Viedma. A linguistic consensusmodel for web 2.0 communities. Applied Soft Computing, 13(1):149–157, 2013.

[3] M.A. Ballester and J.L. Garcıa-Lapresta. A recursive group decision-making proce-dure for choosing qualified individuals. International Journal of Intelligent Systems,24(8):889–901, 2009.

[4] F.J. Cabrerizo, R. Al-Hmouz, A. Morfeq, A.S. Balamash, M.A. Martınez, andE. Herrera-Viedma. Soft consensus measures in group decision making using unbal-anced fuzzy linguistic information. Soft Computing, 21(11):3037–3050, 2017.

[5] F.J. Cabrerizo, E. Herrera-Viedma, and W. Pedrycz. A method based on pso and granu-lar computing of linguistic information to solve group decision making problems definedin heterogeneous contexts. European Journal of Operational Research, 230(3):624–633,2013.

23

[6] F.J. Cabrerizo, J.M. Moreno, I.J. Perez, and E. Herrera-Viedma. Analyzing consensusapproaches in fuzzy group decision making: advantages and drawbacks. Soft Computing,14(5):451–463, 2010.

[7] M. Cao, A. Morse, and B. Anderson. Reaching a consensus in a dynamically chang-ing environment: A graphical approach. SIAM Journal on Control and Optimization,47(2):575–600, 2008.

[8] M. Cao, A. Morse, and B. Anderson. Reaching a consensus in a dynamically changingenvironment: convergence rates, measurement delays, and asynchronous events. SIAMJournal on Control and Optimization, 47(2):601–623, 2008.

[9] N. Capuano, F. Chiclana, H. Fujita, E. Herrera-Viedma, and V. Loia. Fuzzy groupdecision making with incomplete information guided by social influence. IEEE Trans.On Fuzzy Systems, Doi: 10.1109/TFUZZ.2017.2744605, 2018.

[10] G. Daniello, M. Gaeta, S. Tomasiello, and L. Rarita. A fuzzy consensus approach forgroup decision making with variable importance of experts. International Conferenceon Fuzzy Systems (FUZZ-IEEE), Beijing, China, pages 1693–1700, 2016.

[11] G. Deffuant, D. Neau, F. Amblard, and G. Weisbuch. Mixing beliefs among interactingagents. Advances in Complex Systems, 3(01n04):87–98, 2000.

[12] M. J. del Moral, J.M. Tapia, F. Chiclana, and E. Herrera-Viedma. A comparative studyon consensus measures in group decision making. International Journal of IntelligentSystems, Doi: 10.1002/int.21954, 2018.

[13] Q. Dong and O. Cooper. A peer to peer dynamic adaptive consensus reaching model forthe group ahp decision making. European Journal of Operational Research, 250:521–530,2016.

[14] Y.C. Dong, X. Chen, H. Liang, and C. Li. Dynamics of linguistic opinion formation inbounded confidence model. Information Fusion, 32:52–61, 2016.

[15] Y.C. Dong, H. Zhang, and E. Herrera-Viedma. Consensus reaching model in the complexand dynamic magdm problem. Knowledge-Based Systems, 106(1):206–219, 2016.

[16] Y.C. Dong, H. Zhang, and E. Herrera-Viedma. Integrating experts’ weights generateddynamically into the consensus reaching process and its applications in managing non-cooperative behaviors. Decision Support Systems, 84(1):1–15, 2016.

[17] M. Fedrizzi, M. Fedrizzi, and R.A. Pereira. Soft consensus and network dynamics ingroup decision making. International Journal of Intelligent Systems, 14(1):63–77, 1999.

[18] M. Fedrizzi, M. Fedrizzi, and R.A. Pereira. Consensus modelling in group decisionmaking: dynamical approach based on fuzzy preferences. New Mathematics and NaturalComputation, 3(02):219–237, 2007.

24

[19] E. Forman and K. Peniwati. Aggregating individual judgments and priorities with theanalytic hierarchy process. European journal of operational research, 108(1):165–169,1998.

[20] M. Gupta and B.K. Mohanty. An algorithmic approach to group decision making prob-lems under fuzzy and dynamic environment. Expert Systems with Applications, 55:118–132, 2016.

[21] R. Hegselmann and U. Krause. Opinion dynamics and bounded confidence models,analysis, and simulation. Journal of Artificial Societies and Social Simulation, 5(3),2002.

[22] E. Herrera-Viedma, S. Alonso, F. Chiclana, and F. Herrera. A consensus model forgroup decision making with incomplete fuzzy preference relations. IEEE Transactionson Fuzzy Systems, 15(5):863–877, 2007.

[23] J. Kacprzyk. Group decision making with a fuzzy linguistic majority. Fuzzy Sets andSystems, 18:105–118, 1986.

[24] J. Kacprzyk and S. Zadrozny. Soft computing and web intelligence for supportingconsensus reaching. Soft Computing, 14(8):833–846, 2010.

[25] B. Kitchenham, O. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman. Sys-tematic literature reviews in software engineering–a systematic literature review. Infor-mation and software technology, 51(1):7–15, 2009.

[26] Patrick R Laughlin. Social choice theory, social decision scheme theory, and groupdecision-making. Group Processes & Intergroup Relations, 14(1):63–79, 2011.

[27] D. Li, W. Zeng, and L. Li. Fuzzy group decision making based on variable weigthedaveraging operators. International Conference on Fuzzy Systems (FUZZ-IEEE), Beijing,China, pages 1416–1421, 2014.

[28] G. Li, G. Kou, and Y. Peng. Dynamic fuzzy multiple criteria decision making for per-formance evaluation. Technological and Economic Development of Economy, 21(5):705–719, 2015.

[29] H. Li, L. Li, J. Wang, Z. Mo, and Y. Li. Fuzzy decision making based on variableweights. Mathematical and computer Modelling, 39(2-3):163–179, 2004.

[30] W. Liu, Y. Dong, F. Chiclana, F.J. Cabrerizo, and E. Herrera-Viedma. Group decision-making based on heterogeneous preference relations with self-confidence. Fuzzy Opti-mization and Decision Making, 16(4):429–447, 2017.

[31] R. Lourenzutti and R. Krohling. A generalized topsis method for group decision mak-ing with heterogeneous information in a dynamic environment. Information Sciences,330(1):1–18, 2016.

25

[32] P. Luukka, M. Collan, and M. Fedrizzi. A dynamic fuzzy consensus model with randomiterative steps. In System Sciences (HICSS), 2015 48th Hawaii International Conferenceon, pages 1474–1482. IEEE, 2015.

[33] F. Mata, L. Martınez, and E. Herrera-Viedma. An adaptive consensus support modelfor group decision making problems in a multi-granular fuzzy linguistic context. IEEETransactions on Fuzzy Systems, 17(2):279–290, 2009.

[34] I. Palomares, L. Martinez, and F. Herrera. Mentor: a graphical monitoring tool ofpreferences evolution in large-scale group decision making. Knowledge-Based Systems,58:66–74, 2014.

[35] R.O. Parreiras, P. Ekel, and F. Bernardes. A dynamic consensus scheme based on anonreciprocal fuzzy preference relation modeling. Information Sciences, 211:1–17, 2012.

[36] R.O. Parreiras, P. Ekel, J.S.C Martini, and R.M. Palhares. A flexible consensus schemefor multicriteria group decision making under linguistic assessments. Information Sci-ences, 180(7):1075–1089, 2010.

[37] D. Peng and H. Wang. Dynamic hesitant fuzzy aggregation operators in multi-perioddecision making. Kybernetes, 43(5):715–736, 2014.

[38] I.J. Perez, F.J. Cabrerizo, S. Alonso, and E. Herrera-Viedma. A new consensus modelfor group decision making problems with non homogeneous experts. IEEE Transactionson Systems, Man, and Cybernetics: Systems, 44(4):494–498, 2014.

[39] I.J. Perez, F.J. Cabrerizo, and E. Herrera-Viedma. A mobile decision support systemfor dynamic group decision making problems. IEEE Transactions on Systems, Man andCybernetics - Part A: Systems and Humans, 40(6):1244–1256, 2010.

[40] I.J. Perez, F.J. Cabrerizo, and E. Herrera-Viedma. Group decision making problems ina linguistic and dynamic context. Expert Systems with Applications, 38(3):1675–1688,2011.

[41] I.J. Perez, F.J. Cabrerizo, and E. Herrera-Viedma. A mobile group decision makingmodel for heterogeneous information and changeable decision contexts. InternationalJournal of Uncertainty, Fuzziness and Knowledge-Based Systems, 19(1):33–52, 2011.

[42] R. Ramanathan and L.S. Ganesh. Group preference aggregation methods employed inahp: An evaluation and an intrinsic process for deriving members’ weightages. EuropeanJournal of Operational Research, 79(2):249–265, 1994.

[43] T.L. Saaty. Fundamentals of decision making and priority theory with the analytichierarchy process, volume 6. Rws Publications, 2000.

[44] J. Wu and F. Chiclana. Visual information feedback mechanism and attitudinal priori-tisation method for group decision making with triangular fuzzy complementary pref-erence relations. Information Sciences, 279:716–734, 2014.

26

[45] J. Wu, F. Chiclana, H. Fujita, and E. Herrera-Viedma. A visual interaction consensusmodel for social network group decision making with trust propagation. Knowledge-Based Systems, 122:39–50, 2017.

[46] Z. Xu. On multi-period multi-attribute decision making. Knowledge-Based Systems,21(2):164–171, 2008.

[47] Z. Xu. A method based on the dynamic weighted geometric aggregation operator fordynamic hybrid multi-attribute group decision making. International Journal of Un-certainty, Fuzziness and Knowledge-Based Systems, 17(1):15–33, 2009.

[48] Z. Xu. Multi-period multi-attribute group decision-making under linguistic assessments.International journal of general systems, 38(8):823–850, 2009.

[49] Z. Xu and R.R. Yager. Dynamic intuitionistic fuzzy multi-attribute decision making.International Journal of Approximate Reasoning, 48(1):246–262, 2008.

[50] S. Zadrozny and J. Kacprzyk. An internet-based group decision and consensus reach-ing support system. In Applied decision support with soft computing, pages 263–276.Springer, 2003.

27