Guiding Genetic Algorithm via Viral Trait Spreading for Solving Sudoku Puzzle

Nico Saputro and Veronica S. MoertiniDepartment of Informatics

Parahyangan Catholic UniversityBandung, Indonesia

Email: {nico@unpar.ac.id, nico@siu.edu},moertini@unpar.ac.id

Abstract—Genetic Algorithms work iteratively from gener-ation to generation to find the optimal solution of optimiza-tion problems. However, due to the probabilistic operations ofGenetic Algorithms (GAs), the performance of GAs search isunpredictable. Even worst, GAs may not be able to find theoptimal solution after very long iteration. We propose a solutionthat incorporates a human intervention to guide GA achievinga better performance. We adopt the Viral Trait SpreadingFramework for human intervention in the GA operations. Firstly,we classify GAs operation and then put each group of operationin the Framework. Most of all genetic algorithm operations fallinto the Trait Adoption component. We optimized the designof genetic representation and genetic operators to tackle thefixed element constraint and row permutation constraint ofSudoku puzzle. Then, we implemented our approach in netlogo,a multiagent programmable modeling environment. Experimentresults showed that GA is capable of finding the optimal solutionand the human intervention through Viral Trait SpreadingFramework guides the GA in searching processes in the narrowersearch space.

Keywords-Genetic algorithm; viral trait adoption; Sudokupuzzle;

I. INTRODUCTION

Genetic Algorithms (GAs) are often used for finding so-lutions of optimization problems. To find the solutions, GAiteratively searches them in a pool of candidate solutions,called population, using genetic operators and fitness func-tion. Before it begins the searching process, several commonparameters must be set. After all of these have been set and ge-netic operators have been defined, GA works iteratively fromgeneration to generation in searching the optimal solutions inthe population without any intervention from users. GA willstop the searching processes after the maximum generation isexceeded or the optimal solution is found. The users couldonly accept the GA computation results.

The need of human or user intervention during GA search-ing processes become apparent when human or user have acertain knowledge or believe about part of the solutions. Forinstance, GA can be used to solve Sudoku puzzle. While GAis searching for the solution, sometimes users know or believethat a certain Sudoku number should be placed in a certaincell. Unfortunately, there is no other way to let GA know aboutit. If the user can intervene GA and inform his knowledge,GA can speed-up the searching processes by narrowing thefeasible search space. For this reason, multiagent system

will be used to model the user interaction in GA searchingprocesses.

In multi-agent systems, the level of human intervention canbe categorized into two approaches: Man in the Loop (MITL)and Man on the Loop (MOTL). While the former directlyaffects the action of a specific agent, the later indirectly affectsthe actions at the unit or agent community level [1]. Thedisadvantage of MITL approach is in the micromanagementof each agent’s parameter and perhaps also in the manualcomputation of that parameter. It takes times and causesa bottleneck in the loop. On the other hand, the Man-On-The-Loop (MOTL) paradigm removes the role of the humanadministrator in becoming the cause of the bottleneck in theloop and placed it on the loop, such that the human acts asan observer that has the capability to assert control wheneverneeded.

Viral Trait Spreading, a system framework that allowsan administrator to influence and guide the development ofa massive intelligent agent has been proposed in [2]. Thisframework is inspired by the survival of the fittest and viralspread paradigm. Traits coming from the successful agentscan be adopted by the other agent or can be introduced bythe administrator.

Both concepts, MOTL and Viral Trait Spreading, lookpromising for answering the purposes of human interventionin the GA operation. Therefore, the contributions of our workare: We explore the genetic algorithm in a broader view andlooking for the possible implementation of multi-agent ingenetic algorithms, and then we adopt Viral Trait Spreadingas a mean of human intervention in the Genetic Algorithmoperations. Since Viral Trait Spreading and Genetic Algorithmview the systems from the different perspectives (Viral TraitSpreading view the system in the human perspective while Ge-netic Algorithm in the level of natural genetic, tiny life forms),we choose to place the genetic algorithm context in the humanperspective. We classify the Genetic algorithm operationsand fit them into the Viral Trait Spreading component. Wealso propose an optimized genetic representation and geneticoperators that handled two constraints out of four constraintsof Sudoku puzzle so that the fixed element constraints androw permutation constraints are not violated during the geneticoperations. Here, the fitness function only deals with two otherconstraints, which are column permutation constraint and sub

2011 First International Conference on Informatics and Computational Intelligence

978-0-7695-4618-6/11 $26.00 © 2011 IEEE

DOI 10.1109/ICI.2011.33

149

2011 First International Conference on Informatics and Computational Intelligence

978-0-7695-4618-6/11 $26.00 © 2011 IEEE

DOI 10.1109/ICI.2011.33

149

grid permutation constraint.This paper is organized as follows: In the next section,

Section 2, we presented a brief literature review of geneticalgorithms, Sudoku puzzle, and viral trait spreading frame-work. In Section 3, we discuss our approach in detailed.First, we describe the integration of genetic algorithm in theviral trait spreading framework and map the genetic algorithmcomponents into each component of the framework. Then theoptimized genetic representation and genetic operators, as wellas the fitness function are explained. Three types of agents arealso introduced in this section. The multiagent implementationand results are discussed in section 4. Finally, conclusion andfuture works are discussed in the last section.

II. LITERATURE REVIEW

A. Genetic Algorithms

Genetic Algorithms (GAs) are gaining attentions from manyresearchers for years. As a search algorithm that works basedon the natural selection and natural genetic in finding the exactor approximate solutions to optimization problems, geneticalgorithms have been used in many different fields, such asengineering, manufacturing, linguistics, economics, music, art,and bioinformatics. Due to various problems addressed byGAs, different approaches and level of sophistication occurredin genetic algorithms. For example, certain approaches couldbe proposed in genetic representations of the solution domain,genetic operators and selection methods, the complexity of thefitness functions. Combined methods with other optimizationalgorithms could also be proposed. Memetic algorithms orhybrid genetic algorithms are the mixed methods of geneticalgorithms with local search algorithms such as hill-climbing,simulated-annealing, and taboo search. Other approaches areincorporated human in the genetic algorithm processes, forexample: (1) Interactive Genetic Algorithm (IGA) [3], whichhas an interactive human interface to evaluate the members ofthe population since there is no formal model for fitness func-tion such as beauty, harmony, and creativity. (2) Human-BasedGenetic Algorithms (HBGAs), which have an interface tofacilitate human to interact with the genetic algorithm [3] [4].(3) Cultural Algorithms (CAs), which are the extension ofGenetic Algorithms with additional knowledge componentscalled belief space [5] [6].

B. Sudoku Puzzle

Sudoku, also known as Number Place or Nanpure, is alogic-based puzzle game. The game board is composed ofa 9x9 grid that is divided further into nine 3x3 sub grids. Thegame starts with various initial digits given in some cells.Each cell is either empty or is a fixed element containing aninteger from the set {1, ..., 9}. The number of the given initialnumber in the puzzle does not determine the difficulty of thepuzzle [7]. Fig. 1 shows an example of the initial conditionof a Sudoku puzzle where 30 initial numbers are given.

Fig. 1. Sudoku Puzzle

The goal of the puzzle is to place an integer from the set{1, ..., 9} in each cell such that each row, column, and sub gridcontains a unique integer number once and only once. Thesesituations can be seen as a Constraint Satisfaction Problem(CSP) which has 4 constraints: Fixed element constraint, rowpermutation constraint, column permutation constraint, andsub grid permutation constraint. The multi-agent approachto solve Sudoku puzzle as a Constraint Satisfaction Problem(CSP) can be seen in [8]. It should be noted that a stan-dard Sudoku puzzle can be quickly solved by a computer,the simplest method could be a trial-and-error search withbacktracking [9]. To improve the efficiency of this brute forceapproach, a Sudoku solver should incorporate the four con-straints mentioned earlier. These simpler methods, however,consume exponential time in the worst case when the Sudokupuzzle is expanded into the general n2 x n2 grid composedof n x n blocks (where each block contains the numbers 1through n2). This generalized scenario may also be viewed asa graph coloring problem and a Latin square problem whichis NP-complete.

C. Viral Trait Spreading Framework [2]

Viral Trait Spreading Framework is a framework wherethe model is based on the concept of survival of the fittestand viral spread. Its aim is consistent with Man on the Loop(MOTL) goal which allows an administrator to influence andguide the development of a massive network of intelligentagents. Agents adopt the traits of their successful peers or canbe artificially adjusted by the administrator. The frameworkconsists of three main components: Trait Adoption, CandidateIdentification, and Trait Injection as shown in Fig. 2.

In the Trait Adoption component that occurs during agentinteraction, the unsuccessful agent will adopt the traits ofthe more successful agent, either using probabilistic or fuzzyadoption method. Social Network Analysis is used in theCandidate Identification component to determine the mostinfluential and ideal candidates of agents that are outstandingor vital in the network based on certain criteria. The criteriacould be how close an agent to all other agents in thenetwork, or how many information pass through an agent. In

150150

Fig. 2. Viral Trait Spreading framework

the Trait Injection component, administrator guides agents bymodifying the traits of some influential agents and injectingthem back into the agents.

III. APPROACH

In our approach, we integrate genetic algorithm into theViral Trait Spreading Framework [2]. Here, we view thegenetic algorithm in the context of multi-agent system. Inthis context, we define an agent, called genetic agent, whichwould have knowledge of a certain solution to an optimizationproblem. In order to find a better solution, each genetic agentcan interact with other genetic agent through a probabilisticselection mechanism and exchange their knowledge using ananalogy of GA’s crossover operator. It also has the capabil-ity of reasoning and partially changing its own knowledgethrough an analogy of GA’s mutation operator. The selectionmechanism, the agent’s exchange of knowledge, and agentreasoning fit in the Trait Adoption component. Fig. 3 showsthe genetic algorithm approach in each Viral Trait Spreadingcomponent.

For 9x9 grid Sudoku puzzle, a genetic agent has 81elements of knowledge. Each element represents a singlecell of Sudoku puzzle and consists of two values: a Sudokunumber ∈ {1, ..., 9} and a dominance sign ∈ {0, 1, 2}. This

Fig. 3. Mixed Genetic Algorithm Viral Trait Spreading approach

Fig. 4. Sudoku puzzle and agent solution (black color is fixed element)

Fig. 5. The knowledge representation from Fig. 4

dominance sign is inspired by dominance gene and recessivegene in the natural genetic. It is used to distinguish between arecessive knowledge’s element (i.e., knowledge’s element thathas a non-fixed Sudoku number) and dominance knowledge’selement (i.e., knowledge’s element that has a fixed Sudokunumber). A recessive knowledge’s element has the dominancesign of 0, while dominance knowledge’s element has a valueof either 1 or 2 as its dominance sign. The dominancesign of 2 denotes a fixed pre-defined Sudoku number (i.e.,fixed element) and the dominance sign of 1 denotes a fixedSudoku number as a result of the trait injection operation. Thefirst nine elements in the knowledge representation in Fig. 5represent this row. It shows that the value of the dominancesign for the corresponding fixed pre-defined Sudoku numbersin the knowledge is 2.

1) Trait Adoption: There are four main components of thegenetic algorithm approach adopted in trait adoption: Agentselection, agent’s knowledge exchange, agent’s knowledgemodification, and fitness function. In the agent selectionprocess, a genetic agent which will be involved in the knowl-edge exchange process between agents or an agent where itsknowledge will be modified based on a random probabilityis selected. We propose Tournament based selection [10] foragent selection. Tournament selection is a robust selectionmechanism commonly used by GAs. Several randomly chosengenetic agents are selected to participate in the tournament.The champion genetic agent of the tournament (i.e. has thebest fitness) is selected, which then will be used for knowledgeexchange or knowledge modification.

The knowledge exchange between genetic agents will occurwhen the probability of a pair of genetic agents is lessthan or equal to predefined crossover threshold. The operatoremployed for the knowledge exchange between agents is themodified version of Precedence Preservative Crossover (PPX)[11]. Here, PPX is modified in order to cope with the fixedelements constraint and the non-uniqueness of the Sudoku

151151

Algorithm 1 Per-row knowledge exchange(row-knowledge ofparent1, row-knowledge of parent2)

Let d be the duplicate of a row-knowledge from a parentthat has the lowest recessive elements. Set all Sudokunumber of the recessive elements in d to 0Let N be the number of recessive elements in dLet S be the set of Sudoku numbers of the dominanceelements in dLet r1 be the set of Sudoku numbers in parent1 /∈ SLet r2 be the set of Sudoku numbers in parent2 /∈ SLet V be a vector of length N. V is randomly filled withelements of the set {1, 2}for j:=1 to N do

if V[j]=1 thenLet x be the leftmost Sudoku number in r1

elseLet x be the leftmost Sudoku number in r2

end ifReplace with x, the first found recessive element in dwhere its Sudoku number = 0Delete x from both r1 and r2

end forreturn d

numbers in the whole knowledge, i.e., there are nine duplicatevalues of set {1, ..., 9} in an 81 knowledge’s elements. Inbuilding the new knowledge, we apply PPX operator to thepair of nine knowledge’s elements, i.e., per row knowledgeexchange as shown in Algorithm 1. To preserve the dominanceknowledge’s elements in each row, we copy these dominanceelements to the new knowledge at the same exact positions andthen perform PPX operator to the rest, i.e., recessive elements.To build the whole new knowledge, we repeat the algorithmnine times.

A modified swap-mutation operator is used to perform self-modification of genetic agent’s knowledge. Like the PPX oper-ator, we propose per-row swap-mutation. In this way, the swapoperation only occurs between two recessive knowledge’selement. Elitism method is used to maintain the continuationof the current best fitness into its successive generations. Inthis case, the current best genetic agent is protected and unableto modify its knowledge.

Fitness function is used to measure the quality of the geneticagent’s knowledge. Equation (1), which is a part of a simplefitness function proposed in [7], is used to evaluate the qualityof the proposed solution. This fitness function penalizes thecolumn violations (2), subgrid violations (3), missing numberin columns (4), and missing number in subgrids (5). The fixedelements and row constraints are already satisfied during theinitial development of the agent’s knowledge and in the designof knowledge exchange operator and knowledge modificationoperator. The best fitness is equal to zero, which indicates thatthere is no constraint violation.

f(x) = w1 ×

9∑j=1

colj (x) +3∑

k=1

3∑l=1

gridk,l (x)

+ w2 ×

9∑j=1

mcolj (x) +

3∑k=1

3∑l=1

mgridk,l (x)

(1)

where w1 and w2 are weight and :

colj (x) =

∣∣∣∣∣45−9∑

i=1

xi,j

∣∣∣∣∣ (2)

gridk,l (x) =

∣∣∣∣∣∣45−3×(k−1)+3∑

i=3×(k−1)+1

3×(l−1)+3∑j=3×(l−1)+1

xi,j

∣∣∣∣∣∣ (3)

mcolj (x) =

∣∣∣∣∣A−9∑

i=1

xi,j

∣∣∣∣∣ (4)

mgridk,l (x) =

∣∣∣∣∣∣A−3×(k−1)+3∑

i=3×(k−1)+1

3×(l−1)+3∑j=3×(l−1)+1

xi,j

∣∣∣∣∣∣ (5)

where A = {1, 2, 3, 4, 5, 6, 7, 8, 9} and || denotes for thecardinality of a set.

2) Candidate Identification: Candidate Identification dealswith the analysis of the most influential agents and determineswhich agents are the ideal candidates for modification viainjection. We propose a simple identification method to findthe candidate based on the best current genetic agent with thelowest fitness.

3) Trait Injection: In Trait Injection component, a humanadministrator gathers the information from the environmentand manually injects certain trait, i.e. Sudoku number valueand its position, through a human interface. We propose asimple mechanism for trait injection using the best currentgenetic agent with the lowest fitness. We duplicate this agent,perform a swap-mutation operator, and change the dominancesign to 1. In order to help the human administrator to monitorthe current best solution of Sudoku puzzle, a group of infoagents, where each info agent has the responsibility to monitorthe constraint violation in a row, column, or subgrid, will sendmessages to the human administrator by changing its color tored or green. If info agent at each particular row, column, orsubgrid, finds constraint violations in its position, it set itscolor to red. If otherwise, it is set to green.

IV. IMPLEMENTATION AND RESULTS

We implemented our propose method using netlogo, a mul-tiagent programmable modeling environment. Besides threetypes of agent mentioned previously, which are Genetic Agent,Info Agent, and Human Administrator, there is another agentcalled puzzle agent. Puzzle agent is an agent that occupies a

152152

cell in the Sudoku puzzle and can change its state (from 1to 9) based on the information from the current best geneticagent. There are 81 puzzle agents, where each puzzle agenthas its unique position on the puzzle board. Each puzzle-agenthas four possible colors: black, grey, blue, and red. Blackcolor is used for fixed-element cell; grey and blue colorsrepresent an empty-cell which will change to another coloreach time its state change. Puzzle-agent will have red colorin a certain cell if it receives an injected trait from the humanadministrator. There are 27 info-agents, where 9 info-agentsare used to monitor row violation, 9 info-agents are usedto monitor column violation, and 9 info-agents are used fordetecting subgrid violation. Each info agent will set its colorto red or green depending on whether there is a violation inthe info agent responsibility. If there is a violation, the infoagent sets its color to red, otherwise it sets to green.

Our application takes several inputs: (1) A text file con-taining a Sudoku puzzle problem. (2) Pre-defined parameters:population size that define how many hidden genetic-agentsworking together, mutation rate, and crossover rate. (3) An on-off button to activate the elitism method. It also has a humancontrol panel for trait injection, where a human administratorcan use it to inject traits, i.e. a Sudoku number and its position(row and column). There are six outputs of the application:(1) The Sudoku board and its current best solution. (2 and3)Genetic operator statistic: the average knowledge exchangeand average knowledge modification in each of the iterations.(4) The best fitness. (5) The fitness plot. (6) The number ofgenetic agent infected by the trait. Fig. 6 shows the interfaceof our proposed method.

Our simulation shows that Genetic Algorithm is able tofind the solution of a given Sudoku problem without thetrait injection. From several trials, the minimum generationfor finding the solution is 44 generations as shown in Fig.7. Our simulation also shows that the trait injection workswell. Fig. 8 shows a puzzle-agent with the red color, whichis the injected Sudoku number given by human administrator.The injected Sudoku number value is placed in the intendedcell, where the old value of this cell placed into another cell.This operation uses a swap mutation operator. After the swapmutation is performed, the dominance sign of the injected

Fig. 6. The interface of our proposed method

Fig. 7. Best solution for a given Sudoku puzzle

Fig. 8. Effect of subsequence injected traits

Sudoku number is changed to 1. Fig. 9 illustrates the process,for instance when the human administrator injected Sudokunumber 5 into the second knowledge’s element. After the swapoperation, the dominance sign in this element is changed to 1.The speed of the traits spreading into the other genetic agentscan also be seen in the number of genetic agents infectedby the injected trait area. It takes less than 10 generation tospread the traits to the other genetic agents.

When we observe the effect of injected traits to the bestfitness, we find interesting results. Fig. 8 shows that Viral TraitSpreading also addresses the local minima problem. It guidesthe genetic agent towards the better solution. In the figure,we can see that the first and second injection of the correctSudoku number does not decrease the fitness value but instead

Fig. 9. Swap operation for injected trait

153153

it raises the fitness value. However, when the third injectionis introduced, it causes sudden drop in the fitness value.

V. CONCLUSION AND FUTURE WORK

Based on our experiments, we conclude that our proposedmethod works well for solving Sudoku puzzle. The humanintervention through the Viral Trait Spreading improves thesearch process by narrowing the search space of the geneticagent. For future work, we are going to extend our proposedmethod by employing an automated administrator who hasthe capability of recognizing and deciding which trait shouldbe injected to the system automatically. This can be done ifthe automated administrator has the AI capability. IntegratingExpert System to our proposed method is also possible.Currently, the human administrator does the work of analyzingand deciding which trait should be injected based on Sudokupuzzle board values and information obtained from the info-agents. Adding Expert System which is capable in givingadvices which Sudoku position and what is the correct value init will create smarter automated administrator. Another futurework is enhancing the candidate-identification component.Our proposed method simply picks the best agent as thecandidate. Social Network Analysis can further be adoptedin finding an appropriate agent.

REFERENCES

[1] H. Hexmoor, B. McLaughlan, and G. Tuli, “Natural human role insupervising complex control systems,” J. Exp. Theor. Artif. Intell.,vol. 21, pp. 59–77, March 2009.

[2] B. McLaughlan and H. Hexmoor, “Influencing massive multi-agentsystems via viral trait spreading,” in Proceedings of the 2010 FourthIEEE International Conference on Self-Adaptive and Self-OrganizingSystems Workshop, SASOW ’10, (Washington, DC, USA), pp. 287–292, IEEE Computer Society, 2010.

[3] C. Cheng and A. Kosorukoff, “Interactive one-max problem allowsto compare the performance of interactive and human-based geneticalgorithms,” in Genetic and Evolutionary Computation GECCO 2004(K. Deb, ed.), vol. 3102 of Lecture Notes in Computer Science, pp. 983–993, Springer Berlin / Heidelberg, 2004.

[4] A. Kosorukoff, “Human based genetic algorithm,” in Systems, Man, andCybernetics, 2001 IEEE International Conference on, vol. 5, pp. 3464–3469 vol.5, 2001.

[5] R. G. Reynolds, “An introduction to cultural algorithms,” in Proceedingsof the 3rd Annual Conference on Evolutionary Programming, pp. 131– 139, World Scientific Publishing, 1994.

[6] Z. Xue and Y. Guo, “Improved cultural algorithm based on genetic al-gorithm,” in Integration Technology, 2007. ICIT ’07. IEEE InternationalConference on, pp. 117 –122, march 2007.

[7] T. Mantere and J. Koljonen, “Solving, rating and generating sudokupuzzles with ga,” in Evolutionary Computation, 2007. CEC 2007. IEEECongress on, pp. 1382 –1389, sept. 2007.

[8] C. Agerbeck and M. Hansen, “A multi-agent approach to solvingnp-complete problems,” master thesis, Informatics and MathematicalModelling, Technical University of Denmark, Denmark, 2008.

[9] A. Moraglio, J. Togelius, and S. Lucas, “Product geometric crossoverfor the sudoku puzzle,” in Evolutionary Computation, 2006. CEC 2006.IEEE Congress on, pp. 470 –476, 0-0 2006.

[10] B. L. Miller and D. E. Goldberg, “Genetic algorithms, selectionschemes, and the varying effects of noise,” Evol. Comput., vol. 4,pp. 113–131, June 1996.

[11] C. Bierwirth and D. C. Mattfeld, “Production scheduling and reschedul-ing with genetic algorithms,” Evol. Comput., vol. 7, pp. 1–17, March1999.

154154