finding mds-optimal supersaturated designs using computer searches

Journal of Statistical Theory and Practice, 7:703–712, 2013Copyright © Grace Scientific Publishing, LLCISSN: 1559-8608 print / 1559-8616 onlineDOI: 10.1080/15598608.2013.781888

Finding MDS-Optimal Supersaturated DesignsUsing Computer Searches

ARDEN MILLER1 AND BOXIN TANG2

1University of Auckland, Auckland, New Zealand2Simon Fraser University, Burnaby, British Columbia, Canada

Supersaturated designs can be evaluated using the minimal dependent sets (MDSs) ofcolumns in the design matrix. This article describes an extensive computer search ofbalanced two-level supersaturated designs to find those that are MDS-optimal.

Keywords: Aberration; Hadamard matrix; Nonorthogonal design; Nonregular design;Screening design.

1. Introduction

Srivastava (1975) investigated the problem of assessing the ability of a screening designto distinguish between competing linear models—that is, the ability to pick out the truemodel from a set of candidate models. In this groundbreaking work, he introduced theidea of “resolving power” and showed that the ability of a design to distinguish betweentwo competing models was related to its ability to estimate the combined model. Buildingon this work, Miller and Sitter (2004) proposed using minimal dependent sets (MDSs) toevaluate the screening capacity of designs. This idea was expanded in Miller and Sitter(2005). Lin, Miller, and Sitter (2008) introduced the minimum MDS-aberration criteriaand applied it to nonorthogonal foldover designs. This approach was extended to two-levelsupersaturated designs by Miller and Tang (2012).

Research work on two-level supersaturated designs goes back to Booth and Cox(1962), who provided the first systematic construction of such designs. The literature onsupersaturated designs is quite rich, and important work includes Lin (1993), Wu (1993),Nguyen (1996), Tang and Wu (1997), Cheng (1997), Butler et al. (2001), Bulutogluand Cheng (2004) and Xu and Wu (2005). The criterion used most often to evaluatesupersaturated designs is E(s2), which represents the average of the cross products betweenall possible pairs of columns. It can be thought of as a measure of nonorthogonality—clearly a supersaturated design of n runs cannot be orthogonal because the number m offactors is greater than n – 1. This criterion is easy to use and makes intuitive sense, and hasplayed a very important role in the studies of supersaturated designs. However, E(s2) doesnot directly measure the ability of a design to screen for active factors. Miller and Tang(2012) considered the use of MDSs to evaluate the screening capability of supersaturateddesigns and presented some theoretical results on the number and structure of the MDSs

Received 16 March 2012; accepted 26 October 2012.Address correspondence to: Arden Miller, Department of Statistics, The University of

Auckland, Room 203, Science Centre Building, 38 Princess Street 203, Auckland, New Zealand.Email: [email protected]

703

704 A. Miller and B. Tang

in a supersaturated design. Unlike E(s2), there is a direct connection between criteria basedon MDSs and the ability of a design to identify active factors from a candidate set.

The present article reports the results of an extensive computer search for two-levelsupersaturated designs that are optimal with respect to the minimum MDS-aberration crite-ria defined by Lin et al. (2008). These designs are also optimal or near-optimal with respectto (i) the resolving power as defined by Srivastava (1975), (ii) the estimation capacity crite-rion used by Cheng, Steinberg, and Sun (1999), and (iii) the resolution rank criterion usedby Deng, Lin, and Wang (1999). A minimal dependent set (MDS) is a set of columns froma matrix that are linearly dependent, but if any one of the columns is removed, then theresulting set is linearly independent. Miller and Sitter (2004) argued that the ability of ascreening design to differentiate between competing models is directly linked to the MDSsof its design matrix. In simple terms, problems in differentiating between two models canoccur if the combined set of columns associated with those models is linearly dependent.As any linearly dependent set is either an MDS or contains an MDS, the set of MDSs deter-mines which pairs of competing models can be differentiated from each other. Further, asthe number of columns in an MDS decreases the negative impact it has increases—a smallMDS will create problems in differentiating between models involving less factors thana large MDS. Lin et al. (2008) defined MDS-resolution and MDS-aberration as criteriathat can be used to rank designs with respect to their ability to differentiate between com-peting models. Let Ai be the number of MDSs of size i and define the MDS sequence as(A1, A2, . . ., Ak). The MDS-resolution is defined as the smallest i such that Ai �= 0. Thus, anMDS sequence of (0, 0, 0, 1, 3, 4) would indicate a matrix that has no MDSs of size ≤ 3, oneMDS of size 4, three MDSs of size 5, and four MDSs of size 6. Such a design would have anMDS-resolution of 4. Suppose that the MDS sequences for two matrices are compared entryby entry. For the smallest i such that the Ai’s are not identical, the matrix with the smallerAi is said to have less MDS-aberration. For a given class of matrices, a matrix is said tohave minimum MDS-aberration if no other matrix in the class has less MDS-aberration.

We restrict our attention to two-level supersaturated designs for p factors in n runs thatare balanced in that for each factor each level occurs for n/2 of the runs. Miller and Tang(2012) showed that for all such designs A1 = A2 = A3 = 0 and that the maximum possiblesize for an MDS is n. Thus, all MDS sequences reported in this article are of the form (A4,A5, . . ., An). Since every supersaturated design must have at least one MDS (i.e., Ai �= 0 forsome i), the best possible MDS-resolution is n. Miller and Tang (2012) show that MDS-resolution equal to n occurs if and only if every subset of size n is an MDS—that is, An =(p choose n). Thus, if a design exists with an MDS sequence that has A4 = A5 = . . . =An−1 = 0 and An = (p choose n) it must be the minimum MDS-aberration design.

In section 2, the computer algorithms used to search for the MDS-optimal designs aredescribed. These include the algorithms used to evaluate the MDS sequence of a design,the algorithm used to test for isomorphism between two designs, and the algorithms usedto search for the MDS-optimal designs. In section 3, the results of the computer searchesare presented. Also, details of how the best designs found in this search can be accessedthrough a database set up by the authors are given. Section 4 contains a brief discussion ofsome interesting aspects of the MDS-optimal designs found by the computer searches.

2. Computer Algorithms

2.1. The Scope of the Computer Search

This article focuses on searching for balanced two-level designs that have the best possibleMDS sequence for a given run size n and number of factors p where n = 6, 8, . . . 36 and

Finding Supersaturated Designs 705

p = n, n + 1, . . . n + 4. Each design is represented by an n × p binary matrix (levelsare represented as either −1 or +1) where each column contains the levels for one factor.Further, we only consider designs where no two factors are completely confounded witheach other and thus the full search set can be defined as all n × p binary matrices that obeythe following two restrictions:

1. For each column half of the entries must be +1’s and the other half must be −1’s.2. No column can be identical to or the mirror image of any other column.

Since mirror images are not allowed, the number of distinct columns is c =(n choose n/2)/2 and the number of matrices in the full search set is p! × (c choose p).The full search sets for many of the scenarios considered in this paper are exceedinglylarge – as an example for n = 12 and p = 14 then the full search set contains approximately1.65 × 1037 designs. In order to make the computer searches more effective, a subset ofthe full search set was identified such that the entire range of nonisomorphic designs in thefull search set would still be represented in the reduced search set. The motivation was that(i) it would be possible to do a complete census of all designs in the reduced search set forlarger values of n and p than would be the case for the full search set and (ii) for values ofn and p where it was only feasible to take a sample from the search set then (for a fixedsample size) better coverage of the range of nonisomorphic designs should be obtained bysampling from the reduced search set rather than the full search set. We reduced the searchset by only considering designs where (i) the first row consisted entirely of +1’s and (ii)the first column consisted of +1 repeated n/2 times followed by −1 repeated n/2 times.Note that any design in the full search set is isomorphic to a design that obeys these tworestrictions. Although defining a reduced search set in this manner substantially reducesthe number of designs, this set can still be very large. Continuing with the example of n =12 runs and p = 14 factors, there are 11 choose 5 = 462 balanced columns with +1 in thefirst position. Thus, fixing the first column as already indicated and selecting the remainingcolumns from this set results in 461 choose 13 ≈ 5.75 × 1024 designs in the reduced searchset. So although the size of the search set has been decreased by more than a factor of 1012,there are still a huge number of designs in the reduced search set. Using this reduced searchset made it feasible to conduct a complete enumeration of designs for n = 6 and n = 8 butnot for n ≥ 10.

2.2. Search Algorithms

Two different search algorithms were employed: one algorithm that was used to obtain acomplete enumeration of designs for n = 6 and n = 8 and a somewhat different algorithmthat was used to obtain an extensive (but not exhaustive) search of designs where n ≥ 10.These algorithms are similar but there are some important differences. Therefore, the com-plete enumeration algorithm for n = 6 and n = 8 is described first and then the searchalgorithm for n ≥ 10 is described. Key components of both of these search algorithms are(i) a function that finds the MDS-sequence for a given design and (ii) a function that checksto see if two designs are isomorphic. As both of these tasks are nontrivial and were neededto be performed a large number of times for each search, it was important to base these func-tions on efficient algorithms. These algorithms are described in the subsections 2.3 and 2.4.

To do a complete enumeration of nonisomorphic designs for n = 6 and n = 8, thefollowing algorithm was used:

1. Create the complete set of candidate column vectors—this is the set of all balancedvectors of length n that have +1 in the first position.


2. For p = 1 there is only one isomorphic set. This set is represented by the single columnvector that consists of +1 repeated n/2 times followed by −1 repeated n/2 times.

3. For p = 2:(a) Add vectors from the candidate set one at a time to the vector for p = 1.(b) Put the first design created in this way into the set of nonisomorphic designs for

p = 2.(c) For each subsequent design, check to see whether the design is isomorphic to any

design already in the set of nonisomorphic designs for p = 2. If not, add the designto the set.

4. For 2 < p ≤ n + 4 do the following steps for each nonisomorphic design of size p − 1:(a) Add the column vectors from the candidate set one at a time (omit those vectors

that are already in the design).(b) If the new design is not isomorphic with any design in the set of nonisomorphic

designs of size p add it to this set (the first design created of size p is automaticallyput in this set).

5. For p = n through p = n + 4 find the MDS-sequence for each element of thenonisomorphic set of designs with n rows and p columns.

For n ≥ 10, the large number of nonisomorphic designs for values of p ≥ 8 made thepreceding approach of doing a complete census of all nonisomorphic designs for all p ≥2 unfeasible. Instead, extensive searches for the designs that had the best possible MDS-sequences for p ≥ n were undertaken. First, designs of size p = n – 1 were randomlygenerated and those of full rank selected. Then designs with p = n, n + 1, . . . n + 4 werecreated by adding columns one at a time to these designs. At each step, designs of sizep were created by adding columns to the designs of size p – 1 which had the best MDS-sequences. The logic for this is that when an n × p design matrix is created by adding acolumn to an n × (p – 1) matrix, then any MDS for the n × (p – 1) must also be an MDSfor the n × p matrix. Thus, if the starting n × (p – 1) matrix has a poor MDS-sequence(i.e., one that contains small MDSs) then the resulting n × p matrix must also have a poorMDS-sequence. The following search routine was used:

1. For p = n – 1, n × p matrices were randomly generated such that the first row of eachmatrix consisted entirely of +1’s and each column contained n/2 +1’s and n/2 −1’s.

2. The rank of each matrix was found via a singular value decomposition. Matrices of rank< n −1 were discarded, leaving just the full rank matrices.

3. The set of full rank matrices was reduced to a nonisomorphic set of designs (of size p =n −1) via isomorphism checking.

4. For p = n, matrices were generated by adding a column to the nonisomorphic designsof size p = n −1. For n ≤ 14 all possible additional columns were checked but for n ≥16 it was only feasible to try adding a sample of the possible additional columns. Thep = n matrices generated in this manner were then screened and the matrices that hadan MDS with size < n −1 were discarded. The remaining matrices were checked forisomorphism and the set of nonisomorphic matrices were saved. The MDS sequenceswere found for this set of matrices and used to order the matrices.

5. For p = n + 1, . . . n + 4, matrices were generated by adding a column to a matricesfrom the set of non-isomorphic matrices of size p – 1. As in the previous step, a set ofnonisomorphic matrices was created and the matrices were ranked according to theirMDS sequences.


Note that as the number of runs n increased, it was necessary to be increasingly morerestrictive in step 5 with respect to both the number of designs of size p – 1 used and thenumber of candidates for the added column considered for each such design of size p – 1.In all cases, the highest ranked designs of size p – 1 were used but the number of these thatcould be considered decreased as n increased. Further, for smaller values of n it was possibleto systematically try adding all possible additional columns to the starting matrices, but forlarger values of n it was only feasible to randomly select a sample of additional columns tobe added. As a result, the coverage of our searches (i.e., the proportion of nonisomorphicdesigns sampled for given n and p) almost certainly decreases as n increases.

2.3. MDS Sequence Algorithm

Our algorithm to find the MDS-sequence is based on Lemma 1 in Miller and Tang (2012):A set of column vectors from a design matrix forms an MDS if and only if one of thevectors can be uniquely expressed as a linear combination of the remaining vectors whereevery coefficient is nonzero. That is, the set of vectors V1, . . . , Vm is an MDS if and only ifVm can be written as Vm = a1V1 + · · · + am−1Vm−1 where ai �= 0 for all i and the ai’s forma unique set of real numbers. Note that this implies that any solution to the vector equation∑m

1 ciVi = 0 cannot have any ci = 0. Thus, there must be a single linear dependency thatinvolves all of the vectors—our algorithm uses the singular value decomposition of thematrix formed by the column vectors V1, . . . , Vm to check this condition.

The algorithm used to generate the MDS-sequence for an n × p matrix M (where p ≥n) can be described as follows:

1. Set the minimum subset size to 4 and the maximum subset size to n. For s = 4, 5, . . . n,do steps 2 through 4.

2. Generate all possible submatrices that consist of s columns of M.3. Find the singular value decomposition of each submatrix and check the following condi-

tions: (i) There are exactly s – 1 nonzero singular values (i.e., one singular value is zero)and (ii) the right singular vector corresponding to the zero singular value does not haveany entries equal to zero. Note that if both these conditions are true then the submatrixis an MDS of column vectors.

4. Count the number of submatrices that satisfy both conditions in step 3.

2.4. Checking for Isomorphism

To check for isomorphism between two matrices, it is necessary to determine whether onematrix can be constructed from the other through some combination of (i) permuting rows,(ii) permuting columns, and (iii) taking the mirror image of some set of the columns. Thus,for an n × p matrix, there are n!p!2p possible arrangements to be considered. Clearly thisnumber becomes exceedingly large as n and p increase—for p = n = 12 the number ofarrangements is > 1020 and for p = n = 20 it is > 1042. The algorithm used to checkfor isomorphism employs techniques to make this process as efficient as possible. First, a“pretest” is done to check whether the two matrices have the same singular values. If twomatrices are isomorphic they must have identical sets of singular values, whereas noni-somorphic matrices may or may not have identical sets of singular values. Thus, if thematrices have nonidentical singular values they can be designated as nonisomorphic, but ifthey have identical singular values then further testing is necessary. This prescreening onsingular values appears to have been very effective for our purposes since it is much quicker


to get the singular values using R than it is to create all possible rearrangements of one ofthe matrices and (for the class of designs we considered) it appears that it is relatively rareto have two nonisomorphic matrices with identical singular values. Second, the algorithmemployed a simple method to determine whether one matrix could be obtained just throughpermuting and/or “mirror-imaging” the columns of the other—this meant that it was onlynecessary to actually permute the rows of one of the matrices.

Second, the algorithm employed a simple method to determine whether one matrixcould be obtained just through permuting and/or “mirror-imaging” the columns of theother—this meant that it was only necessary to actually permute the rows of one of thematrices. It was feasible to do this since all the matrices we considered had the propertythat no two columns were either identical or mirror images of each other. Let M1 and M2 betwo such n × p matrices. Then clearly M1 can be obtained just through permuting and/or“mirror-imaging” the columns M2 if and only if each column of M1 is either identical toor the mirror image of exactly one column of M2. To check this we can simple look atMt

1M2 and count the number of entries that are equal to ±n. If there are p such entries, eachcolumn of M1 is either identical to or the mirror image of exactly one column of M2 and itfollows that the two matrices must be isomorphic.

The algorithm used to check for isomorphism between two n × p matrices M1 and M2

can be described as follows:

1. Compare the singular values of the two matrices. If the matrices do not have identicalsingular values then they can be declared to be nonisomorphic and no further checkingis done. If the matrices have identical singular values then proceed to step 2.

2. Count the number of entries of Mt1M2 that are equal to ±n. If there are p of these the

matrices are declared isomorphic and no further checking is done.3. Systematically generate all matrices M∗

2 that can be formed by permuting the rows ofM2. For each of these, repeat step 2 with M∗

2 taking the place of M2. If any M∗2 is found

to isomorphic to M1, then M1 and M2 are declared isomorphic and the procedure stops.If all possible M∗

2 are checked and none are found to be isomorphic to M1, then M1 andM2 are declared to be nonisomorphic.

3. Results and Discussion

This article can be considered as a companion article to Miller and Tang (2012), whichexplored theoretical aspects of using MDS sequences to evaluate supersaturated designs.This article presents actual designs that are MDS-optimal (i.e. minimum MDS-aberrationdesigns) or close to MDS-optimal. An archive of the best designs found by our searches isavailable from the authors upon request.

Searches were done to find the MDS-optimal balanced n-run supersaturated designsfor n = 6, 8, . . . 36. Designs that contain up to five factors beyond saturation (p = n, n +1, . . . n + 4) were explored. Table 1 summarizes the best (least aberration) MDS sequencesfound for these n-run p-factor designs. Out of the 80 entries in this table, it can be said withabsolute certainty that 66 represent minimum MDS-aberration designs. This statement isbased on two facts:

1. A complete census of nonisomorphic designs was done for all cases where n = 6 orn = 8.

2. If a design exists with an MDS sequence that has A4 = A5 = . . . = An−1 = 0 and An =(p choose n) it must be the minimum MDS-aberration design (a justification for thisassertion is given in the introduction). For many of the combinations of n and p inTable 1, designs were found that satisfy this condition.


Table 1Best MDS sequences found for n-run p-factor designs

n p MDS sequence (A4, A5, . . ., An) n pMDS sequence(A4, A5, . . ., An)

6 6 0 0 1 22 22 0 0 . . . 0 0 17 2 0 1 23 0 0 . . . 0 0 238 5 0 2 24 0 0 . . . 0 0 2769 9 0 6 25 0 0 . . . 0 0 2300

10 15 0 15 26 0 0 . . . 0 0 149508 8 0 0 0 0 1 24 24 0 0 . . . 0 0 1

9 0 0 0 3 3 25 0 0 . . . 0 0 2510 0 0 0 10 15 26 0 0 . . . 0 0 32511 0 0 10 11 35 27 0 0 . . . 0 0 292512 0 0 21 24 102 28 0 0 . . . 0 0 20475

10 10 0 0 0 0 0 0 1 26 26 0 0 . . . 0 0 111 0 0 0 0 0 1 9 27 0 0 . . . 0 0 2712 0 0 0 0 0 7 45 28 0 0 . . . 0 0 37813 0 0 0 0 2 28 154 29 0 0 . . . 0 0 365414 0 0 0 0 18 70 429 30 0 0 . . . 0 0 27405

12 12 0 0 0 0 0 0 0 0 1 28 28 0 0 . . . 0 0 113 0 0 0 0 0 0 0 0 13 29 0 0 . . . 0 0 2914 0 0 0 0 0 0 0 0 91 30 0 0 . . . 0 0 43515 0 0 0 0 0 0 0 14 399 31 0 0 . . . 0 0 449516 0 0 0 0 0 0 7 84 1295 32 0 0 . . . 0 0 35960

14 14 0 0 0 0 0 0 0 0 0 0 1 30 30 0 0 . . . 0 0 115 0 0 0 0 0 0 0 0 0 0 15 31 0 0 . . . 0 0 3116 0 0 0 0 0 0 0 0 0 0 120 32 0 0 . . . 0 0 49617 0 0 0 0 0 0 0 0 0 27 572 33 0 0 . . . 0 0 545618 0 0 0 0 0 0 0 0 4 97 2528 34 0 0 . . . 0 0 46376

16 16 0 0 0 0 0 0 0 0 0 0 0 0 1 32 32 0 0 . . . 0 0 117 0 0 0 0 0 0 0 0 0 0 0 0 17 33 0 0 . . . 0 0 3318 0 0 0 0 0 0 0 0 0 0 0 0 153 34 0 0 . . . 0 0 56119 0 0 0 0 0 0 0 0 0 0 0 7 941 35 0 0 . . . 0 0 654520 0 0 0 0 0 0 0 0 0 0 0 76 4465 36 0 0 . . . 0 0 58905

18 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 34 34 0 0 . . . 0 0 119 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 35 0 0 . . . 0 0 3520 0 0 0 0 0 0 0 0 0 0 0 0 0 0 190 36 0 0 . . . 0 0 63021 0 0 0 0 0 0 0 0 0 0 0 0 0 5 1310 37 0 0 . . . 0 0 777022 0 0 0 0 0 0 0 0 0 0 0 0 0 47 7080 38 0 0 . . . 0 0 73815

20 20 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 36 36 0 0 . . . 0 0 121 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 37 0 0 . . . 0 0 3722 0 0 0 0 0 0 0 0 0 0 0 0 0 0 231 38 0 0 . . . 0 0 70323 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1767 39 0 0 . . . 0 0 913924 0 0 0 0 0 0 0 0 0 0 0 0 0 19 10531 40 0 0 . . . 0 0 91390


Thus, there are only 14 entries in Table 1 for which it cannot be said with certainty thatthey represent minimum MDS-aberration designs: the entries for n = 10 and p = 11 through14 and the entries for n = 12 through 20 and p = n +3 or n + 4. However, these entries arebased on very extensive computer searches and thus the authors are extremely confidentthat they represent designs that are either minimum MDS-aberration designs or are veryclose to minimum MDS-aberration designs.

An important question arises: For fixed n, what is the maximum number of columnsp that a balanced two-level supersaturated design can contain and have an MDS sequenceof the (optimal) form A4 = A5 = . . . = An−1 = 0 and An = (p choose n)? Note that a suc-cinct way of designating designs of this form is that they have MDS-resolution n. Refer tothis number as p(n). Miller and Tang (2012) present an algorithm that produces an n-runbalanced two-level supersaturated design for p = n factors that has MDS-resolution n forany n ≥ 6. Thus, a lower bound of p(n) ≥ n has been established for n ≥ 6. The computersearches done for this article for n = 6 and for n = 8 were in fact complete censuses ofnonisomorphic designs, and no designs that improved on this lower bound were found. Thesearches for n ≥ 10 did not represent complete censuses and thus these searches can pro-vide an improvement on the lower bound for p(n) but do not establish its value; note thatthe coverage of the set of non-isomorphic designs decreased as n increased. The search forn = 10 also did not find any design that improved on the lower bound, the searches for n =12 to n = 20 were able to raise the lower bound to p(n) ≥ n + 2, and the searches for n =22 to n = 36 raised the lower bound to p(n) ≥ n + 4. The following table summarizes ourfindings:

n 6 8 10 12 − 20 22 − 36p(n) 6 8 ≥10 ≥ n + 2 ≥ n + 4

The general trend is that as n increases, the number of columns beyond saturationthat can be included in a balanced two-level supersaturated design of MDS-resolutionn increases. This leads us to the following conjecture: p(n) − n is a nondecreasing func-tion. Our belief in this conjecture is strengthened by the following observation from oursearches: For n ≥ 22 as n increased it became increasingly easy to find designs with p = n+ 4 columns that had MDS-resolution n. In particular, for n = 22 we only found one such(nonisomorphic) design, for n = 24 we found seven such (nonisomorphic) designs, and forn ≥ 26 we were able to identify hundreds of such (nonisomorphic) designs. An additionalintriguing observation is that the lower bound for p(n) − n jumps by 2 from n = 10 to n =12 and again by 2 from n = 20 to n = 22. We do not know of any explanation as to whythis should occur.

Computer searches were also performed for orthogonal base designs of size n = 12,16, . . . 36. These are n-run designs where the first n – 1 columns form an orthogonalmatrix and additional columns are added to this “orthogonal base.” This type of designhas been explored by Deng, Lin, and Wang (1996) and Yamada and Lin (1997). Millerand Tang (2012) proved that it is not possible to have an orthogonal base design of size≤ 16 that has MDS-resolution n. For n = 12, 16, 20, and 24 we did exhaustive searchesfor adding one additional column to an orthogonal base—that is, for each n we used everynonisomorphic Hadamard matrix as the orthogonal base and tried adding every possiblebalanced column to it. For n = 28, 32, and 36, exhaustive searches were not feasible, so wesampled both the set of nonisomorphic Hadamard matrices that we used as the orthogonalbase and the set of balanced columns that were added. The following table records themaximum MDS-resolution Rmax that we found for each n:


n 12 16 20 24 28 32 36Rmax 10 14 18 23 27 32 36

Note that for n ≤ 24, these results provide an upper bound on the maximum MDS-resolution possible for any orthogonal base supersaturated design of that size. As a result,we conclude that restricting n-run balanced supersaturated designs to those that contain anorthogonal base of n – 1 columns results in designs that are substantially less desirable interms of MDS aberration. As a case in point, consider designs of size n = 24. If we startwith a orthogonal base and add even one column, the best we can do is MDS-resolution 23.However, if we don’t restrict to an orthogonal base, then we can add at least five columnsbeyond saturation and still achieve MDS-resolution 24 (see Table 1).

4. Concluding Remarks

As mentioned previously, an archive of the best designs found by our computer searches isavailable from the authors upon request. This archive contains files that allow design matri-ces to be created using R for all of the MDS-optimal designs corresponding to Table 1. Formost values of n and p, a number of nonisomorphic designs that result in the best possibleMDS-sequence are given and in some cases (mainly for smaller values of n) alternativedesigns that come close to the MDS-optimal designs are also included.

Acknowledgment

The research of Boxin Tang is supported by the Natural Sciences and Engineering ResearchCouncil of Canada.

ReferencesBooth, E. H., and D. R. Cox. 1962. Some systematic supersaturated designs. Technometrics, 4,

489–495.Bulutoglu, D. A., and C. S. Cheng. 2004. Construction of E(s2)-optimal supersaturated designs. Ann.

Stat., 32, 1662–1678.Butler, N. A., R. Mead, K. M. Eskridge, and S. G. Gilmour. 2001. A general method of constructing

E(s2)-optimal supersaturated designs. J. R. Stat. Soc. B, 63, 621–632.Cheng, C. S. 1997. E(s2)-optimal supersaturated designs. Stat. Sin., 7, 929–939.Cheng, C. S., D. M. Steinberg, and D. X. Sun. 1999. Minimum aberration and model robustness for

two level fractional factorial designs. J. R. Stat. Soc. Ser. B, 61, 85–93.Deng, L. Y., D. K. J. Lin, and J. Wang. 1996. Marginally oversaturated designs. Commun. Stat. Theory

Methods, 25(11), 2557–2573.Deng, L. Y., D. K. J. Lin, and J. Wang. 1999. A resolution rank criteria for supersaturated designs.

Stat. Sin., 9, 605–610.Lin, C. D., A. E. Miller, and R. R. Sitter. 2008. Folded over non-orthogonal designs. J. Stat. Plan.

Inference, 134, 3107–3124.Lin, D. K. J. 1993. A new class of supersaturated designs. Technometrics, 35, 28–31.Miller, A. E. and R. R. Sitter. 2004. Choosing columns from the 12-run Plackett–Burman design.

Stat. Probability Lett., 67, 193–201.Miller, A. E., and R. R. Sitter. 2005. Using folded over non-orthogonal designs. Technometrics, 47,

502–513.


Miller, A. E., and B. Tang. 2012. Using minimal dependent sets to evaluate supersaturated designs.Stat. Sin., 22, 1273–1285.

Nguyen, N. K. 1996. An algorithmic approach to constructing supersaturated designs. Technometrics,38, 69–73.

Srivastava, J. N. 1975. Designs for searching non-negligible effects. In A survey of statistical designand linear models. Amsterdam, The Netherlands, North-Holland.

Tang, B., and C. F. J. Wu. 1997. A method for constructing supersaturated designs and its E(s2)-optimality. Can. J. Stat., 25, 191–201.

Wu, C. F. J. 1993. Construction of supersaturated designs through partially aliased interactions.Biometrika, 80, 661–669.

Yamada, S., and D. K. J. Lin. 1997. Supersaturated design including an orthogonal base. Can. J. Stat.,25, 203–213.

Xu, H., and C. F. J. Wu. 2005. Construction of optimal multi-level supersaturated designs. Ann. Stat.,33, 2811–2836.

finding mds-optimal supersaturated designs using computer searches

Documents