■ CARMA is an advisory system for rangelandgrasshopper infestations that demonstrates how AItechnology can deliver expert advice to compen-sate for cutbacks in public services. CARMA uses twoknowledge sources for the key task of predictingforage consumption by grasshoppers: (1) casesobtained by asking a group of experts to solve rep-resentative hypothetical problems and (2) anumeric model of rangeland ecosystems. Theseknowledge sources are integrated through thetechnique of model-based adaptation, in whichcase-based reasoning is used to find an approxi-mate solution, and the model is used to adapt thisapproximate solution into a more precise solution.CARMA has been used in Wyoming counties since1996. The combination of a simple interface, flex-ible control strategy, and integration of multipleknowledge sources makes CARMA accessible to inex-perienced users and capable of producing advicecomparable to that produced by human experts.Moreover, because CARMA embodies diverse formsof expertise, it has been used in ways that its devel-opers did not anticipate, including pest manage-ment research, development of industry strategies,and in-state and federal pest-management policydecisions.

Grasshoppers are the most serious range-land pest in the western United States,consuming 21 to 23 percent of range-

land forage and causing an estimated $400 mil-lion in losses (Hewitt and Onsager 1983). Fig-ure 1 illustrates grasshopper infestationdensities in the western United States during2000, a fairly typical year. In years of heavyinfestation, grasshopper densities and econom-ic losses might be much higher. For example,during the 1986 to 1987 outbreak, over 20 mil-lion acres of rangeland were treated forgrasshoppers in the western United States at acost of more than $75 million.

In Wyoming, the estimated total annual lossto grasshoppers is roughly $19 million. Thesoutheastern quadrant of the state is particu-larly prone to grasshopper infestations, withsignificant areas of high-grasshopper densitiesin 30 of the last 34 years.

Various chemical and biological pesticidesare available for treatment of grasshopperinfestations, but the cost of using these agentsoften outweighs the value of the forage savedby their application. Moreover, indiscriminatepesticide application can be damaging torangeland ecology. Despite their potential fordamage, the majority of grasshopper speciesare usually innocuous or even beneficial tograssland ecosystems. Of more than 400species of grasshoppers in the western UnitedStates, perhaps only 15 can be considered seri-ous pests; many of the other species are bene-ficial in terms of weed control, nutrientcycling, and food for wildlife (Lockwood1993a, 1993b). The decision whether to useinsecticides or other control measures is a com-plex task depending on a multiplicity of fac-tors, including not only short-term economiccosts and benefits but also preservinggrasshoppers’ natural enemies (Joern andGaines 1990), safeguarding biodiversity, andprotecting environmental and human health.

Before 1996, the United States Departmentof Agriculture (USDA) paid the entire cost oftreatment on federal land, one-half the cost onstate land, and one-third the cost on privateland. In addition, the USDA provided inten-sive surveys and pest-management advice toranchers about treatment selection. Subse-quently, however, the USDA stopped provid-ing these subsidies (except for infestations onfederal rangelands that represent an immedi-ate threat to adjacent crops) and the level of

Articles

SUMMER 2002 49

CARMAA Case-Based

Rangeland Management Adviser

John Hastings, Karl Branting, Jeffrey Lockwood

Copyright © 2002, American Association for Artificial Intelligence. All rights reserved. 0738-4602-2002 / $2.00

ecosystem. Various approaches to behavioralprediction are possible. In systems for which aprecise model exists and accurate values ofstate variables can be determined, simulationcan be used to predict the system’s behavior.Alternatively, if there are sufficient historicaldata, empirical methods such as case-based rea-soning (CBR) (Aamodt and Plaza 1994), deci-sion tree induction (Quinlan 1993), or statisti-cal techniques can be lead to accurateprediction.

Precise models exist for the behavior ofmany simple physical systems. However, mod-els of agricultural, ecological, and other biolog-ical systems are often incomplete, eitherbecause a complete state description for suchsystems cannot be determined or because thenumber and type of interactions between sys-tem elements are poorly understood. More-over, although historical data often exist forsuch systems, they are often insufficient foraccurate prediction using empirical methods.As illustrated in figure 2, biological systemsoften occupy an intermediate point in the con-tinuum between highly analytic domains,such as celestial mechanics and the prediction

survey and logistical support was substantiallydecreased. This change in policy has increas-ingly shifted both the cost of treatment andthe task of determining when treatment isdesirable to ranchers themselves. CARMA wasdeveloped to help compensate for thedecreased availability of federal assistance byhelping ranchers identify and balance the fac-tors relevant to pest-control decisions.

Task DescriptionCARMA’s task is to help ranchers determine themost cost-effective responses to rangelandgrasshopper infestations within user-definedenvironmental constraints. Determining themost cost-effective responses requires, at aminimum, estimating (1) the value of the for-age that is likely to be consumed by grasshop-pers if no action is taken, (2) the value of theportion of this forage that would be saved incurrent and future years under each treatmentoption, and (3) the cost of each option.

Estimating grasshopper forage consumptionrequires predicting the behavior of a rangeland

Articles

50 AI MAGAZINE

Figure 1. Western United States Rangeland Grasshopper Densities for 2000.

of artifact behavior, and highly empiricaldomains, such as sociology (Allen and Hoek-stra 1992). In such biological systems, bothmodels and empirical data exist, but neither isas such sufficient for accurate prediction. Accu-rate prediction of the behavior of such systemsrequires exploitation of multiple, individuallyincomplete, knowledge sources.

Rangeland ecosystems typify biological sys-tems having an extensive but incompletecausal theory and limited empirical data.Although model-based reasoning can play arole in rangeland grasshopper management,there is a general recognition that the interac-tions affecting grasshopper population dynam-ics are too poorly understood and too complexto permit precise prediction through numericsimulation alone (Allen and Hoekstra 1992;Lockwood and Lockwood 1991; Pimm 1991).However, pest-management experts appearable to provide useful recommendations toranchers, indicating that other sources ofknowledge can compensate for the absence ofa complete rangeland ecosystem model.

CARMA’s performance objective is to emulateas closely as possible the performance of pest-management experts. The shortage of humanexperts makes it important for CARMA to be notonly accurate but also sufficiently intuitivethat it can easily be used and understood byany of its users, including ranchers, range man-agers (who often lack pest-management exper-tise), and pest managers (who might lack expe-rience with rangeland grasshoppers).

To explicate the process whereby expertsmake forage-loss estimations, we performed aprotocol analysis of “solve-aloud” problemsolving by several experts in rangelandgrasshopper management at the University ofWyoming (Hastings, Branting, and Lockwood1996). The protocol analysis suggested thatexperts predict the proportion of available for-age that will be consumed by grasshoppers bycomparing the current situation to prototypi-cal cases.

An example of a prototypical case is a moder-ate density of emerging grasshoppers in a cool,wet spring. In this situation, only a low propor-tion of forage is typically consumed because wetconditions both increase forage growth and pro-mote growth of fungal pathogens that decreasegrasshopper populations, and cool conditionstend to prolong the early developmental phasesduring which grasshoppers are most susceptibleto pathogens and other mortality factors. In pre-dicting forage consumption by comparing newcases to prototypical cases, such as the cool, wetspring prototype, experts appear to be using aform of CBR.

If a particular new case differs in some waysfrom the most similar prototypical case, theexpert can perform causal reasoning to adaptthe prediction associated with the case toaccount for the differences. For example, if thepopulation density of emerging grasshoppersin a cool, wet spring is high (rather than mod-erate), an expert might predict moderately low(rather than low) forage consumption becausehigher density generally means more con-sumption.

Experts seem to reason about prototypicalcases in terms of abstract features that are rele-vant to the expert’s model of rangeland ecosys-tems, such as grasshopper species, develop-mental phases, and population density. Incontrast, a rancher’s description is almostalways in terms of directly observable features,such as the color, size, and behavior ofgrasshoppers; temperatures; and precipitation.As a result, determining the most similar pro-totypical case requires inferring the relevantabstract features from a set of observations pro-vided by the rancher. Experts exhibit great flex-ibility in inferring these features. For example,if a rancher is unable to provide the informa-tion that discriminates most reliably amonggrasshopper species (for example, whether thegrasshoppers have slanted faces or a spur ontheir “throats”), the expert is able to ask ques-tions that are less reliable but easier to answer(for example, are the grasshoppers brown orgreen?).

If it appears that grasshoppers will consumeforage needed by livestock, the expert deter-mines which interventions are compatiblewith local conditions, using knowledge such asthat wet conditions preclude the use of thechemical malathion and that all chemicaltreatments are precluded by environmentalsensitivity. Finally, the expert estimates the rel-ative value of the forage saved in this andfuture seasons and the cost of each control

Articles

SUMMER 2002 51

Empirical(exemplars)

Analytic(models)

law biologicalsystem

behavior

celestialmechanics

socialsystem

behavior

artifactbehavior

Figure 2. The Continuum from Highly Empirical Domains to Highly Analytic Domains.

their conclusions. Moreover, entomologistscan generate causal predictions of the effects ofincremental variations on case facts.

Opportunism: Human experts can use avariety of different strategies to solve a singlegiven problem depending on the availableinformation. Human experts don’t address thesubgoals that arise in decision making in aninvariant order but adapt their problem-solv-ing behavior to the particular facts of a givencase.

In summary, the protocol analysis indicatedthat experts in rangeland pest managementuse an eclectic approach that includes case-based reasoning for consumption prediction,rules for inferring case features and acceptablecontrol measures, and causal reasoning foradaptation and explanation. Moreover, expertproblem solving is fast and tolerant of inaccu-racies in data.

Application DescriptionCARMA is designed to model the problem-solv-ing behavior of experts in managing grasshop-per infestations, as described in the previoussection. CARMA emulates expert human adviceby providing treatment recommendations sup-ported by explanations in terms of causal, eco-nomic, and pragmatic factors, including anumeric estimate of the proportion of forageconsumed and a cost-benefit analysis of thevarious treatment options.

measure based on market price. The expertthen advises the rancher to take the most eco-nomical action, either applying the most cost-effective control measure or doing nothing.Experts can justify their advice by appealing toan underlying causal model but seem to usethis model only in explaining and adapting thepredictions associated with prototypes and notin performing any sort of simulation.

The protocol analysis identified four impor-tant characteristics of human expert problemsolving in this field: (1) graceful degradation,(2) speed, (3) causal explanations, and (4)opportunism.

Graceful degradation: Human experts canuse, but do not require, highly precise informa-tion of the type required for accurate model-based reasoning. Less accurate information candegrade the quality of advice an expert cangive but doesn’t preclude useful advice. In theworst case, human experts can provide plausi-ble advice based merely on the location of therangeland and the date.

Speed: Human experts can provide usefuladvice quickly, suggesting, consistent with ourprocess description, that human experts canuse highly compiled knowledge in the form ofprototypical cases.

Explanations in terms of a causal model:Although the speed and graceful degradationof human expert performance suggest thatexperts can use compiled knowledge, they canalso readily provide causal explanations for

Articles

52 AI MAGAZINE

User'sobservations:

SizeColor

Number observed...

Derivedfeatures:

SpeciesDevelopmental phase

Density...

Adaptedconsumption

estimate withouttreatment

Consumptionestimate under

acceptabletreatmentoptions

Cost-benefitanalysis oftreatmentoptions Markov model

of reinfestationprobability

Pesticideefficacy andgrasshopper

consumptionmodels

Heuristicrules

Casematching

Model-basedcase adaptation

Prototypicalcase

Prototypicalcase

Figure 3. The Main Steps in CARMA’s Consultation Process.

OverviewCARMA’s consultation process, summarized infigure 3, consists of the following steps: First,determine the relevant facts of the infestationcase from information provided by the user bymeans of heuristic rules. Second, estimate theproportion of available forage that will be con-sumed by each distinct grasshopper popula-tion (that is, subcase) by matching and adapt-ing the prototypical infestation cases that bestmatch the facts of the current case. Third, com-pare total grasshopper consumption with theproportion of available forage needed by live-stock. Fourth, if the predicted forage consump-tion will lead to economic loss, determinewhat possible treatment options are excludedby the case conditions. Fifth, provide an eco-nomic analysis for each viable treatmentoption by estimating both the first-year andlong-term savings. CARMA’s overall architectureis depicted in figure 4.

Determining Relevant Case FeaturesCARMA begins a consultation by eliciting obser-vations from the user through a window-basedinterface. These observations are used to inferthe relevant features of a new case, such as thespecies, population density, and developmen-tal phases of the grasshoppers. CARMA uses mul-tiple levels of rules for inferring each case fea-ture, ordered by a qualitative estimate of eachrule’s accuracy or reliability. The rules areapplied in succession until either the user can

provide the necessary information, or a defaultrule is reached.

For example, if the value of the case feature“total number of grasshoppers per square yard”is unknown to the user, CARMA instructs theuser to estimate the number of grasshoppersthat would be present in 18 square-foot circles(2 square yards) and divide the total by 2. If theuser can’t provide this information, the systemattempts to infer this feature using the heuris-tic that grasshopper density is equal to two-thirds the number of grasshoppers seen hop-ping away from the user with each step takenin the field. Otherwise, the value defaults tothe historic average for the area. By applyingrules in order of their accuracy or reliability,CARMA reasons with the best information avail-able.

A typical interface window for determiningthe observed grasshopper-type distributionappears in figure 5. It includes the options whyfor describing why this information is impor-tant to the consultation, help for advising theuser about the various window features andtheir operations, how to to explain the properprocedure for gathering the required informa-tion, not sure to trigger the selection of an alter-native rule for inferring the feature, back toreturn to the previous screen in the consulta-tion, and OK to accept the answer chosen bythe user. Display planthopper shows a smallinsect in the order Homoptera that resembles animmature grasshopper in both form andbehavior and that the user should distinguish

Articles

SUMMER 2002 53

Probabilistic reasonerReinfestation probability

Markov infestation model

calculator

Model-based reasonerProjection module

Critical-period adapter

Rangeland ecosystem model

Feature weights

Pesticide efficacy model

Consultation manager

Provide cost-benefit analysis

Infer relevant case features

Determine treatments options

Predict forage consumption

Case matcher

Feature adapter

Prototypical cases

New case

Case-basedreasoner

Rule-based reasoner

Feature inference rules

Feature inference module

Treatment exclusion rules

General rules

Tasks Modules Information Information paths

Infestation history map

Rangeland value map

Weather station info.

informationGeographical

Figure 4. Organizational Overview of CARMA’s Components.

particular, representative point in time selectedby the entomologist. In general, this represen-tative point is one at which the grasshoppersare at a developmental phase in which treat-ment is feasible. An example prototypical caseappears as case 18 in table 1.

A tract of rangeland almost invariably con-tains multiple grasshopper species, which candiffer widely in consumption characteristics. Inparticular, grasshoppers that spend the winteras nymphs consume far less during the growingseason than grasshoppers overwintering aseggs. CARMA therefore partitions the overall pop-ulation of a new case into subcases according tooverwintering type. For example, the new caseset forth in table 1 is split into two subcas-es—(1) subcase A and (2) subcase B—based onoverwintering type. Prototypical cases each rep-resent a single grasshopper population.

To predict the forage loss of a subcase, CARMA

first retrieves all prototypical cases whose over-wintering type matches that of the subcase. Theweighted sum of feature differences betweeneach prototypical case and the new subcase iscalculated to determine the most similar proto-typical case. Match weights are determinedfrom the mutual information gain betweencase features and qualitative consumption cat-egories in a given set of training cases(Wettschereck and Dietterich 1995). Separatematch weights are computed for each grasshop-per overwintering type’s seven case features: (1)precipitation, (2) temperature, (3) rangelandvalue, (4) infestation history, (5) average devel-opmental phase, (6) density, and (7) feedingtype. Quantitative features, such as density, areconverted to qualitative values for computationof mutual information gain because smallquantitative variations seemed to have littleeffect on matching. The difference between twoindividual feature values is determined by find-ing the difference between the positions of thevalues in an ordered qualitative feature valuelist. For example, rangeland value can equalone of the qualitative values in the ordered set{low, low-moderate, moderate, high-moderate,and high}, so that the matching feature differ-ence between low and high, the maximum pos-sible difference, is 4. The forage-loss predictionassociated with the given case is then adaptedto compensate for differences between the cur-rent case and the most similar prototypical caseusing model-based adaptation, discussed in thenext subsection.

Forage-Loss EstimationAfter adaptation, the consumption predictionsfor each subcase are summed to produce anoverall consumption estimate. Because of vari-

from a recently hatched grasshopper nymph.Because a complete case specification is not

always required for useful advice, CARMA fills inthe facts of a new case opportunistically, askingthe user for information only when the corre-sponding case feature is required for the rea-soning process to continue. At the earliestpoint at which a decision can be made, thecase-feature inference process halts, advice isgiven, and the consultation is ended. Thisopportunistic policy minimizes the amount ofinput required for CARMA to make a decision,thereby accelerating consultations. For exam-ple, if the date and location of an infestationindicate that it is too early to assess the severityof a grasshopper infestation, CARMA advises theuser to rerun the consultation at a later timewithout prompting for further information.

Case MatchingThe protocol analysis indicated that pest man-agers estimate forage consumption by compar-ing new cases to prototypical cases. These pro-totypical cases differ from conventional casesin two important respects. First, the prototypi-cal cases are not expressed in terms of observ-able features (for example, “Whenever I take astep, I see four grasshoppers with brightly col-ored wings fly”), but, rather, in terms ofabstract-derived features (for example,“Approximately six nymphal-overwinteringgrasshoppers in the adult phase per squareyard”). Second, the prototypical cases areextended in time, each representing the histo-ry of a particular grasshopper population overits life span, and stored as a “snapshot” at a

Articles

54 AI MAGAZINE

Figure 5. Elicitation of Grasshopper-Type Information in CARMA.

ability resulting from the imprecise nature ofrangeland ecosystems, this estimate is convert-ed to a qualitative range (for example, high,meaning that approximately 60 to 100 percentof the available forage will be lost). The win-dow explaining estimated forage loss, shownin figure 6, gives both aggravating and mitigat-ing factors (that is, factors tending to increaseand factors tending to reduce estimated forageloss). Explanation text in this and other CARMA

windows is produced using conventionalschema-based techniques (Moore 1995). If theproportion of available forage that will be lostto grasshoppers, and the proportion needed forlivestock (and wildlife) exceeds 100% of theforage available, CARMA concludes that grass-hoppers will cause economic losses.

Determining Treatment OptionsIf grasshoppers will cause economic losses, CAR-MA applies a set of rules to determine the treat-ment options that are excluded by the condi-tions of the case. Some of the information

necessary for determining exclusion is alreadyknown from the case features (for example, thepresence of grasshoppers in the first nymphalinstar—the earliest, readily observable develop-mental phase that hatches from the egg pod,which lies beneath the soil surface, into theabove-ground environment—indicates anongoing hatch, which precludes malathionand carbaryl bait from consideration). Otherconditions must be determined from furtheruser input (for example, will it be hot at thetime of treatment? If so, exclude malathion.).

Treatment RecommendationFor each acceptable treatment option, CARMA

provides estimates of the reduced probabilityof future reinfestation and current-year andlong-term savings. From the estimated savings,carma recommends the treatment or treat-ments that are most economical.

CARMA calculates the total reduced probabili-ty of future reinfestation for each treatmenttype using a Markov model of infestation prob-

Articles

SUMMER 2002 55

New CaseCase 18Subcase A Subcase B

Case 18 afterProjection

OverwinteringType

Egg Nymph Egg Egg

Feeding TypesGrass 90%

Mixed 10%Grass 100%

Grass 40%

Mixed 60%

Grass 90%

Mixed 10%

Average Phase 5.0 7.0 4.5 4.5

Density 32.0 3.0 27.0 33.3

Date June 29 June 20 June 25

Precipitation Normal Dry Normal

Temperatures Normal Cool Normal

Infestation History Moderate High-Moderate Moderate

RangelandValue

Moderate-Low Moderate Moderate-Low

Forage Loss 90% (high) ? 90% (high)

Table 1. Case Examples.

save the most under a best-case scenario. Usu-ally, the worst- and best-case scenarios producethe same recommended treatment. Followingthe treatment recommendation, the initialphase of the consultation is complete.

Optionally, the user can then rerun the con-sultation with one or more case facts or treat-ment parameters altered. To facilitate the alter-ing of treatment parameters, CARMA includes atreatment-matrix window, shown in figure 8,that permits the user to change the default val-ues of any of the variables (for example, cost ofchemical, cost of carrier, rate of application,cost of application, expected efficacy) thatdetermine the cost of an acre protected of eachtreatment option or the cost of adding entirelynew treatments. Although calculation of costof an acre protected is straightforward, experi-ence has shown that it is a common source oferrors, particularly when users are attemptingto determine the costs of reduced agent-area

ability for each location derived from historicaldata collected by the USDA and synthesized bythe University of Wyoming Entomology Sec-tion (Lockwood and Kemp 1987). CARMA com-putes the current-year savings as the differencebetween the value of forage saved and thetreatment cost. CARMA calculates the savings forfuture years for each treatment type by multi-plying the reduced probabilities of reinfesta-tion by the estimated forage loss for each sub-sequent year.

A typical treatment recommendation win-dow, including estimates of future reinfesta-tion and economic savings, appears in figure 7.CARMA lists both worst- and best-case scenariosfor most calculations. Note that this analysisincludes no treatment as an option and thatnegative savings indicate a loss.

CARMA recommends the treatment that isestimated to save the most under a worst-casescenario and the treatment that is estimated to

Articles

56 AI MAGAZINE

Figure 6. CARMA’s Forage-Loss Estimation Window.

Figure 7. CARMA’s Treatment Recommendation Screen.

treatments (RAATs), which involve applyinginsecticide to only a fraction of the infestedarea. The treatment matrix permits entirelynew parameters and products to be added toCARMA as they are developed, an important fea-ture in view of rapid recent refinements ingrasshopper management methods. Easy mod-ification of treatment parameters permits“gaming” to explore which combination ofinsecticide, rate, carrier, and coverage providesthe best return on investment. For example,such gaming can permit ranchers and pesticidevendors to negotiate a price for a pesticide thatis economical given the current infestation lev-el and the price of substitute forage.

Uses of AI TechnologyCARMA uses AI technology in two distinct ways.First, as described earlier, CARMA’s control strat-egy emulates human experts’ speed, oppor-tunism, explanation capability, flexibility ineliciting relevant case features through a vari-ety of alternative heuristic rules, and ability tointegrate multiple knowledge sources. Second,CARMA uses model-based adaptation for the keyreasoning step of predicting the amount of for-age that will be consumed by grasshoppers.

Model-based adaptation consists of usingCBR to find an approximate solution and mod-el-based reasoning to adapt this approximatesolution into a more precise solution. Model-

based adaptation is useful in domains in whichboth cases and models are available, but nei-ther is individually sufficient for accurate pre-diction. Such domains are typified by chemicalor biological systems with well-developed, butimperfect, models. Model-based adaptationhas been applied for bioprocess recipe plan-ning in SOPHIST (Aarts and Rousu 1996; Rousuand Aarts 1996) for selecting colorants for plas-tic coloring in FORMTOOL (Cheetham and Graf1997) and in design reuse (Goel 1991).

Model-based adaptation is appropriate forCARMA’s advisory task because both empiricalknowledge, in the form of cases, and a grass-land ecology model are available, but neither isindividually sufficient for accurate predictionof forage consumption, given the informationthat ranchers can typically provide.

Case-Based Reasoning in CARMA

The initial impetus for using CBR for forageconsumption prediction was cognitive veri-similitude. The protocol analysis suggestedthat human experts in this domain reasonusing prototypes, which is consistent with var-ious cognitive studies that have demonstratedthat examples or prototypes often play a cen-tral role in human concept structure (Klein andCalderwood 1988; Smith and Medin 1981).

During the development of CARMA, however,CBR’s ability to facilitate knowledge acquisi-tion grew in importance. Few precise records of

Articles

SUMMER 2002 57

Figure 8. CARMA’s Treatment-Matrix Window.

Adaptationcarma uses three techniques for adaptation: (1)temporal projection, (2) feature adaptation, and(3) and critical period adaptation. Two of thesetechniques—(1) temporal projection and (2)critical period adaptation—make use of therangeland ecosystem model. Temporal projec-tion is needed because the feature values of eachprototypical case are represented at a specificpoint in the life history of the grasshopper pop-ulation. To determine the match between thegrasshopper population densities of each proto-typical case and a new subcase, the life history ofthe prototypical case must be projected forwardor backward to align its average developmentalphase with that of the new subcase. This projec-tion requires a model to simulate grasshopperattrition, which depends on developmentalphase, precipitation, and developmental rate(which, in turn, depends on temperature)throughout the interval of the projection.

rangeland grassland infestations are available.However, there are a number of expert pestadvisers with many years of experience withrangeland grasshopper infestations.

To capture human expertise, we sent ques-tionnaires describing hypothetical Wyominginfestation cases to entomologists (includingpest managers) recognized for their work in thearea of grasshopper management and ecology.Each expert received 10 cases randomly drawnfrom a total of 20 representative cases. Thedescriptions of the 20 cases contained at leastas much information as a rancher ordinarilyprovides to an entomologist. The question-naire asked the expert to estimate the probableforage loss. Eight sets of responses werereceived from Wyoming experts, who had amean of 18.0 years experience. CARMA’s caselibrary consists of the 20 hypothetical cases.The consumption prediction associated witheach case is the mean of the experts’ predic-tions for that case.

Articles

58 AI MAGAZINE

Average phase

Density

Hatch Adult

NC

PC'

PC

Figure 9. Projection of a Prototypical Case PC to PC’ to Align Its Developmental Phase with New Case NC.

Figure 9 illustrates how the population inprototypical case PC must be projected back-ward in time to PC’ to match the average devel-opmental phase of new subcase NC. Projectionbackward in time increases grasshopper densi-ty by removing the effect of attrition over theinterval of the projection, whereas projectionforward in time decreases grasshopper densityby adding attrition during this interval. Thevertical bar corresponding to PC and PC’ indi-cates the confidence range for grasshopperdensity, which always increases (indicatinggreater uncertainty) as a function of the inter-val projected.

In feature adaptation, the forage loss predict-ed by the best-matching prototypical case ismodified to account for any feature differences(other than average developmental phase)between it and the subcase. The modification isa linear function of the feature differences. Thecoefficients of the linear function are deter-mined by a form of introspective learning(Hanney and Keane 1997; Leake, Kinley, andWilson 1995), consisting of hill climbingthrough parameter space to optimize leave-one-out predictive accuracy within the caselibrary (Branting, Hasting, and Lockwood1997).

Critical-period adaptation is needed becausegrasshopper consumption is most damaging ifit occurs during the portion of the growing sea-

son during which forage losses cannot fully bereplaced by forage growth, termed the criticalperiod. The forage loss predicted by a prototyp-ical case must be adapted if the proportion ofthe life span of the grasshoppers overlappingthe critical period in the new case differs fromthat in the prototypical case. This adaptationrequires determining, for both the new caseand the prototypical case, the proportion ofthe grasshopper population’s lifetime con-sumption occurring in the critical period. For amore complete description of model-basedadaptation in CARMA, see Branting, Hastings,and Lockwood (1997).

Experimental Evaluation of Model-Based AdaptationThe design of CARMA’s forage-consumptioncomponent was based on the hypothesis thatan integration of model-based and case-basedreasoning can lead to more accurate forage-consumption predictions than the use of eithertechnique individually. This hypothesis wasbased on the observation that neither thecausal model nor the empirical data availablefor rangelands are individually sufficient foraccurate prediction. To test this hypothesis, anablation study was performed in which CARMA’sempirical and model-based knowledge compo-nents were each tested in isolation and theresults compared to the performance of the full

Articles

SUMMER 2002 59

0 5 10 15 20 25 30 35 40

Numerical simulation

Linear regression

ID3

Factored-NN

Global weights

Specific weights

Wyomingmedian sets

Wyomingexpert sets

13.3

14.2

21.1

34.9

25.6

29.6

9.7

10.0

22.8

35.2

11.9

28.8

CARMA:

Empirical only:

Model-based only:

Figure 10. Root-Mean-Squared Error Rate (in Percent) for Leave-One-Out Test Results.

approaches—21.1 percent for factored-NN,34.9 percent for ID3, and 25.6 percent for linearregression—and for the purely model-basedapproach—29.6 percent. CARMA-SPECIFIC andCARMA-GLOBAL were also more accurate than thealternative methods on the Wyoming medianset, although linear regression was only slight-ly less accurate.

The initial confirmation of the hypothesisthat integrating model-based and case-basedreasoning through model-based adaptationleads to more accurate forage-consumptionpredictions than the use of either techniqueindividually is tentative because the relativelylow level of agreement among experts and theabsence of any external standard give rise touncertainty about what constitutes a correctprediction. A detailed description of the empir-ical evaluation of CARMA is set forth in Branting,Hastings, and Lockwood (1997).

Application Use and PayoffIn June 1996, CARMA 2.0 was distributed to theUniversity of Wyoming Cooperative ExtensionOffices and Weed and Pest District Offices ineach of Wyoming’s 23 counties and was madeavailable for download from a University ofWyoming web site. CARMA 2.0 was used byWyoming ranchers and pest managers everysummer from 1996 to 2001. CARMA has beenendorsed and advocated for use by pest man-agers by the United States National Grasshop-per Management Board (NGMB 2001). Perhapsthe greatest interest in the system has beenexpressed by the county-level Weed and PestDistrict supervisors, who—with the withdrawalof USDA support—have become the “front-line” agency in grasshopper pest management.Workshops to train these individuals in theoptimal use of CARMA were developed anddelivered at the request of the agency.

Although CARMA was designed as an advisorysystem for ranchers, CARMA’s ability to robustlyintegrate a variety of knowledge sources led itto be applied in several ways that were notimagined when the program was developed.First, CARMA’s economic analysis has been usedto justify pest management policy decisions. In1998, CARMA’s economic analysis was used togenerate a declaration of grasshopper disasterareas by Wyoming County Commissions, lead-ing to low-interest, federal loans by the FarmService Administration. CARMA’s economicanalysis played a role in the NGMB’s recom-mendation of a new treatment approach,RAATs (Nelson 1999), a strategy now adoptedin six states.

Second, CARMA’s analysis was incorporated

CARMA prediction system under both global andcase-specific adaptation weight modes.

Each predictive method was tested using aseries of leave-one-out tests in which a set ofcases (S) from a single expert was split into onetest case (C) and one training set (S - C). Themethods were trained on the forage-loss pre-dictions of the training set and tested on thetest case. This method was repeated for eachcase within the set (S).

CARMA’s empirical component was evaluatedby performing leave-one-out tests for CARMA’sforage-consumption module with all model-based adaptation disabled. CARMA’s forage-con-sumption module with model-based adapta-tion disabled is termed factored nearest-neighborprediction (factored-NN) because under thisapproach, prediction is based simply on thesum of nearest-neighbor predictions for eachsubcase. Two other empirical methods wereevaluated as well: (1) decision tree inductionusing ID3 (Quinlan 1986) and (2) linear regres-sion using QR factorization (Hager 1988) tofind a least squares fit to the feature values andassociated predictions of the training cases.

The predictive ability of CARMA’s model-based component in isolation was evaluated bydeveloping a numeric simulation based on CAR-MA’s model of rangeland ecology. This simula-tion required explicit representation of twoforms of knowledge implicit in CARMA’s cases:(1) the forage to an acre based on the range-land value of the location and (2) the foragetypically eaten in a day by each grasshopper foreach distinct grasshopper overwintering typeand developmental phase.

The accuracy of each approach was evaluat-ed using leave-one-out testing for the respons-es from each of the eight Wyoming experts andfor a data set consisting of the median of thepredictions of the Wyoming experts on eachcase. The full CARMA prediction system was test-ed using both global-adaptation weights (CAR-MA-GLOBAL) and case-specific adaptationweights (CARMA-SPECIFIC).

The root-mean-squared error rates for eachof the methods are set forth in figure 10. Theserates provide initial confirmation for thehypothesis that integrating model-based andcase-based reasoning through model-basedadaptation leads to more accurate forage-con-sumption predictions than the use of eithertechnique individually. The smallest root-mean-squared error rate was obtained byCARMA-SPECIFIC. On the Wyoming expert sets,the root-mean-squared error rate was 13.3 per-cent for CARMA-SPECIFIC and 14.2 percent forCARMA-GLOBAL. The root-mean-squared errorrate was higher both for the empirical

Articles

60 AI MAGAZINE

into industry strategies. Uniroyal (CK Witco)developed recommendations for the use ofDimilin, a new chemical pesticide, usingCARMA’s analysis. Similarly, RhônePoulenc(Aventis) developed recommendations for theuse of Fipronil based on CARMA’s analysis.

Finally, CARMA-based economic analysis wasincorporated into pest management researchin Lockwood et al. (1999) and Lockwood andSchell (1997).

Development, Deployment, and Maintenance

CARMA was developed as a dissertation project(Hastings 1996). The out-of-pocket develop-ment costs were small, consisting of severalyears of graduate research assistant supportand the license fees for Franz Allegro CommonLisp, the language in which CARMA was devel-oped. However, the path to the developmentof CARMA was quite circuitous, with a variety ofdifferent approaches to grasshopper advisinghaving been developed, tested, and rejected.Thus, the development costs would have beenmuch higher outside an academic environ-ment.

In the years since the distribution of CARMA

2.0, there have been a number of changes inpest-treatment practices. In 2001, CARMA 2.0was updated to CARMA 3.3 to reflect thesechanges and include the treatment matrix dis-cussed earlier for calculating treatment costsfor each acre under various alternative eco-nomic conditions. CARMA’s declarative knowl-edge representation made revising the programstraightforward. These changes were funded bya grant from a producer of a pesticide intro-duced after the distribution of CARMA 2.0 andtherefore not included as a treatment option inthe earlier version. CARMA can be downloadedfrom the USDA’s grasshopper-control web siteor from the University of Wyoming’s Grass-hoppers of Wyoming and the West web site.1,2

ConclusionCARMA demonstrates how AI technology can beused to deliver expert advice to compensate forcutbacks in public services. CBR proved to bean appropriate AI technique for the forage-pre-diction component both because experts inthis domain appear to reason with cases andbecause asking experts to solve example caseswas an effective knowledge-acquisition tech-nique. Model-based adaptation provided amechanism for incorporating rangelandecosystem models into the system without theslow performance, sensitivity to noise, and

diminished explanation capability that wouldhave resulted from a purely simulation-basedapproach.

A key factor in CARMA’s acceptance amongusers is its simple interface and speed, whichmake using CARMA straightforward. The combi-nation of a simple interface, flexible controlstrategy, and integration of multiple knowl-edge sources makes CARMA accessible to inexpe-rienced users and capable of producing advicecomparable to that produced by humanexperts.

Notes1. www.sidney.ars.usda.gov/grasshopper.

2. www.sdvc.uwyo.edu/grasshopper/carma.htm.

AcknowledgmentsThis research was supported by a grant fromthe University of Wyoming College of Agricul-ture, a faculty grant in aid from the Universityof Wyoming Office of Research, National Sci-ence Foundation career grant IRI-9502152, anda grant by Uniroyal (CK Witco). We gratefullyacknowledge the contributions of Scott Schellto the development of CARMA’s task analysisand knowledge base.

ReferencesAamodt, A., and Plaza, E. 1994. Case-Based Reason-ing: Foundational Issues, Methodological Variations,and System Approaches. AI Communications 7(1):39–59.

Aarts, R., and Rousu, J. 1996. Toward CBR for Bio-process Planning. Paper presented at the Third Euro-pean Workshop on Case-Based Reasoning (EWCBR-96), 14–16 November, Lausanne, Switzerland.

Allen, T. F. H., and Hoekstra, T. W. 1992. Toward aUnified Ecology. New York: Columbia University Press.

Branting, K.; Hastings, J.; and Lockwood, J. 1997.Integrating Cases and Models for Prediction in Bio-logical Systems. AI Applications 11(1): 29–48.

Cheetham, W., and Graf, J. 1997. Case-Based Reason-ing in Color Matching. In Proceedings of the SecondInternational Conference on Case-Based Reasoning,1–12. New York: Springer.

Goel, A. 1991. A Model-Based Approach to CaseAdaptation. Paper presented at the Thirteenth Annu-al Conference of the Cognitive Science Society, 7–10August, Chicago, Illinois.

Hager, W. 1988. Applied Numerical Linear Algebra.Englewood Cliffs, N.J.: Prentice Hall.

Hanney, K., and Keane, M. 1997. The AdaptationKnowledge Bottleneck: How to Ease It by Learningfrom Cases. In Proceedings of the Second InternationalConference on Case-Based Reasoning, 359–370. NewYork: Springer.

Hastings, J. 1996. A Mixed-Paradigm ReasoningApproach to Problem Solving in Incomplete Causal

Articles

SUMMER 2002 61

His current research interests includeempirical natural language processing,legal applications of AI, and integration ofcase-based reasoning with other problem-solving paradigms. His e-mail address iskarl.branting@livewirelogic.com.

Jeffrey Lockwood is aprofessor of entomologyin the Department ofRenewable Resources atthe University of Wy-oming. His research isconcentrated on theecology of superabun-dant (pest management)

and rare (conservation biology) grasshop-per populations. He is the executive direc-tor of The Orthopterists Society (a 350-member international organizationdevoted to the study of grasshoppers, crick-ets, and katydids) and the director of theAssociation for Applied Acridology Interna-tional (the first and only humanitarian-based, nongovernmental organization ofentomologists in the world, providingexpert advice, training, and applied res-earch to people and nations in need). Lock-wood teaches courses in insect populationbiology, insect anatomy and physiology,and biodiversity. He has published in theareas of pest management, populationmodeling, biological control, agriculturalethics, and conservation biology; he serveson the editorial board of the Journal ofInsect Conservation. His e-mail address islockwood@uwyo.edu.

International Journal of Pest Management46(1): 29–42.

Moore, J. 1995. Participating in ExplanatoryDialogues. Cambridge, Mass.: MIT Press.

Nelson, D., ed. 1999. Proceedings of theNational Grasshopper Management BoardMeeting, Colorado, 12–13 January, Denver,Colorado.

NGMB. 2001. National Grasshopper Man-agement Board Position Statement. Paperpresented at the National GrasshopperManagement Board Meeting, 16–17 Janu-ary, Denver, Colorado.

Pimm, S. L. 1991. The Balance of Nature:Ecological Issues in the Conservation of Speciesand Communities. Chicago: University ofChicago Press.

Quinlan, J. R. 1986. Induction of DecisionTrees. Machine Learning 1(1): 81–106.

Quinlan, R. 1993. C4.5: Programs forMachine Learning. San Francisco, Calif.:Morgan Kaufmann.

Rousu, J., and Aarts, R. 1996. AdaptationCost as a Criterion for Solution Evaluation.Paper presented at the Third EuropeanWorkshop on Case-Based Reasoning(EWCBR-96), 14–16 November, Lausanne,Switzerland.

Smith, E. E., and Medin, D. L. 1981. Cate-gories and Concepts. Cambridge, Mass.: Har-vard University Press.

Wettschereck, D., and Dietterich, T. 1995.An Experimental Comparison of the Near-est-Neighbor and Nearest-HyperrectangleAlgorithms. Machine Learning 19(1): 5–28.

John Hastings is anassistant professor ofcomputer science at theUniversity of Nebraska atKearney (UNK). He re-ceived his B.S. (1989,with honors), M.S.(1991), and Ph.D. (1996)in computer science

from the University of Wyoming. Afterworking several years as an expert systemsdeveloper and consultant, he joined thefaculty at UNK in 2001. His research inter-ests include case-based reasoning, machinelearning, and applications of AI to naturalresource management. His e-mail address ishastingsjd@unk.edu.

Karl Branting is a princi-ple research scientist atLiveWire Logic, Inc., inMorrisville, North Car-olina. He received a B.A.magna cum laude in phi-losophy from the Uni-versity of Colorado, aJ.D. from Georgetown

University, and a Ph.D. in computer sci-ence at the University of Texas at Austin.

Theory Domains. Ph.D. thesis, Departmentof Computer Science, University ofWyoming.

Hastings, J.; Branting, K.; and Lockwood, J.1996. A Multiparadigm Reasoning Systemfor Rangeland Management. Computers andElectronics in Agriculture 16(1): 4–67.

Hewitt, G., and Onsager, J. 1983. Control ofGrasshoppers on Rangeland in the UnitedStates: A Perspective. Journal of Range Man-agement 36(2): 202–207.

Joern, A., and Gaines, S. B. 1990. Popula-tion Dynamics and Regulation inGrasshoppers. In Biology of Grasshoppers,eds. R. F. Chapman and A. Joern, 415–482.New York: Wiley.

Klein, G. A., and Calderwood, R. 1988. HowDo People Use Analogues to Make Deci-sions? In Proceedings of the DARPA Workshopon Case-Based Reasoning, 209–218. SanFrancisco, Calif.: Morgan Kaufmann.

Leake, D.; Kinley, A.; and Wilson, D. 1995.Learning to Improve Case Adaptation byIntrospective Reasoning and CBR. In Case-Based Reasoning Research and Development:First International Conference on Case-BasedReasoning (ICCBR-95), 229–240. LectureNotes in Artificial Intelligence 1010. NewYork: Springer.

Lockwood, J. A. 1993a. Benefits and Costsof Controlling Rangeland Grasshoppers(Orthoptera: Acrididae) with Exotic Organ-isms: Search for a Null Hypothesis and Reg-ulatory Compromise. Environmental Ento-mology 22(5): 904–914.

Lockwood, J. A. 1993b. EnvironmentalIssues Involved in Biological Control ofRangeland Grasshoppers (Orthoptera: Acri-didae) with Exotic Agents. EnvironmentalEntomology 22(3): 503–518.

Lockwood, J., and Kemp, W. 1987. Proba-bilities of Rangeland Grasshopper Out-breaks in Wyoming Counties. WyomingAgricultural Experiment Station BulletinB(896).

Lockwood, J., and Lockwood, D. 1991.Rangeland Grasshopper (Orthoptera: Acridi-dae) Population Dynamics: Insights fromCatastrophe Theory. Environmental Ento-mology 20(4): 970–980.

Lockwood, J., and Schell, S. 1997. Decreas-ing Economic and Environmental Coststhrough Reduced Area and Agent Insecti-cide Treatments (RAATs) for the Control ofRangeland Grasshoppers: Empirical Resultsand Their Implications for Pest Manage-ment. Journal of Orthoptera Research 6(1):19–32.

Lockwood, J.; Schell, S.; Foster, R.; Reuter,C.; and Rachadi, T. 1999. Reduced Agent-Area Treatments (RAATs) for Managementof Rangeland Grasshoppers: Efficacy andEconomics under Operational Conditions.

Articles

62 AI MAGAZINE