ijade web-miner: an intelligent agent framework …gkmc.utah.edu/7910f/papers/ieee tkde ijade web...
TRANSCRIPT
iJADE Web-Miner: An Intelligent AgentFramework for Internet ShoppingRaymond S.T. Lee, Member, IEEE, and James N.K. Liu, Member, IEEE
Abstract—There is growing interest in using intelligent software agents for a variety of tasks, including navigating and retrieving
information from the Internet and from databases, online shopping activities, user authentication, negotiation for resources, and
decision making. This paper proposes an integrated framework for information retrieval and information filtering in the context of
Internet shopping. The work focuses on applying agent technology, together with Web mining technology, to automate a series of
product search and selection activities. It is based on a multiagent development platform, namely, iJADE (Intelligent Java Agent
Development Environment), which supports various e-commerce applications. The framework comprises an automatic facial
authentication utility and six other modules, namely, customer requirements definition, a requirement-fuzzification scheme, a fuzzy
agents-negotiation scheme, a fuzzy product-selection scheme, a product-defuzzification scheme, and a product-evaluation scheme. A
series of experiments were carried out and favorable results were produced in executing the framework. From an experimental point of
view, we used a database of 1,020 facial images that were obtained under various conditions of facial expression, viewing perspective
and size. An overall correct recognition rate of over 85 percent was attained. For the product selection test of our fuzzy shopper
system, an average matching rate of more than 81 percent was achieved.
Index Terms—iJADE Web miner, Web mining, visual data mining, e-commerce, intelligent agents.
�
1 INTRODUCTION
DUE to the rapid developments in e-commerce, rangingfrom C2C (Consumer-to-Consumer) e-commerce appli-
cations such as an e-auction to sophisticated B2B (Business-to-Business) e-commerce activities such as e-Supply ChainManagement (eSCM), the Internet is becoming a commonvirtual marketplace in which we can do business, search forinformation, and communicate with each other. Because ofthe ever-increasing amount of information in cyberspace,knowledge discovery and Web mining are becoming criticalfor successfully conducting business in the cyber world.
With the advance of PC computing technology in termsof its computational speed and its popularity for Webbrowsing, intelligent software applications known asagents—with their autonomous properties, automatic dele-gation of jobs, and highly mobile and adaptive behavior inthe Internet environment—are becoming a potential area ofdevelopment for e-business in the new millennium [18].
In a typical e-shopping scenario, there are two funda-mental areas of functionality where Web mining and visualdata mining might be able to help. The first area is customerauthentication. Traditional authentication is based on ausername and password transmitted using a secure trans-port protocol such as the Secure Socket Layer (SSL).Although this provides a secure user authenticationscheme, it requires proactive login from a customer beforeaccess rights are granted, which may discourage somecustomers from shopping. Other authentication schemes
that are based on digital certificates with smart cardtechnology [34], or biometric authentication techniquesbased on iris or palmprint recognition, might provide somefeasible alternatives. However, those approaches needspecial authentication equipment that limits their use inan e-commerce environment, not to mention the legalimplications of accessing personal and private data, such asiris and palmprint patterns. In contrast, automatic authen-tication based on human-face recognition can help toovercome these limitations. In terms of the visual proces-sing equipment required, a standard Web camera is alreadysufficient for facial pattern extraction and, nowadays, this iscommon equipment for Web browsing. This kind ofauthentication scheme can provide a truly automaticscheme in which a customer does not need to provide anyspecial identification information and, more importantly, itdoes not exploit any “confidential” or “sensitive” data, suchas fingerprints or iris patterns.
The second area of functionality is the automation of theonline shopping process via agent technology. Traditionalshopping models include consumer buying behavior mod-els, such as the Blackwell model [11], the Bettman model [6],and the Howard-Sheth model [16], which all share a similarlist of six fundamental stages in consumer buying behavior.The six stages are:
1. consumer requirements definition,2. product brokering,3. merchant brokering,4. negotiation,5. purchase and delivery, and6. after-sales service and evaluation.
The first three stages involve a wide range of uncertaintyand many possibilities, what we call “fuzziness,” rangingfrom the setting of purchasing criteria (and the supplying of
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004 461
. The authors are with the Department of Computing, Hong KongPolytechnic University, Hung Hom, Kowloon, Hong Kong, China.E-mail: {csstlee, csnkliu}@comp.polyu.edu.hk.
Manuscript received 13 Dec. 2000; revised 15 Aug. 2002; accepted 7 Feb.2003.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number 113292.
1041-4347/04/$20.00 � 2004 IEEE Published by the IEEE Computer Society
products by a merchant) to selecting goods. So far, these areall “gray areas” that need to be thoroughly explored toapply agent technology to an e-commerce environment.
In this paper, we propose an integrated, intelligentagent-based framework, known as iJADE (pronounced as“IJ”)—the intelligent Java Agent-based Development En-vironment. The aim of this framework is to overcome thedeficiency of contemporary agent software platforms, suchas IBM Aglets [2] and ObjectSpace Voyager Agents [38],which mainly focus on multiagent mobility and commu-nication. iJADE provides an important layer, called the“Intelligent Layer,” that supports the implementation ofdifferent AI functionalities for multiagent applications.From an implementation point of view, we describe oneof the most important applications of iJADE in thee-commerce environment: iJADE Web miner. The iJADEWeb miner is an intelligent agent-based Web miningapplication that consists of two major modules: 1) Anagent-based facial-pattern authentication scheme, calledFAgent, which uses an innovative, visual data mining andvisualization scheme that is based on the EGDLM (ElasticGraph Dynamic Link Model) technique [22]. 2) A Webmining tool that is based on fuzzy shopping agents forproduct selection and brokering, called FShopper.
The rest of this paper is organized as follows: Section 2presents an overview of Web mining, which is an extensionof data mining to cyberspace. Section 3 gives a generaldescription of agent systems for e-commerce applications.Section 4 presents the model framework of iJADE and thetwo major components of iJADE Web miner: FAgent andFShopper. The implementation of the system is discussed inSection 5, which is followed by a brief conclusion.
2 WEB MINING—A PERSPECTIVE
As an important extension of data mining [12], Web miningis a technology that integrates various research fields,including computational linguistics, statistics, informatics,artificial intelligence (AI), and knowledge discovery [39]. Itcan also be interpreted as the discovery and analysis ofuseful information from the Web. Although there is nodefinite principle in a Web mining model, Web mining canbe categorized into two main areas: content mining andstructural mining [9], [39]. A taxonomy of Web mining isdepicted in Fig. 1.
Content mining focuses on the extraction of Web contentand knowledge discovery (mining) from that information,ranging from HTML and XML documents found in Webservers to data sources (e.g., databases) attached to theback-end of Web systems. Structural mining focuses onknowledge discovery from the structure of the Websystems. This includes the mining of user preferences whilethey browse the Web (Web usage mining), examining theusage of different URLs in a particular Web site (URLmining), mining external structure (hyperlinks betweendifferent Web pages), and mining internal structure(hyperlinks within a particular Web page). Active researchincludes Spertus [36] on mining internal structure andPitkow [33] on mining Web usage data.
In content mining, popular search engines (such asLycos, WebCrawler, Infoseek, and Alta Vista) provide some
Web searching functionality. However, they fail to provideconcrete and structural information [27]. In recent years,interest has been focused on how to provide a higher levelof organization for semistructured (and even unstructured)data on the Web using Web mining techniques. Within thistopic, there are two main areas of research: the databaseapproach and the agent-based approach. The databaseapproach to Web mining tries to focus on techniques fororganizing the semistructured and unstructured data (onthe Web) into more structured information and associatedresources, so that traditional query tools and data miningcan be applied for data analysis and knowledge discovery.Typical examples can be found in the ARANEUS system[3], which uses a multilevel database for Web mining, andChe et al. [8] in which a complex Web query system is usedfor Web mining of G-Protein Coupled Receptors (GPCRs).The agent-based approach focuses on the provision of“intelligent” and “autonomous” Web mining tools that arebased on agent technology. Typical examples can be foundin the FAQFinder system [14] for intelligent search enginesand the Firefly system [35] for personalized Web agents.
3 AGENT TECHNOLOGY IN E-COMMERCE
3.1 Background
The current World Wide Web system is catalyzing thedevelopment of e-commerce using the Internet. The currentInternet commerce systems are primarily based on theclient-server architecture. All transactions are carried out bymany request/response interactions over the Internet. Asthe Internet is a best-effort network, sometimes a user mayexperience a long response time. Another approach is to usea mobile agent-based system. This involves sending amobile software agent to a remote system using varioustechnologies (such as IBM Aglets [2], ObjectSpace VoyagerAgents [38], the FTP Software Agent [13], the General MagicOdyssey Agent System [32], and the Agent BuilderEnvironment from IBM [1]). The agent can conduct multipleinteractions with the software resident on a remote system.The output of the interactions is then sent back to the user.An agent can also interact with other agents on the Internet
462 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
Fig. 1. A taxonomy of Web mining.
before returning to the originating system. It is expectedthat this type of agent-based system will complement theexisting client-server Internet commerce systems by provid-ing more advanced services, including agent cloningservices, multiagent collaboration, and messaging services.Currently, there are many different e-commerce systems inuse around the world, ranging from simple online shops tomore complex systems that provide different types ofservices. Some examples include the following:
. BargainFinder—a database search engine for search-ing online music stores [5],
. AuctionBot—a generic auction server that allowssuppliers to auction products [4],
. Metabroker—a generic framework for creating elec-tronic brokers [7], and
. MAGNET—a system for networked electronictrading [10].
3.2 Advantages of Using Mobile AgentTechnologies for E-Commerce Applications
3.2.1 Reduce Network Load
Distributed systems often rely on communications protocolsthat involve multiple interactions to accomplish a given task.Mobile agents allow one to package and dispatch tasks to adestination host where the interactions can take place locally.
3.2.2 Overcome Network Latency
E-commerce applications often involve real-time interac-tions, such as online shopping and e-auctions. They all needto respond in real time to changes in their environment.Controlling such systems through a factory network of asubstantial size involves significant latencies, which is notacceptable for these real-time operations. Mobile agentsoffer a solution because they can be dispatched from acentral controller to act locally and to directly execute acontroller’s directions.
3.2.3 Execute Asynchronously and Autonomously
Very often, mobile devices must rely on expensive or fragilenetwork connections. Tasks that require a continuouslyopen connection between a mobile device and a fixednetwork will probably not be economically or technicallyfeasible. Using mobile agent technologies, tasks can beembedded into mobile agents, which can then be dis-patched into a network and operated asynchronously andautonomously.
3.2.4 Adapt Dynamically
One major characteristic of using agent technologies is theability to “sense” the execution environment (e.g., themarketplace in shopping operations) and react autono-mously to changes. Multiple mobile agents possess a uniqueability to distribute themselves among the hosts in a networkso that they can maintain an optimal configuration for anye-commerce operations, such as “e-window-shopping.”
3.2.5 Naturally Heterogeneous
Network computing is fundamentally heterogeneous,often from the perspective of both the hardware andsoftware. Because mobile agents are generally computer
and transport-layer independent, they depend only ontheir execution environment. Thus, they possess optimalcapabilities for seamless system integration, especiallyunder circumstances where other efforts have failed.
3.2.6 Robust and Fault-Tolerant
Mobile agents react dynamically to unfavorable situationsand events, making it easier to build robust and fault-tolerant distributed systems. This is a vital criterion for anye-commerce application being implemented in the Internetenvironment.
3.3 Shortcomings of Current Agent-Based Systems
Although contemporary agent-based systems provide aneffective framework for the dispatching, communication,and management of multiple mobile agents in the Internetenvironment, from the point of view of the AI, there is alack of support for intelligent functionality in these systems.For instance, IBM Aglets [2] provide comprehensive APIsfor mobile agent, which support creating, dispatching,cloning, retracting, and disposing of aglets using theAgletContext class. They also support aglet messaging andcollaboration using the Aglet Message classes and Mobilityadapters. However, intelligent capabilities—such as frame-based learning and data mining at a macroscopic level, orneural-network modeling, fuzzy, and genetic learning at adetailed level—have not been adopted. This makes thedevelopment of AI-based agent applications more difficult.
4 IJADE ARCHITECTURE
4.1 iJADE Framework
The iJADE system framework is shown in Fig. 2. Unlikecontemporary agent systems and APIs such as IBM Aglets[2] and ObjectSpace Voyager [38], which focus on themultiagent communication and autonomous operations, theaim of iJADE is to provide comprehensive APIs for“intelligent” agents and applications for future e-commerceand Web mining applications.
Fig. 2 depicts the two levels of abstraction that are usedin the iJADE system: a) the iJADE system level—AITSmo-del, and b) the iJADE data level—DNA model. The AITSmodel consists of
1. the Application Layer,2. the Intelligent Layer,3. the Technology Layer, and the4. Supporting Layer.
The DNA model is composed of 1) the Data Layer, 2) theNeural-Network Layer, and 3) the Application Layer.
Compared with contemporary agent systems, whichprovide minimal and elementary data managementschemes, the iJADE DNA model provides a comprehensivedata manipulation framework that is based on neural-network technology. The “Data Layer” corresponds to theraw data and input “stimuli” (such as the facial imagescaptured from a Web camera and the product informationin a cyber store) from the environment. The “Neural-Network Layer” provides the clustering of different types ofneural networks for the purpose of organization, inter-pretation, analysis, and forecasting operations that are
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 463
based on the inputs from the “Data Layer.” The neural
networks are used by the iJADE applications in the
“Application Layer.”
4.2 Application Layer including the iJADE WebMiner
TheApplication Layer is the uppermost layer,which consists
of different intelligent agent-based applications. These
iJADE applications are developed through the integration
of intelligent-agent components from the “Intelligent Layer”
and the data “knowledge fields” from the DNA model.Current applications (iJADE v1.6) implemented in this
layer include the following:
. iJADE Stock Advisor, which is an intelligent agent-based stock prediction system using the time seriesneuro-oscillatory prediction technique [20],
. iJADE Web miner, the intelligent Web mining agentsystem that is discussed in this paper, and
. iJADE WeatherMAN, which is an intelligent weath-er forecasting agent that is the extension of previousresearch on multistation weather forecasting usingfuzzy neural networks [28].
4.3 Intelligent Layer
This layer provides the AI capabilities of the iJADE system
using the agent components provided by the “Technology
Layer.” The “Intelligent Layer” consists of the following
three main functional areas:
1. “Sensory Area”—for the recognition and interpreta-tion of incoming stimuli. It includes a) the visualsensory agents using the EGDLM (Elastic GraphDynamic Link Model) for invariant visual objectrecognition [24], [25], [26], and b) the auditory agents
based on a wavelet-based feature extraction andinterpretation technique [15].
2. “Logical Reasoning Area”—an area providing dif-ferent AI tools for logical “thinking” and rule-basedreasoning, such as fuzzy and GA (Genetic Algo-rithms) rule-based systems [23].
3. “Analytical Area”—consists of various AI tools foranalytical calculations, such as recurrent neural-network-based analysis for real-time prediction anddata mining [21].
4.4 Technology Layer Using IBM Aglets and JavaServlets
This layer provides all of the APIs that are necessary for
mobile agent implementation and for the development of
intelligent agent components in the “Intelligent Layer.”In the current version (v1.6) of the iJADE model, IBM
Aglets [2] are used as the agent “backbone.” The main
function of the AgletProxy class is to provide a handle that is
used to access the aglet. It also provides location transpar-
ency by forwarding requests to remote hosts and returning
results to the local host. Actually, all communication with
an aglet occurs through its aglet proxy. The AgletContext
class provides the runtime execution environment for aglets
within an agent server. Thus, when an aglet is dispatched to
a remote site, it is detached from the current AgletContext
object, serialized into a message bytestream, sent across the
network, and reconstructed in a new AgletContext, which in
turn provides the execution environment at the remote site.
The other critical component of the Aglet environment is
security. Aglets provide a security model in the form of an
AgletSecurityManager, which is a subclass of the “standard”
Java SecurityManager.
464 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
Fig. 2. System architecture of the iJADE (v 1.6) model (http://www.iJADK.com).
4.5 Supporting Layer
This layer provides all of the necessary system support forthe “Technology Layer.” This includes:
1. programming language support based on Java,2. support for network protocols such as HTTP
(Hypertext Transfer Protocol), HTTPS (secureHTTP), ATP (Asynchronous Transfer Protocol),etc., and
3. support for markup languages such as HTML(Hypertext Markup Language), XML (ExtensibleMarkup Language), and WML (Wireless MarkupLanguage).
5 IMPLEMENTATION
The iJADE Web miner consists of two main modules: amodule for visual data mining for intelligent user authenti-cation based on the FAgent (Face-recognition Agent) and amodule for Web product mining using a neuro-fuzzyshopping agent based on the FShopper (Fuzzy Shopper).
5.1 Intelligent Visual Data Mining for UserAuthentication Using the FAgent
5.1.1 FAgent: System Overview
In this section, we present FAgent (Face-recognition Agent),which is an automatic authentication scheme using a mobileagent-based, human-face-recognition system. Technically,the proposed system involves the integration of twocontemporary technologies: First, mobile agent technologyto facilitate mobile, heterogeneous, and fault-tolerantfunctionality in an e-commerce environment. Second, theElastic Graph Dynamic Link Model to perform neural-network-based face recognition using an elastic graphmatching technique, which has proven results in variousproblem domains ranging from human-face recognition[24] and scene analysis [26] to sophisticated tropical cyclonerecognition [25].
5.1.2 Invariant Face Recognition Using the Elastic
Graph Dynamic Link Model
Object recognition using the EGDLM is based on theframework of the Dynamic Link Architecture (DLA) [31],which describes the visual recognition problem as an elasticgraph matching mechanism between the attribute graphs ofthe image vectors (input layer) and a set of “memory”graphs from an object gallery (memory layer). The neuralinteractions are governed by the onset and offset of thedynamic links between the memory and input layers, whichsimulate the functionality of dynamic memory associationin short-term memory. Active research in this area includeshandwritten character recognition [29], [30] and roboticgesture recognition [37].
Unlike the traditional DLA model, the EGDLM inte-grates with the Active Snake Model [17] for the extraction ofthe facial contours. Sophisticated facial matching becomes asimplified elastic graph-matching problem. To enhance therecognition rate in terms of speed and accuracy, featurevectors are extracted from the “facial landmarks” of thefacial image.
One of the most striking features of the EGDLM is its“invariant” property. In the network model, only thetopological relations between the feature vectors areencoded into the network. The pattern matching processresembles “elastic graph matching” [31], which is invariantunder various transformations, such as translation, rotation,reflection, dilation, and occlusion. These transformationscommonly occur in pattern recognition problems such ashuman face recognition and scene analysis.
5.1.3 FAgent System—System Framework
The system architecture of the FAgent mainly consists oftwo subsystems, one at the client (i.e., customer) site andthe other at the server (e.g., a virtual shopping mall) site.A schematic diagram of the whole FAgent system isshown in Fig. 3.
In summary, the three kinds of intelligent agents operat-ing within the FAgent system are as follows:
1. FAgent Feature Extractor—a stationary agent at aclient machine that is used to extract facial featuresfrom a facial image of the client captured using theclient’s digital camera.
2. FAgent Messenger—a mobile agent that acts as a“messenger” and carries the facial image to theserver-side agent and reports back the latest status tothe client machine.
3. FAgent Recognizer—a stationary agent located atthe server. Its main duty is to perform invariant-facial pattern matching against the server-side facialdatabase.
The client-side subsystem consists of the following threeoperations:
1. Facial image capture using the client’s desktop videocamera.
2. Facial contour extraction using the Active ContourModel (ACM) [17].
3. Automatic facial landmark extraction using a Gaborfeature extractor [24].
5.1.4 Facial Contour Extraction—the Active Contour
Model (ACM)
The Active Contour Model involves the use of a “snake”[17] to locate the facial contours. The “snake” is acontinuous curve that forms an initial state (facial template)and then tries to deform itself dynamically on the image.External forces attract the snake toward image features, andinternal forces maintain the smoothness of the snake’sshape (Fig. 4). The sum of the membrane energy, denotingthe snake stretching, and the thin-plate energy, denoting thesnake bending, gives the following snake energy [17]:
EintðuðsÞÞ ¼ �ðsÞ usj ðsÞj2 þ �ðsÞ ussj ðsÞj2; ð1Þ
where uðsÞ ¼ ðxðsÞ; yðsÞÞ is the snake curve and s is the arc-length of the curve and ussðsÞ denotes the second orderderivatives of the snake curve [17]. The parameters ofelasticity � and � control the smoothness of the snake’scurve. This energy function is composed of the first-orderterm controlled by �ðsÞ and the second-order termcontrolled by �ðsÞ. The first-order term makes the snakeact like a membrane and the second-order term makes it act
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 465
like a thin plate. Adjusting the weights of these two
components controls the relative importance of the mem-
brane and thin-plate terms.The deformation of the “snake” is governed by the
external forces. These forces are associated with a potential
P ðx; yÞ, which is generally defined in terms of the gradient
module of the image convoluted by a Gaussian function:
P ðx; yÞ ¼ � rðGðx; yÞ � Iðx; yÞÞjj ð2Þ
or as a distance map of the edge points:
P ðs; yÞ ¼ dðx; yÞ; P ðx; yÞ ¼ �e�dðx;yÞ2 ; ð3Þ
where dðx; yÞ denotes the distance between a pixel ðx; yÞ andits closest edge point. The snake is moved by the potential
forces and tries to fall into a valley as if it were under the
effect of gravity.The total snake energy is given by:
Esnake ¼Z10
Eint þEext ds
¼Z10
�ðsÞ usj ðsÞj2 þ �ðsÞ ussj ðsÞj2 þ P ðuðsÞÞds:
ð4Þ
The minimum of the snake energy satisfies an Euler-
Lagrange equation:
� d
dsð�usðsÞÞ þ d2
ds2ð�ussðsÞÞ þ rP ðuðsÞÞ ¼ 0 ð5Þ
and boundary conditions.
5.1.5 Automatic Facial Landmarks Extraction Scheme
A total of 120 feature vectors of different attributes were
extracted automatically from 50 facial landmark positions
(e.g., noses, eyes, eyebrows, mouth, facial contours, etc.)
(Fig. 5). Gabor filters of 15 different frequency bands ð�Þand eight different orientations ð�Þ were used. The filter
function is given as follows [31]:
g�;�ðx; yÞ ¼1
�ffiffiffi�
p e �x2þy2
2�2
� �e2�i �ðx cos �þy sin �Þ: ð6Þ
466 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
Fig. 3. Schematic diagram of the FAgent system.
Fig. 4. Facial contour extraction using the ACM.
5.1.6 Server-Side Subsystem
The server-side subsystem, basically, consists of the
following modules:
1. The Dynamic Links Initialization scheme.2. The elastic attribute graph matching scheme (be-
tween the query image and the images from thefacial database).
In the Dynamic Link Initialization process, dynamic links
ðzij;klÞ are initialized between “memory” facial attribute
graphs and figure objects from the image gallery according
to the following rules:
zij;kl ¼ "JijJkl for Jij 2 A; Jkl 2 B; ð7Þ
where the Jij and Jkl are the feature vectors extracted from
the facial landmarks, zij;kl are the dynamic link values, " is a
parameter value between 0 and 1, and A and B denote the
figure graph and memory graph, respectively.In the elastic attribute graph matching module, the
attribute graph of the figure is “dynamically” matched with
each “memory” object attribute graph by minimizing the
following energy function:
HðzÞ ¼ �X
i;j2B;k;l2Azijzjl zik zkl þ �
Xi2B
Xk2A
zik � 1
!2
þ �Xk2A
Xi2B
zik � 1
!2
ð8Þ
within tolerance level �.HðzÞ is minimized using gradient descent
zijðtþ 1Þ ¼ zijðtÞ � @HðzðtÞÞ@zijðtÞ
� �w; ð9Þ
where ½. . .�w denotes the value of zij confined to the interval
½0; w�. At equilibrium (within a chosen tolerance level �),
HðzÞ will be minimized and the connection pattern in the
memory layer represents the pattern recalled by the figure
pattern.
5.2 Neural-Fuzzy Agent-Based Shopping Using theFShopper
The system framework of the FShopper consists of the
following six main modules:
1. Customer requirements definition (CRD).
2. Requirement-fuzzification scheme (RFS).3. Fuzzy agents-negotiation scheme (FANS).4. Fuzzy product-selection scheme (FPSS).5. Product-defuzzification scheme (PDS).6. Product-evaluation scheme (PES).7. A schematic diagram of the fuzzy shopper, FShop-
per, is shown in Fig. 6.
5.2.1 Customer Requirements Definition (CRD)
As shown in Fig. 6, at the client machine, there are twotypes of iJADE FShopper agents:
. FShopping Broker—a stationary agent that acts as abroker for a customer. This is an autonomoussoftware agent that contains all of the necessaryinformation (e.g., e-form) and analytical techniquesto act as a broker, such as the requirement forfuzzification and defuzzication and product evalua-tion techniques.
. Fuzzy Buyer—a mobile agent that acts as a virtualbuyer in the virtual marketplace. It performs allagent communication, interaction, and negotiationoperations.
In the CRD module, the customer (via a browser) isprovided (by the FShopping Broker) with an electronic form(e-form) to specify his or her requirements. The FShoppingBroker also provides a set of “cues” to assist in productdefinition.
5.2.2 Requirement-Fuzzification Scheme (RFS)
Once a customer has input his or her product requirements(e.g., color, size, style, fit), the FShopping Broker convertsthese (fuzzy) requirements into fuzzy variables by using“embedded” knowledge (i.e., the membership functions)that is stored in its knowledge base. The FShopping Broker isalso responsible for validating the data in the e-form. Samplefuzzymembership functions for selected attributes for shoes,including color and degree of fit, are shown in Fig. 7.
5.2.3 Fuzzy Agents-Negotiation Scheme (FANS)
After the (fuzzy) requirements of the customer are collected,a Fuzzy Buyer starts its buying activity in the cyber world.To speed up the buying process, a Fuzzy Buyer makes useof the agent cloning capability to create duplicates so thatparallel buying activities can be executed.
In each relevant Virtual Shopping Mall (VSM), a FuzzyBuyer communicates and negotiates with a Fuzzy Seller,which is a stationary (selling) agent that acts as a virtualsalesman in relation to the shopping activities.
5.2.4 Fuzzy Product-Selection Scheme (FPSS)
Once a Fuzzy Seller has collected all of the customer (fuzzy)requirements, it performs the product selection activitiesbased on a fuzzy neural network. The fuzzy neural networkis an integration of fuzzy technology and a Feed-ForwardBack-Propagation (FFBP) neural network, as shown in Fig. 8.
Fig. 8 illustrates the FPSS using a fuzzy neural networkfor product selection (e.g., selection of a pair of shoes). Thefuzzy neural network consists of two parts: the fuzzymodule and the Feed-Forward Back-Propagation (FFBP)neural network module. The fuzzy module provides the
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 467
Fig. 5. Facial features extraction scheme.
network with a bundle of fuzzy variables as the inputnodes. In the example, the fuzzy variables consist of colorcomponents (i.e., red, yellow, and blue), size, length, degreeof fit, and price.
The FFBP neural network is responsible for the productselection, which is a kind of pattern classification schemeusing amultilayer neural network. The output layer is the listof items that are available in the store (under the appropriateproduct category).
To proceed with product classification, the fuzzy FFBP
needs to be trained beforehand. Implementation details on
network training are discussedwith the experimental results.
5.2.5 Product-Defuzzification Scheme (PDS) and
Product-Evaluation Schemes (PES)
Depending on a customer’s preferences, a Fuzzy Buyer willreturn information about a number of recommendedproducts to a client machine. Before these products aredisplayed in a client browser, two operations have to beperformed: the Product-Defuzzification Scheme (PDS) andthe Product-Evaluation Scheme (PES).
In the PDS, each recommended product undergoes adefuzzification process to return the fuzzy descriptions forall of the related product attributes (for ease of under-standing for a user). In addition, according to a user’s
468 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
Fig. 7. Sample membership functions for color and degree of fit.
Fig. 6. Schematic diagram of the FShopper.
predefined ranking of the “importance” of every product
attribute, an objective evaluation scheme can be used to
rank all of the recommended products, which can then be
displayed in a customer’s Web browser so that they can
make the final purchasing decision.
6 EXPERIMENTAL RESULTS
6.1 FAgent Test
In our experiment, images of 100 human subjects were
used to train the system. We tested the system using a set
of 1,020 patterns resulting from different facial expres-
sions, viewing perspectives, and various sizes of the stored
templates. A series of tested facial patterns were obtained
using a CCD camera, which provided a standard video
signal and digitized images at 512� 384 pixels with 8 bits
of resolution.
6.1.1 FAgent Test I: Training Image Retrieval Test
By using the 100 facial images for the ACM contour
extraction, followed by the EGDLM network coding
mechanism, the model was tested against these training
images to determine its image retrieval capability. With the
number of iterations for the ACM modeling ranging from 0to 1,000, the retrieval accuracy is shown in Fig. 9.
As shown in Fig. 9, the pattern retrieval accuracy issaturated when the number of iterations used in the ActiveContour Model goes beyond 300. The overall retrievalaccuracy is promising, with an average correct recognitionrate of over 96 percent for the 100 training patterns.
6.1.2 FAgent Test II: Facial Pattern Illumination Test
In the illumination test, we used 100 test patterns of variousdegree of brightness for facial recognition, with the degreeof brightness varying from +30 percent to -30 percent of the“optimum” brightness level. The experimental results areshown in Table 1.
The overall correct recognition rates range from 79 to95 percent under the varying conditions.
6.1.3 FAgent Test III: Viewing Perspective Test
In this test,weused aviewingperspective ranging from -30 toþ30 degrees (with reference to the horizontal and verticalaxis), with 100 test patterns for each viewing perspective. Therecognition results are presented in Table 2.
According to the “rotation invariant” property of theelastic graph implemented in the EGDLM model [24], [26],
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 469
Fig. 8. Fuzzy neural network for product selection.
Fig. 9. Stored pattern retrieval test.
TABLE 1Results for the Illumination Test
the FAgent possesses the same characteristic in the “contourmaps elastic graph matching” process. The overall correctrecognition rates range between 82 percent and 92 percent.
6.1.4 FAgent IV: Facial Pattern Dilation / Contraction
Test
In this test, we used 300 test patterns, with size ratio rangingfrom -30 percent (pattern contraction) to +30 percent(pattern dilation). We also tested “partial” dilation andcontraction. The recognition results are shown in Table 3.
Because of the “elastic graph matching” characteristic ofthe Elastic Graph Dynamic Link Model, the system alsopossesses the “dilation and contraction invariance.” Thisproperty is similar to the “dilation invariance” of the Chinesecharacters investigated in [29] and [30]. The overall correctrecognition rates range between 79 percent and 92 percent.
6.1.5 FAgent Test V: The Facial Pattern Occlusion and
Distortion Test
In this test, we divided the 120 test patterns into threecategories (Fig. 10):
. Wearing spectacles or other accessories,
. Partial occlusion of the face by obstacles such as cupsand books (in the reading and drinking processes),and
. Various facial expressions (such as laughing, angry,and gimmicky faces).
The pattern recognition results are shown in Table 4.Among the three different categories of facial occlusion,
“wearing spectacles” had the least negative affect on facialrecognition. This is due to the fact that all the main facialcontours are preserved in this situation. In the second typeof occlusion, the influence on the recognition rate dependson the proportion and portion of the face that is obscured.Nevertheless, the lowest correct recognition rate (of thethree categories) is 72 percent.
For the different categories of occlusion, facial expres-sions and gimmicky faces create the most difficult recogni-tion tasks. However, in our test, both these recognition tasksproduce promising results. Due to the “elastic graph”characteristic of our model, the recognition engine “inher-its” the “distortion invariance” property. The overall correctrecognition rates are between 72 percent and 87 percentunder the three different types of occlusion.
470 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
TABLE 2Results for the Viewing Perspective Test
TABLE 3Results of the Facial Pattern Dilation and Contration Test
Fig. 10. Occluded facial patterns.
6.2 The FShopper Test
In the product database, we used more than 200 items undereight categories to construct an electronic catalog (e-catalog).These categories were: T-shirts, shirts, shoes, trousers, skirts,sweaters, tablecloths, and napkins. We deliberately chosesoft-good items instead of hard-goods, such as books or CDs(as commonly found in most e-shopping agent systems), toallow more room for the definition of fuzzy user require-ments and product selection. For neural network training, allof the e-catalog items were “pretrained” in the sense that theattribute descriptions were predefined for all of the itemsthat were “fed” into a fuzzy neural network (for eachcategory of product) for product training. Thus, in total, eightdifferent neural networks were constructed, one for eachcategory of products.
From an experimental point of view, two sets of testswere conducted: the Round Trip Time (RTT) test and theProduct Selection (PS) test. The RTT test was aimed at theevaluation of the “efficiency” of the FShopper in the sensethat it involved calculating the complete round-trip time ofthe agents instead of calculating the difference between thearrival and departure times at the server. The RTT testcalculated the entire time based on the time associated witheach “component” of the system. This started at thecollection of user requirements, through the fuzzificationprocess to the product selection and evaluation steps, sothat a total picture of the efficiency of the system’sperformance could be determined. In the PS test, sincethere was no definite answer as to whether a product would“fit” the taste of a customer, a sample group of 40 candidates
were used to judge the “effectiveness” of the FShopper. Thedetails of these tests are illustrated in the following sections.
6.2.1 FShopper Test I: The Round Trip Time (RTT) Test
In this test, two iJADE servers were used: T1server andT2server. T1server was situated within the same LAN as theclient machine, whereas the T2server was in a remote site(but still within the campus). The results of the mean RTTafter 100 trials for each server are shown in Table 5.
As shown in Table 5, the total RTT is dominated by theFuzzy Product Selection Scheme, but the time spent is stillwithin an acceptable timeframe of five to seven seconds.The difference in RTT between the server situated withinthe same LAN as the client and the server at the remote siteis not significant except for the Fuzzy Agents-negotiationScheme (FANS). The Fuzzy Buyer needs to take a slightlylonger “trip” than for a server at the remote site. Of course,in reality, this depends mainly on the network traffic.
6.2.2 FShopper Test II: The Product Selection (PS) Test
Unlike the RTT test, in which objective figures can beobtained easily, the PS test results rely completely on userpreference. To get a more objective result, a sample group of40 candidates were invited to evaluate the system. In thistest, each candidate “bought” one product from eachcategory according to his or her own requirements. In theevaluation, the candidates browsed through the e-catalogand chose a list of their “best five choices” (L), which bestmatched their tastes. This list was compared with the topfive products that were recommended by the fuzzyshopper. To evaluate the effectiveness of the fuzzyshopper’s recommendation, the “Fitness Value” (FV) iscalculated as follows:
FV ¼
P5n¼1
n� i
15where i ¼ 1 if i 2 L
0 otherwise:
�ð10Þ
In the above calculation, scores from 5 to 1 (from the firstto the last ranked choice) are given to each of the correctmatches between a candidate’s list and the recommendationsfrom the FShopper. For example, if the same products thatare ranked 1, 2, 3, and 5 appear in the fuzzy shopper
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 471
TABLE 4Recognition Results of the Occlusion and Distortion Test
TABLE 5Mean RTT Summary after 100 Trials
recommendation, then the fitness value is 73 percent (i.e., the
sum of 1, 2, 3, and 5 divided by 15). The results under theeight different categories of products are shown in Table 6.
It is obvious that the performance of the FShopper is
highly dependent on the “variability” (or “fuzziness”) that
is associated with a description of the merchandise. The
higher the fuzziness (which means more variety), the lower
the fitness value (score). As shown in Table 6, the two
product categories of skirts and shoes are typical examples
in which skirts score 65 percent and shoes score 89 percent.
Nevertheless, the average score for the system is still over
81 percent. Note that these figures are only for illustration
since human justification and product variety in real
scenarios will vary case by case.
7 CONCLUSION
This paper proposes an innovative, intelligent agent-basedWeb mining application: the iJADE Web miner. This isbased on the integration of neuro-fuzzy-based Web mining
technology and intelligent visual data mining technology toautomate user authentication. We hope that it will provide anew era of Web-based data mining and knowledge
discovery using intelligent agent-based systems.The major contributions of this paper can be summarized
in two areas: research and application. From a researchpoint of view, this paper demonstrates a feasible andefficient solution (iJADE Web miner) for major aspects of
Web mining: 1) automatic human-face identification andrecognition and 2) interactive and mobile agent-basedproduct search from a large database. With respect to
human-face recognition, this paper proposes an automaticelastic graph matching model for human-face “contour”extraction, identification, and face recognition. Through theintegration of the Active Contour Model (ACM) with the
Elastic Graph Dynamic Link Model (EGDLM), this paperillustrates the efficiency and effectiveness of iJADE Webminer for automatic online human-face recognition. Withrespect to the interactive and mobile agent-based product
search, this paper illustrates how intelligent agent technol-ogy can be successfully integrated with AI technology foronline product identification and search. Through the
implementation of a fuzzy neuro-based product classifica-tion and search function in our proposed iJADE framework,
this paper illustrates the flexibility and efficiency of iJADE
Web miner for interactive online shopping.From an application point of view, this paper demon-
strates how various AI technologies (including fuzzy logicand neural networks) can be successfully integrated withmobile-agent technology to provide a truly intelligent,mobile, and interactive Web mining solution. We hope thatthe implementation of the iJADE Web miner can provide anew era for future e-commerce: intelligent e-commerce.
ACKNOWLEDGMENTS
The authors are grateful for the partial support provided by
the Central Research Grants, B-Q569 and G-T375, and the
Departmental (Department of Computing) Grant for the
iJADE project (Z042) from the Hong Kong Polytechnic
University.
REFERENCES
[1] ABE, http://www.research.ibm.com/iagents/home/html, 2003.[2] Aglets, http://www.trl.ibm.co.jp/aglets/, 2003.[3] P. Atzeni, A. Masci, G. Mecca, P. Merialdo, and E. Tabet,
“ULIXES: Building Relational Views over the Web,” Proc. 13thInt’l Conf. Data Eng., p. 576, 1997.
[4] AuctionBot, http://auction.eecs.umich.edu, 1999.[5] BargainFinder, http://bf.cstar.ac.com/bf, 2001[6] J. Bettman, An Information Processing Theory to Consumer Choice.
Addison-Wesley, 1979.[7] D. Caughey, I. Ingham, and P. Watson, “Metabroker: A Generic
Broker for Electronic Commerce,” Computer Networks and ISDNSystems, vol. 30, no. 1, pp. 619-620, 1998.
[8] D. Che, Y. Chen, K. Aberer, and H. Eisner, “The Advanced WebQuery System of GPCRDB,” Proc. 11th Int’l Conf. Scientific andStatistical Database Management, pp. 879-885, 1999.
[9] R. Cooley, B. Mobasherand, and J. Srivastava, “Web Mining:Information and Pattern Discovery on the World Wide Web,”Proc. Ninth Int’l Conf. Tools with A. I. ‘97, pp. 558-567, 1997.
[10] P. Dasgupta, N. Narasimhan, L.E. Moser, and P.M. Smith,“MAGNET: Mobile Agents for Networked Electronic Trading,”IEEE Trans. Knowledge and Data Eng., vol. 11, no. 4, pp. 509-525,July/Aug. 1999.
[11] J. Engeland and R. Blackwell, Consumer Behavior. CBS CollegePublishing, 1982.
[12] U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy,Advances in Knowledge Discovery and Data Mining. AAAI Press/The MIT Press, 1996.
[13] FTP software agents, http://www.ftp.com, 2000.[14] K. Hammond, R. Burke, C. Martin, and S. Lytinen, “FAQ-Finder:
A Case-Based Approach to Knowledge Navigation,” WorkingNotes of the AAAI Spring Symposium: Information Gathering fromHeterogeneous, Distributed Environments, pp. 1136-1142, AAAIPress, 1995.
[15] I. Hossain, J. Liu, and R. Lee, “A Study of Multilingual SpeechFeature: Perspective Scalogram Based on Wavelet Analysis,” Proc.IEEE Int’l Conf. Systems, Man, and Cybernetics (SMC ’99), vol. II,pp. 178-183, 1999.
[16] J. Howard and J. Sheth, The Theory of Buyer Behavior. John Wileyand Sons, 1969.
[17] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active ContourModels,” Int’l J. Computer Vision, pp. 321-331, 1987.
[18] M. Klusch, Intelligent Information Agents: Agent-Based InformationDiscovery and Management on the Internet. Berlin: Springer-Verlag,1999.
[19] R.S.T. Lee and J.N.K. Liu, “iJADE IWShopper—A New Age ofIntelligent Mobile Web Shopping System Based on Fuzzy-NeuroAgent Technology,” Web Intelligence: Research and Development(LNAI 2198), pp. 403-412, 2001.
[20] R.S.T. Lee and J.N.K. Liu, “iJADE Stock Advisor—A New Era ofE-Finance Using an Intelligent Agent-Based Time Series StockPrediction Model,” Proc. Int’l Conf. Artificial Intelligence (ICAI2001), pp. 43-46, 2001.
472 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004
TABLE 6Fitness Values for the Eight Different Product Categories
[21] R.S.T. Lee and J.N.K. Liu, “Tropical Cyclone Identification andTracking System Using Integrated Neural Oscillatory ElasticGraph Matching and Hybrid RBF Network Track MiningTechniques,” IEEE Trans. Neural Network, vol. 11, no. 3, pp. 680-689, 2000.
[22] R.S.T. Lee and J.N.K. Liu, “FAgent—An Innovative E-ShoppingAuthentication Scheme Using Invariant Intelligent Face Recogni-tion Agent,” Proc. Int’l Conf. Electronic Commerce (ICEC 2000),pp. 47-53, 2000.
[23] R.S.T. Lee and J.N.K. Liu, “Teaching and Learning the A.I.Modeling,” Innovative Teaching Tools: Knowledge-Based Paradigms(Studies in Fuzziness and Soft Computing 36), pp. 31-86, Physica-Verlag, Springer, 2000.
[24] R.S.T. Lee and J.N.K. Liu, “An Integrated Elastic Contour Fittingand Attribute Graph Matching Model for Automatic Face Codingand Recognition,” Proc. Third Int’l Conf. Knowledge-Based IntelligentInformation Eng. Systems (KES ’99), pp. 292-295, 1999.
[25] R.S.T. Lee and J.N.K. Liu, “An Automatic Satellite Interpretationof Tropical Cyclone Patterns Using Elastic Graph Dynamic LinkModel,” Int’l J. Pattern Recognition and Artificial Intelligence, vol. 13,no. 8, pp. 1251-1270, 1999.
[26] R.S.T. Lee and J.N.K. Liu, “An Oscillatory Elastic Graph MatchingModel for Scene Analysis,” Proc. Int’l Conf. Imaging Science,Systems, and Technology (CISST ’99), pp. 42-45, 1999.
[27] H.V. Leighton and J. Srivastava, “Precision among WWWSearch Services (Search Engines): Alta Vista, Excite, Hotbot,Infoseek, Lycos,” http://www.winona.musus.edu/is-f/library-f/webind2/webind2.htm, 1997.
[28] J.N.K. Liu and R.S.T. Lee, “Rainfall Forecasting from MultiplePoint Source Using Neural Networks,” Proc. IEEE Int’l Conf.Systems, Man, and Cybernetics (SMC ’99), vol. II, pp. 429-434, 1999.
[29] J.N.K. Liu and R.S.T. Lee, “Invariant Character Recognition inDynamic Link Architecture,” Proc. Int’l Conf. Knowledge and DataExchange (KDEX ’97), pp. 188-195, 1997.
[30] J.N.K. Liu and R.S.T. Lee, “Invariant Handwritten ChineseCharacter Recognition,” Proc. Int’l Conf. Neural Information Proces-sing (ICONIP ’98), pp. 275-278, 1998.
[31] C. Malsburg, “Pattern Recognition by Label Graph Matching,”Neural Networks, vol. 1, no. 1, pp. 141-148, 1988.
[32] Odyssey, http://www.isysdev.com/index.shtml, 2003.[33] J. Pitkow, “In Search of Reliable Usage Data on the WWW,” Proc.
Sixth Int’l WWW Conf., pp. 451-463, 1997.[34] W. Rankl and W. Effing, Smart Card Handbook. Wiley, 1997.[35] U. Shardanand and P. Maes, “Social Information Filtering:
Algorithms for Automating ‘Word of Mouth’,” Proc. Int’l Conf.Human Factors in Computing Systems ‘95, pp. 210-217, 1995.
[36] E. Spertus, “ParaSite: Mining Structural Information on the Web,”Proc. Sixth Int’l WWW Conf., pp. 485-492, 1997.
[37] J. Triesch and C. Malsburg, “Robotic Gesture Recognition,” Proc.Second Conf. Automatic Face and Gesture Recognition ‘96, pp. 170-175,1996.
[38] Voyager, http://www.recursionsw.com/osi.asp, 2003.[39] J. Wang, Y. Huang, G. Wu, and F. Zhang, “Web Mining:
Knowledge Discovery on the Web,” Proc. Int’l Conf. Systems,Man and Cybernetics (SMC ’99), vol. 2, pp. 137-141, 1999.
Raymond S.T. Lee received the BSc degreefrom the Hong Kong University in 1989, the MScdegree in information technology, and the PhDdegree from the Hong Kong Polytechnic Uni-versity in 1997 and 2000, respectively. Aftergraduation from the Hong Kong University, hejoined the Hong Kong Government in the HongKong Observatory as a meteorological scientiston weather forecasting and developing meteor-ological telecommunication information systems
from 1989 to 1993. Prior to joining Hong Kong Polytechnic University in1998, he worked as an MIS manager and system consultant in HongKong. He is now an assistant professor in the Department of Computingat the Hong Kong Polytechnic University, working in the areas of neuralnetwork and pattern recognition. His current research interests include:artificial intelligence, intelligent e-commerce systems, intelligent agents,weather simulation and forecasting, and chaotic neural networks. He is amember of the IEEE and the ACM.
James N.K. Liu received the BSc (Hons) andMPhil degrees in mathematics and computa-tional modeling from Murdoch University, Aus-tralia, in 1982 and 1987, respectively. Hereceived the PhD degree in artificial intelligencefrom La Trobe University, Australia, in 1992.While working on his degree, he worked as acomputer scientist at the Defence Signal Direc-torate in Australia from 1988 until 1990. Hejoined the Aeronautical Research Laboratory
(ARL) at the Defence Science and Technology Organization in Australiaas a research scientist in 1990. Dr Liu joined the Department ofComputing at the Hong Kong Polytechnic University in 1994 and is nowan associate professor. His current interests include intelligent businesscomputing, multilingual system development, weather simulation andforecasting, data mining and Web-based information systems, andagent modeling. He is a member of the IEEE and AAAI.
. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.
LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 473