electronic commerce technologies
TRANSCRIPT
Electronic Commerce TechnologiesCSC 513
Spring 2011
Munindar P. Singh, [email protected]
Department of Computer ScienceNorth Carolina State University
Munindar P. Singh (NCSU) Electronic Commerce Technologies 1 / 303
Mechanics
I Scope
I GradingI Policies
I Especially, academic integrity
Munindar P. Singh (NCSU) Electronic Commerce Technologies 2 / 303
Scope of this Course
I Directed at computer science students
I Emphasizes concepts and theory
I Requires a moderate amount of work
I Fairly easy if you don’t let things slip
Munindar P. Singh (NCSU) Electronic Commerce Technologies 3 / 303
Challenges of Electronic Business
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 4 / 303
Challenges of Electronic Business
Electronic Business
I B2C: retail, finance
I B2B: supply chains (more generally, supply networks)I Different perspectives
I Traditionally: merchant, customer, dealmakerI Trends: collaboration among various parties; virtual enterprises;
coalition formation
Main technical consequence: interacting across enterprise boundaries oradministrative domains
Munindar P. Singh (NCSU) Electronic Commerce Technologies 5 / 303
Challenges of Electronic Business
Properties of Business Environments
I Traditional computer science deals with closed environmentsI Business environments are open
I Autonomy: independent action (how will the other party act?)I Heterogeneity: independent design (how will the other party represent
information?)I Dynamism: independent configuration (which other party is it?)
I Usually, also large scale
I Need flexible approaches and arms-length relationships
Munindar P. Singh (NCSU) Electronic Commerce Technologies 6 / 303
Challenges of Electronic Business Business Environments
Outline
Challenges of Electronic BusinessBusiness EnvironmentsService Engagements
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 7 / 303
Challenges of Electronic Business Business Environments
Autonomy
Independence of business partnersI Sociopolitical or economic reasons
I Ownership of resources by partnersI Control, especially of access privilegesI Payments
I Technical reasons: opacity with respect to key features, e.g.,precommit
I Model components as autonomous to simplify interfaces “assumenothing”
I Model components as autonomous to accommodate underlyingexceptions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 8 / 303
Challenges of Electronic Business Business Environments
Heterogeneity
Independence of component designers and system architects
I Historical reasonsI Sociopolitical reasons
I Differences in local needsI Difficulty of achieving agreement
I Technical reasons: difficulty in achieving homogeneityI Conceptual problems: cannot easily agreeI Fragility: a slight change can mess it up
Munindar P. Singh (NCSU) Electronic Commerce Technologies 9 / 303
Challenges of Electronic Business Business Environments
Dynamism
Independence of system configurers and administratorsI Sociopolitical reasons
I Ownership of resourcesI Changing user preferences or economic considerations
I Technical reasons: difficulty of maintaining configurations by handI Same reasons as for network administrationI Future-proofing your system
Munindar P. Singh (NCSU) Electronic Commerce Technologies 10 / 303
Challenges of Electronic Business Business Environments
Coherence
Think of this as an alternative to consistencyI There may be no state (of the various databases) that can be
considered consistentI Maintaining consistency of multiple databases is difficultI Unexpected real-world events can knock databases out of sync with
reality
I What matters isI Are organizational relationships preserved?I Are processes followed?I Are appropriate business rules applied?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 11 / 303
Challenges of Electronic Business Business Environments
Integration
Yields with one integrated entity
I Yields central decision making by homogeneous entity
I Requires resolving all potential inconsistencies ahead of time
I Fragile and must be repeated whenever components change
Obsolete way of thinking: tries to achieve consistency (and fails)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 12 / 303
Challenges of Electronic Business Business Environments
Locality and Interaction
A way to maintain coherence in the face of opennessI Have each local entity look after its own
I Minimize dependence on othersI Continually have interested parties verify the components of the state
that apply to them
I Approach: replace global constraints with protocols for interactionI Lazy: obtain global knowledge as neededI Optimistic: correct rather than prevent violationsI Inspectable: specify rules for when, where, and how to make corrections
Munindar P. Singh (NCSU) Electronic Commerce Technologies 13 / 303
Challenges of Electronic Business Business Environments
Interoperation
Ends up with the original number of entities working together
I Yields decentralized decision making by heterogeneous entities
I Resolves inconsistencies incrementally
I Potentially robust and easy to swap out partners as needed
Also termed “light integration” (bad terminology)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 14 / 303
Challenges of Electronic Business Business Environments
Example: Selling
Update inventory, take payment, initiate shipping
I Record a sale in a sales database
I Debit the credit card (receive payment)
I Send order to shipper
I Receive OK from shipper
I Update inventory
Munindar P. Singh (NCSU) Electronic Commerce Technologies 15 / 303
Challenges of Electronic Business Business Environments
Potential Problems Pertaining to Functionality
I What if the order is shipped, but the payment fails?
I What if the payment succeeds, but the order was never entered orshipped?
I What if the payments are made offline, i.e., significantly delayed?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 16 / 303
Challenges of Electronic Business Business Environments
Architectural ConsiderationsArchitecture is motivated by additional considerations besides functionality
I Instance level, nonfunctional properties such as the availability of aspecific service instance
I What if the payments are made offline, i.e., significantly delayed?
I Metalevel properties such as the maintainability of the softwaremodules and the ease of the upgradability of the system
Munindar P. Singh (NCSU) Electronic Commerce Technologies 17 / 303
Challenges of Electronic Business Business Environments
In a Closed Environment
I Transaction processing (TP) monitors ensure that all or none of thesteps are completed, and that systems eventually reach a consistentstate
I But what if the user is disconnected right after he clicks on OK? Didorder succeed? What if line went dead before acknowledgmentarrives? Will the user order again?
I The TP monitor cannot get the user into a consistent state
Munindar P. Singh (NCSU) Electronic Commerce Technologies 18 / 303
Challenges of Electronic Business Business Environments
In an Open Environment: 1
I Reliable messaging (asynchronous communication, which guaranteesmessage delivery or failure notification)
I Maintain state: retry if needed
I Detect and repair duplicate transactions
I Engage user about credit problems
Matter of policies to ensure compliance
Munindar P. Singh (NCSU) Electronic Commerce Technologies 19 / 303
Challenges of Electronic Business Business Environments
In an Open Environment: 2
I Not immediate consistency
I Eventual “consistency” (howsoever understood) or just coherence
I Sophisticated means to maintain shared state, e.g., conversations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 20 / 303
Challenges of Electronic Business Business Environments
Challenges
I Information system interoperation
I Business operations
I Exception handling
I Distributed decision-making
I Personalization
I Service selection (location and assessment)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 21 / 303
Challenges of Electronic Business Business Environments
Information System Interoperation
Supply chains: manage the flow of materiel among a set of manufacturersand integrators to produce goods and configurations that can be suppliedto customers
I Requires the flow of information and negotiation aboutI Product specificationsI Delivery requirementsI Prices
Munindar P. Singh (NCSU) Electronic Commerce Technologies 22 / 303
Challenges of Electronic Business Business Environments
Business OperationsModeling and optimization
I Inventory management
I Logistics: how to optimize and monitoring flow of materiel
I Billing and accounts receivable
I Accounts payable
I Customer support
Munindar P. Singh (NCSU) Electronic Commerce Technologies 23 / 303
Challenges of Electronic Business Business Environments
Exception Conditions
Virtual enterprises to construct enterprises dynamically to provide moreappropriate, packaged goods and services to common customers
I Requires the ability toI Construct teamsI Enter into multiparty dealsI Handle authorizations and commitmentsI Accommodate exceptions
I Real-world exceptions
I Compare with PL or OS exceptions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 24 / 303
Challenges of Electronic Business Business Environments
Distributed Decision-Making: Closed
Manufacturing control: manage the operations of factoriesI Requires intelligent decisions to
I Plan inflow and outflowI Schedule resourcesI Accommodate exceptions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 25 / 303
Challenges of Electronic Business Business Environments
Distributed Decision-Making: Open
Automated markets as for energy distributionI Requires abilities to
I Set prices, place or decide on others’ bidsI Accommodate risks
I Pricing mechanisms for rational resource allocation
Munindar P. Singh (NCSU) Electronic Commerce Technologies 26 / 303
Challenges of Electronic Business Business Environments
Personalization
Consumer dealings to make the shopping experience a pleasant one for thecustomer
I RequiresI Learning and remembering the customer’s preferencesI Offering guidance to the customer (best if unintrusive)I Acting on behalf of the user without violating their autonomy
Munindar P. Singh (NCSU) Electronic Commerce Technologies 27 / 303
Challenges of Electronic Business Business Environments
Service Selection
What are some bases for selecting the parties to deal with?I Specify services precisely and search for them
I How do you know they do what you think they do (ambiguity)?I How do you know they do what they say (trust)?
I Recommendations to help customers find relevant and high qualityservices
I How do you obtain and aggregate evaluations?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 28 / 303
Challenges of Electronic Business Service Engagements
Outline
Challenges of Electronic BusinessBusiness EnvironmentsService Engagements
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 29 / 303
Challenges of Electronic Business Service Engagements
The Evolution of IT
I Applications: Control of computations hidden in code; integration anightmare
I Workflows: Control abstracted out; integration still difficult
I Standards-driven orchestration: Integration improved; limitedsupport for autonomy
I Messaging: Integration simplified by MoM and transformations;limited support for autonomy
I Choreography: Model conversations over messages; limited supportfor autonomy
I Governance: Administer resources via interactions amongautonomous parties
Munindar P. Singh (NCSU) Electronic Commerce Technologies 30 / 303
Challenges of Electronic Business Service Engagements
Technical Service
I Generally, an abstraction of a computational objectI Traditional, as in web or grid servicesI Improved: Abstraction of a “capability”
I Well encapsulated, i.e., a black box
I Interface defined at the level of methods or messages
Munindar P. Singh (NCSU) Electronic Commerce Technologies 31 / 303
Challenges of Electronic Business Service Engagements
Service EngagementAn aggregation of business relationships
I Trillions of dollars worth of commerce conducted every yearI Characterized by
I Independence of business partnersI Coproduction
I Participation by all, though not at the same levelI Symmetric relationships: complementary capabilities and goals
I Complex contracts among the partnersI Participants are not black boxes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 32 / 303
Challenges of Electronic Business Service Engagements
Business ServiceParticipant in a service engagement
I Characterized by transfer of value, not bits
I Typically long-lived with on demand enactmentsI Instantiated on the fly
I Unlike a productI Though may be constructed using products or about products
Munindar P. Singh (NCSU) Electronic Commerce Technologies 33 / 303
Challenges of Electronic Business Service Engagements
Conceptual Elements of a Service Engagement
I Transactional: main purpose and enactment, specifying valueexchanged
I Structural: partnerships and contractsI Contextual: setting of the engagement
Munindar P. Singh (NCSU) Electronic Commerce Technologies 34 / 303
Challenges of Electronic Business Service Engagements
Traditional Technical ApproachesQuite unlike a real-life service engagement
I Take participants flows (e.g., in BPEL, BPMN) as units ofabstraction
I Mix private policies and public interactionsI Proprietary: may not be available for reuseI Context-laden: even when available, cannot be readily reused
I Focus on low-level (e.g., WS-CDL) or data-level meanings (e.g.,OWL)
I Ignore business-level significance of messagesI Ambiguous; not verifiable
BPEL, BPMN, WS-CDL, OWL are well-known standards
Munindar P. Singh (NCSU) Electronic Commerce Technologies 35 / 303
Challenges of Electronic Business Service Engagements
A Real-Life Service EngagementOperationally over-specified as interacting flows
Munindar P. Singh (NCSU) Electronic Commerce Technologies 36 / 303
Architecture in IT
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 37 / 303
Architecture in IT
Architecture in ITIn the sense of information systems
I Important themesI Conceptualizing architectureI Enterprise architecturesI Tiered architecturesI Architecture as a basis for governance (next section)
I Not quite the same as conventional software architecture, though thetopics are converging
Munindar P. Singh (NCSU) Electronic Commerce Technologies 38 / 303
Architecture in IT
Architecture ConceptuallyAs opposed to the description of a system via a blueprint
I How a system is organizedI An over-used, vaguely defined term
I Software architectureI Standards, e.g., Berners-Lee’s “layer cake” and the networking
standardsI May include processes
I That exercise the systemI By which the system is built and maintainedI By which the system is administered
I May include human organizations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 39 / 303
Architecture in IT
Understanding Architecture: 1
I Two main ingredients of a systemI ComponentsI Interconnections
I Openness entails specifying the interconnections cleanlyI Physical components disappearI Their logical traces remain
I Information environments mean that the interconnections areprotocols
Munindar P. Singh (NCSU) Electronic Commerce Technologies 40 / 303
Architecture in IT
Exercise: Examples of ArchitectureIdentify the main components and interconnections
I Buildings
I Plumbing
I Power systems
Munindar P. Singh (NCSU) Electronic Commerce Technologies 41 / 303
Architecture in IT
Understanding Architecture: 2
I Components and interconnections are not sufficient to characterize anarchitecture
I Two additional ingredients of an architectureI Constraints on the components and interconnnectionsI Patterns involving the components and interconnnections
I Openness entails the constraintsI Do not apply on the physical components directly
Munindar P. Singh (NCSU) Electronic Commerce Technologies 42 / 303
Architecture in IT
Exercise: Examples of ArchitectureIdentify the main constraints and key patterns
I Buildings
I Plumbing
I Power systems
Munindar P. Singh (NCSU) Electronic Commerce Technologies 43 / 303
Architecture in IT
Understanding Protocols
I Protocols encapsulate IT interactions, i.e., interconnections overwhich information is the main thing that flows
I Connect: conceptual interfacesI Separate: provide clean partitions among logical components
I Wherever we can identify protocols, we canI Make interactions explicitI Enhance reuseI Improve productivityI Identify new markets and technologies
I Protocols yield standards; their implementations yield products
Munindar P. Singh (NCSU) Electronic Commerce Technologies 44 / 303
Architecture in IT
Examples of Logical Architectural ComponentsEach logical component class serves some important function
I Power: UPS
I Network connectivity
I Storage: integrity, persistence, recovery
I Policy management
I Decision-making
I Knowledge and its management
What are some products in the above component classes?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 45 / 303
Architecture in IT Enterprise Architecture
Outline
Challenges of Electronic Business
Architecture in ITEnterprise ArchitectureTiered ArchitectureWeb ArchitectureMiddlewareDeployment Architecture
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 46 / 303
Architecture in IT Enterprise Architecture
IT Architectures
The term architecture is used more broadly in IT settings
I The organization of an IT system
I The extensibility and modifiability of a system
I Even the governance of a system, which inevitably accommodates thehuman organization where the system is deployed
Munindar P. Singh (NCSU) Electronic Commerce Technologies 47 / 303
Architecture in IT Enterprise Architecture
IT and SOA GovernanceThe human administration of an IT system
I IT Governance: How IT resources are administered
I SOA Governance: How services are created, deployed, removed, . . .I Goes hand-in-hand with architecture
I IncorporatesI The human organization of a systemI The processes through which a system is updated or upgradedI Nontechnical aspects, such as flows of responsibility
I Sometimes confused with architecture, but distinct
Munindar P. Singh (NCSU) Electronic Commerce Technologies 48 / 303
Architecture in IT Enterprise Architecture
Governance in the Service Life CycleKey determinations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 49 / 303
Architecture in IT Enterprise Architecture
Enterprise Models: Information ResourcesCapture static and dynamic aspects
I StaticallyI Databases and knowledge basesI Applications, business processes, and the information they create,
maintain, and use
I Through explicit representations, dynamically enableI Integrity validationI ReusabilityI Change impact analysisI Software engineering: Automatic database and application generation
via CASE tools
Munindar P. Singh (NCSU) Electronic Commerce Technologies 50 / 303
Architecture in IT Enterprise Architecture
Enterprise Models: Rationales
I Capture (human) organizational structureI Document business functions
I Rationales behind designs of databases and knowledge basesI Justifications for applications and business processes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 51 / 303
Architecture in IT Enterprise Architecture
Enterprise Architecture Objectives
At the top-level, to support the business objectives of the enterprise; thesecommonly translate into
I Accommodating change by introducing newI UsersI ApplicationsI Ways of interaction (e.g., ongoing push toward mobility)
I Managing information resourcesI Preserving prior investments by interoperating with legacy systemsI Upgrading resources
I Developing blueprints to guide resource and application installationand decommissioning
Munindar P. Singh (NCSU) Electronic Commerce Technologies 52 / 303
Architecture in IT Enterprise Architecture
Enterprise Architecture Observations
Continual squeeze on funds, staffing, and time available for IT resources
I Demand for rapid development and deployment of applications
I Demand for greater ROII Essential tension
I Need to empower users and suborganizations to ensure satisfaction oftheir local and of organizational needs
I Ad hoc approaches with each user or each suborganization doing itsown IT cause failure of interoperability
Munindar P. Singh (NCSU) Electronic Commerce Technologies 53 / 303
Architecture in IT Enterprise Architecture
Enterprise Architecture PrinciplesBusiness processes should drive the technical architecture
I Define dependencies and other relationships among stakeholders(including users) and suborganizations of an organization
I Message-driven approaches are desirable because they decouplesystem components
I Event-driven approaches are desirable because they help make asystem responsive to events that are potentially visible and significantto users
Munindar P. Singh (NCSU) Electronic Commerce Technologies 54 / 303
Architecture in IT Enterprise Architecture
Architecture Modules: Applications
I Often directly visible to usersI Application deploymentI Data modeling and integrityI Business intelligence: decision support and analytics
I More technical but indirectly visible to users
I Interoperation and cooperationI Ontologies: representations of domain knowledge
I Component and model repositoriesI Business process management
Munindar P. Singh (NCSU) Electronic Commerce Technologies 55 / 303
Architecture in IT Enterprise Architecture
Architecture Modules: Systems
Functionality used by multiple applications
I Middleware: enabling interoperation, e.g., via messaging
I Identity management, e.g., ID across a system to support Single SignOn
I Security and audit
I Accessibility
I Policy repositories and engines
Munindar P. Singh (NCSU) Electronic Commerce Technologies 56 / 303
Architecture in IT Enterprise Architecture
Architecture Modules: Infrastructure
I Connectivity
I Platform: hardware and operating systems
I Storage
I System management
Munindar P. Singh (NCSU) Electronic Commerce Technologies 57 / 303
Architecture in IT Enterprise Architecture
Functionalities in a Working Enterprise System
I Presentation: user interactionI A large variety of concerns about device constraints and usage scenarios
I Business logicI Application-specific reasoningI General rules
I Data managementI Ensuring integrity, e.g., entity and referential integrity (richer than
storage-level integrity)I Enabling access under various kinds of problems, e.g., network
partitionsI Supporting recovery, e.g., application, operating system, or hardware
failures
Munindar P. Singh (NCSU) Electronic Commerce Technologies 58 / 303
Architecture in IT Enterprise Architecture
Enterprise Functionalities
Bases for choosing the above three-way partitioning as opposed to someother
I Size of implementations
I Organizational structure: who owns what and who needs whatI Staff skill sets
I User Interface: usability and designI ProgrammingI DatabaseI Policy tools
I Products available in the marketplace
Munindar P. Singh (NCSU) Electronic Commerce Technologies 59 / 303
Architecture in IT Tiered Architecture
Outline
Challenges of Electronic Business
Architecture in ITEnterprise ArchitectureTiered ArchitectureWeb ArchitectureMiddlewareDeployment Architecture
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 60 / 303
Architecture in IT Tiered Architecture
One-Tier and Two-Tier Architectures
I One tier: monolithic systems; intertwined in the code baseI Historically the firstI Common in legacy systemsI Difficult to staff, maintain, and scale up
I Two-tier: separate data from presentation and business logicI Classical client-server (or fat client) approachesI Mix presentation with application business rulesI Change management
Munindar P. Singh (NCSU) Electronic Commerce Technologies 61 / 303
Architecture in IT Tiered Architecture
Three-Tier Architecture
I Presentation tier or frontendI Provides a view to user and takes inputsI Invokes the same business logic regardless of interface modalities:
voice, Web, small screen, . . .
I Business logic tier or middle tierI Specifies application logicI Specifies business rules
I Application-level policiesI InspectableI Modifiable
I Data tier or backendI Stores and provides access to dataI Protects integrity of data via concurrency control and recovery
Munindar P. Singh (NCSU) Electronic Commerce Technologies 62 / 303
Architecture in IT Tiered Architecture
Multitier ArchitectureAlso known as n-tier
I Best understood as a componentized version of three-tier architecturewhere
I Functionality is assembled from parts, which may themselves beassembled
I Supports greater reuse and enables greater dynamismI But only if the semantics is characterized properly
I Famous subclass: service-oriented architecture
Munindar P. Singh (NCSU) Electronic Commerce Technologies 63 / 303
Architecture in IT Tiered Architecture
Architectural Tiers EvaluatedThe tiers reflect logical, not physical partitioning
I The more open the architecture the greater the decoupling amongcomponents
I Improves development through reuseI Enables composition of componentsI Facilitates governance, including scaling up of resourcesI Sets boundaries for organizational control
I In a narrow sense, having more moving parts can complicatemanagement
I But improved architecture facilitates management through divide andconquer
Munindar P. Singh (NCSU) Electronic Commerce Technologies 64 / 303
Architecture in IT Tiered Architecture
XML-Based Information System
Let’s place XML in a multitier architecture
Munindar P. Singh (NCSU) Electronic Commerce Technologies 65 / 303
Architecture in IT Tiered Architecture
How About Database Triggers?
I Pros: essential for achieving high efficiencyI Reduce network load and materializing and serializing costsI Leave the heavy logic in the database, under the care of the DBA
I Cons: rarely port well across vendorsI Difficult to introduce and manage because of DBA controlI Business rules are context-sensitive and cannot always be applied
regardless of how the data is modified
Munindar P. Singh (NCSU) Electronic Commerce Technologies 66 / 303
Architecture in IT Web Architecture
Outline
Challenges of Electronic Business
Architecture in ITEnterprise ArchitectureTiered ArchitectureWeb ArchitectureMiddlewareDeployment Architecture
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 67 / 303
Architecture in IT Web Architecture
Web ArchitecturePrinciples and constraints that characterize Web-based information systems
I URI: Uniform Resource Identifier
I HTTP: HyperText Transfer ProtocolI Metadata must be recognized and respected
I Enables making resources comprehensible across administrativedomains
I Difficult to enforce unless the metadata is itself suitably formalized
Munindar P. Singh (NCSU) Electronic Commerce Technologies 68 / 303
Architecture in IT Web Architecture
Uniform Resource Identifier: 1
I URIs are abstract
I What matters is their (purported) uniqueness
I URIs have no proper syntax per seI Kinds of URIs include
I URLs, as in browsing: not used in standards any moreI URNs, which leave the mapping of names to locations up in the air
Munindar P. Singh (NCSU) Electronic Commerce Technologies 69 / 303
Architecture in IT Web Architecture
Uniform Resource Identifier: 2
Good design requirements
I Ensure that the identified resource can be located
I Ensure uniqueness: eliminate the possibility of conflicts throughappropriate organizational and technical means
I Prevent ambiguity
I Use an established URI scheme where possible
Munindar P. Singh (NCSU) Electronic Commerce Technologies 70 / 303
Architecture in IT Web Architecture
HTTP: HyperText Transfer Protocol
Intended meanings are quite strict, though not constrained byimplementations
I Text-based, statelessI Key verbs (and some others)
I GetI PostI Put
I Error messages for specific situations, such as resources not available,redirected, permanently moved, and so on
ReST: Representational State Transfer
Munindar P. Singh (NCSU) Electronic Commerce Technologies 71 / 303
Architecture in IT Web Architecture
Representational State Transfer
ReST is an architectural style for networked systems that constrains theconnectors
I Models the Web as a network of hyperlinked resources, each identifiedby a URI
I Models a Web application as a (virtual) state machine
I A client selecting a link effects a state transition, resulting in receivingthe next page (next state) of the application
Munindar P. Singh (NCSU) Electronic Commerce Technologies 72 / 303
Architecture in IT Web Architecture
Characteristics of ReST
I Client-ServerI Statelessness: in terms of sessions
I What is an advantage of statelessness?I Where is the session state kept then?
I Focus on resources being manipulated and their representations beingtransferred
I Uniform Interface: URIs, hypermedia
I Caching: responses can be labeled as cacheable
Munindar P. Singh (NCSU) Electronic Commerce Technologies 73 / 303
Architecture in IT Middleware
Outline
Challenges of Electronic Business
Architecture in ITEnterprise ArchitectureTiered ArchitectureWeb ArchitectureMiddlewareDeployment Architecture
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 74 / 303
Architecture in IT Middleware
Middleware PracticallyComponents with routine, reusable functionality
I Abstracted from the application logic or the backend systems
I Any functionality that is being repeated is a candidate for beingfactored out into middleware
I Enables plugging in endpoints (e.g., clients and servers) according tothe stated protocols
I Often preloaded on an application server
I Simplify programmer’s task and enable refinements and optimizations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 75 / 303
Architecture in IT Middleware
Middleware ConceptuallyComponents that implement important architectural interfaces
Key examples: transaction and persistenceI Explicit:
I Invoke specialized APIs explicitlyI Ties application to middleware API: Difficult to create, maintain, port
I Implicit:I Container invokes the appropriate APIsI Based on declarative specificationsI Relies on request interceptions or reflection
Munindar P. Singh (NCSU) Electronic Commerce Technologies 76 / 303
Architecture in IT Middleware
ContainersDistributed object management
I Architectural abstraction geared for hosting business components(objects)
I Remote method invocationI ThreadingI MessagingI Transactions
I Implementations for JEE and .NET
Munindar P. Singh (NCSU) Electronic Commerce Technologies 77 / 303
Architecture in IT Middleware
JEE TechnologyVernadat
EJB Server
Entity Bean
Session
Bean
J2EE
Connector
Relational
DBMS
Legacy
System
CORBA Client
Java Applet in
Browser
Java
Applications
(Swing, AWT)
Web Browser
Servlet JSP
RMI, IIOP
XML, HTML, HTTP (SSL)
RMI
RMI
RMI
Java Message
Service
Java Naming and
Directory
Interface
Operating System (Windows,
Linux, Mac, Solaris…)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 78 / 303
Architecture in IT Middleware
.NET TechnologyVernadat
Microsoft Transaction Server
Shared Property
Manager
COM+
COMPONENT
Babylon Integration
Server
Relational
DBMS
Legacy
System
ADO,
OCEDB,
ODBC
CORBA Client
ActiveX Control
in Browser
Applications
Web Browser
IIS/ASP
XML, HTML, HTTP (SSL)
Microsoft
Message
Queue
Active Directory
Windows Operating System
Munindar P. Singh (NCSU) Electronic Commerce Technologies 79 / 303
Architecture in IT Middleware
Basic Interaction Models
Interactions among autonomous and heterogeneous partiesI Adapters: what are exposed by each party to enable interoperation
I Sensors ⇐ informationI Effectors ⇒ actions
I Invocation-based adapters
I Message-oriented middleware
I Peer-to-peer computing
Munindar P. Singh (NCSU) Electronic Commerce Technologies 80 / 303
Architecture in IT Middleware
Invocation-Based Adapters: TypesRealized in distributed objects, e.g., EJB, DCOM, CORBA
I Synchronous: blocking method invocation
I Asynchronous: nonblocking (one-way) method invocation withcallbacks
I Deferred synchronous: (in CORBA) sender proceeds independently ofthe receiver, but only up to a point
Munindar P. Singh (NCSU) Electronic Commerce Technologies 81 / 303
Architecture in IT Middleware
Invocation-Based Adapters: ExecutionExecution is best effort: application must detect any problems
I At most onceI More than once is
I OK for idempotent operationsI Not OK otherwise: application must check
Munindar P. Singh (NCSU) Electronic Commerce Technologies 82 / 303
Architecture in IT Middleware
Message-Oriented Middleware
I Main varietiesI Queues: point to point, support posting and reading messagesI Topics: logical multicasts, support publishing and subscribing to
application-specific topics; thus more flexible than queues
I Can offer reliability guarantees of delivery or failure notification tosender
I Analogous to store and forward networksI Usually implemented over databases
Munindar P. Singh (NCSU) Electronic Commerce Technologies 83 / 303
Architecture in IT Middleware
A standardized callback for messages
I Accessed only via messages, not invoked
I No need for specialized interfaces, such as home, remote, . . .I Easy interface to implement: onMessage()
I Programmer defines what actions to take on receipt of a messageI Limited message typing
I Stateless: thus no conversations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 84 / 303
Architecture in IT Middleware
Peer-to-Peer ComputingTraditionally, hard coded into specific applications, such as file sharing
I Symmetric client-server: (callbacks) each party can be the client ofthe other
I Asynchrony: while the request-response paradigm corresponds to pull,asynchronous communication corresponds to push
I Generally to place the entire intelligence on the server (pushing) side
I Federation of equals: (business partners) when the participants canenact the protocols they like
I Business protocols being defined in terms of business interactions, notlow-level messaging
Munindar P. Singh (NCSU) Electronic Commerce Technologies 85 / 303
Architecture in IT Middleware
Enterprise Best Practices
I Enterprise Service BusI Builds on top of messagingI Provide orchestration as a way to realize business processesI Shields programmer from transport and location considerations
I Asset repositoryI Building on directory services
I Data dictionaryI Evolving into one based on ontologies
Munindar P. Singh (NCSU) Electronic Commerce Technologies 86 / 303
Architecture in IT Deployment Architecture
Outline
Challenges of Electronic Business
Architecture in ITEnterprise ArchitectureTiered ArchitectureWeb ArchitectureMiddlewareDeployment Architecture
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 87 / 303
Architecture in IT Deployment Architecture
Deployment Architecture: Web Server
I Frontend
I Supports HTTP operations
I Usually multithreaded
Munindar P. Singh (NCSU) Electronic Commerce Technologies 88 / 303
Architecture in IT Deployment Architecture
Deployment Architecture: Application Server
I Mediates interactions between browsers and backend databases: runscomputations, invoking DB transactions as needed
I Provides a venue for the business logicI Different approaches (CGI, server scripts, servlets, Enterprise
JavaBeans) with tradeoffs inI Overhead: OS processes versus threadsI ScalabilityI Security
Munindar P. Singh (NCSU) Electronic Commerce Technologies 89 / 303
Architecture in IT Deployment Architecture
Application Server as an Architectural AbstractionSeparates business logic from infrastructure
I Load balancing
I Distribution and clustering
I Availability
I Logging and auditing
I Connection (and resource) pooling
I Security
Separate programming from administration roles
Munindar P. Singh (NCSU) Electronic Commerce Technologies 90 / 303
Architecture in IT Deployment Architecture
Deployment Architecture: Database Server
I Holds the data, ensuring its integrityI Manages transactions, providing
I Concurrency controlI Recovery
Transaction monitors can manage transactions across database systems,but within the same administrative domain
Munindar P. Singh (NCSU) Electronic Commerce Technologies 91 / 303
Architecture in IT Deployment Architecture
Data Center Architecture
I Demilitarized zone (DMZ)I External routerI Load balancer
I Firewall: only the router can contact the internal networkI Internal networkI Web serversI Application serversI Database servers
Munindar P. Singh (NCSU) Electronic Commerce Technologies 92 / 303
Contracts and Governance
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 93 / 303
Contracts and Governance
ContractBinding agreement specifying each party’s expectations on the others
I A contract structures interactions among autonomous partiesI People and corporationsI Governmental agencies
I Unlike a contract in programming
I Key questions: how to create, modify, perform, or monitor a contract
Munindar P. Singh (NCSU) Electronic Commerce Technologies 94 / 303
Contracts and Governance
Motivation for Contracts
I Provide a basis for service agreementsI Crucial in open environments
I Emphasize behavior: observable by othersI Constrain behavior: limit autonomyI Except where needed, generally disregard internal implementations,
thus facilitating heterogeneity
Munindar P. Singh (NCSU) Electronic Commerce Technologies 95 / 303
Contracts and Governance
What is a Contract?A description of business-level interactions
A reusable description of an interaction understood to preserve theparticipants’ autonomy
I Analogous to an abstract class or interface for objects
I Specifies well-defined rolesI Specifies messages among the roles and how they affect interaction
stateI Capturing commitments on a business partner playing a roleI Setting local policies while complying with a protocol
I Stored in a repository, i.e., as an asset or resource in its own right
I Refined and composed for implementation
Munindar P. Singh (NCSU) Electronic Commerce Technologies 96 / 303
Contracts and Governance
Importance of GovernanceStakeholders using resources to best serve their needs
I Share resources in a controlled manner
I Configure and reconfigure
I Enable unanticipated uses for resources
I Administer respecting human organizational needs
In particular, stakeholders administer themselves
Munindar P. Singh (NCSU) Electronic Commerce Technologies 97 / 303
Contracts and Governance
Governance versus ManagementAlternative approaches to administration
I Management: by superiors of subordinatesI Control over managed resourcesI Necessary but not sufficient
I Governance: by autonomous equals of themselvesI Collaborative decision-making among stakeholdersI Share resources flexibly, enabling unanticipated usesI Administer respecting human organizational needs
I Governance is what is needed, yet metaphors and approaches dealwith management
Thus, governance is hidden: manual via out-of-band communications
Munindar P. Singh (NCSU) Electronic Commerce Technologies 98 / 303
Contracts and Governance
Difficulty of GovernanceIndependence of stakeholders motivates high-level normative descriptions
I Autonomy: Stakeholders behave independently, constrained only bytheir agreements
I Heterogeneity: Stakeholders are independently constructed,constrained only by interface descriptions
I Dynamism: The set of stakeholders and their mutual relationshipsmay change continually
Munindar P. Singh (NCSU) Electronic Commerce Technologies 99 / 303
Contracts and Governance
Understanding GovernancePhilosophy
Governance is about how stakeholders administer their resources
I Focus on stakeholders
I Focus on interactions among stakeholders, framed as normativerelationships
I Focus on policies (capture autonomy)
I Focus on where the policies apply (where each party acts)
I Focus on perspicuous specification of policies
Munindar P. Singh (NCSU) Electronic Commerce Technologies 100 / 303
Contracts and Governance
Applying Contracts in IT AdministrationGovernance of service engagements
I Currently, humans achieve governance manually
I Low productivityI Poor scalability to fine-grained, real time governance decisionsI Hidden, implicit considerations yield low confidence in correctness and
poor maintainability
I Can we address governance through contracts?I Applied commonly for external services: SLAs generally, cloud servicesI Apply within Org as well
Munindar P. Singh (NCSU) Electronic Commerce Technologies 101 / 303
Contracts and Governance
Approach: Contracts and PoliciesBoth are centered on interaction, but . . .
I Contracts are modules of abstraction
I Policies are inherently private
I Policies lead each party to adopt a contract and decide whether andhow to act given a contract
I Methodologically, we advocate going top downI Identify contractsI Identify policy points in a contractI Thus improving modularity and reusability
Munindar P. Singh (NCSU) Electronic Commerce Technologies 102 / 303
Contracts and Governance Contracts Conceptually
OutlineChallenges of Electronic Business
Architecture in IT
Contracts and GovernanceContracts Conceptually
CommitmentsOrganizational ConceptsModeling EngagementsPulling Concepts Together
PolicyCase Study: OOI
XML Concepts and Techniques
XML Modeling and Storage
Summary and DirectionsMunindar P. Singh (NCSU) Electronic Commerce Technologies 103 / 303
Contracts and Governance Contracts Conceptually
Prerequisites for Realizing Contracts
I Formal, computational representations
I Determining (by each party) how to act
I Monitoring, especially locally by each party
I Judging compliance
I Enforcing contracts
I Dealing with legacy contracts
I Maintaining (includes creating) contracts
I Analyzing contracts, whether to adopt or enact
Munindar P. Singh (NCSU) Electronic Commerce Technologies 104 / 303
Contracts and Governance Contracts Conceptually
HypothesisGovernance is a basis for understanding contracts even outside of IT
I Each contract is governedI Reify organization into an Org, where
I Members are stakeholdersI The Org itself is a stakeholderI The Org provides the context of the contract
I The Org handlesI IdentityI Enrollment: who can contractI Enforcement
I Each member handlesI How to act: policiesI Where to monitorI Whether to escalate
Munindar P. Singh (NCSU) Electronic Commerce Technologies 105 / 303
Contracts and Governance Contracts Conceptually
Commitments as Elements of a ContractExpress meanings of interactions
I Are atoms of contractual relationshipsI Enable correctness checking of contractsI Yield precise meanings and verifiability
Munindar P. Singh (NCSU) Electronic Commerce Technologies 106 / 303
Contracts and Governance Contracts Conceptually
Commitment Life Cycle (and Patterns)C(debtor, creditor, antecedent, consequent)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 107 / 303
Contracts and Governance Contracts Conceptually
Commitment Operations
I create(C(x, y, p, q)) establishes the commitment
I detach(C(x, y, p, q)) turns it into a base commitment
I discharge(C(x, y, p, q)) satisfies the commitment
I cancel(C(x, y, p, q)) cancels the commitment
I release(C(x, y, p, q)) releases the debtor from the commitmentI delegate(z, C(x, y, p, q)) replaces x by z as the debtor
I x remains ultimately responsible (in our work)
I assign(w, C(x, y, p, q)) replaces y by w as the creditor
Munindar P. Singh (NCSU) Electronic Commerce Technologies 108 / 303
Contracts and Governance Contracts Conceptually
Example: Commitment Progression
C(Buyer, Seller, goods, pay)I If goods ∧ C(Buyer, Seller, goods, pay) Then
I C(Buyer, Seller, T, pay)
I If pay ∧ C(Buyer, Seller, T, pay) ThenI Satisfied
I If pay ∧ C(Buyer, Seller, goods, pay) ThenI Satisfied
Can be nested:C(Seller, Buyer, pay, C(Shipper, Buyer, T, deliverGoods))
Munindar P. Singh (NCSU) Electronic Commerce Technologies 109 / 303
Contracts and Governance Contracts Conceptually
A Real-Life Service Engagement (Repeated)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 110 / 303
Contracts and Governance Contracts Conceptually
Example Contractual Relationships (at Outset)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 111 / 303
Contracts and Governance Contracts Conceptually
Munindar P. Singh (NCSU) Electronic Commerce Technologies 112 / 303
Contracts and Governance Contracts Conceptually
Achieving Governance: Agents and OrgsPut collaboration center stage
I Agents represent the stakeholders: people and organizationsI Provide a locus for interaction
I Orgs are like institutions: have an identity and life time distinct fromtheir members; also modeled as agents
I Examples: NCSU, UNC System, . . .I Provide a locus for roles and authorizationsI Enforce behavioral constraints on members
I Their main hold over their members is the threat of expulsion
Munindar P. Singh (NCSU) Electronic Commerce Technologies 113 / 303
Contracts and Governance Contracts Conceptually
Duality of Contracts and Orgs
I A set of contracts define an OrgI Roles, with their qualifications, privileges, liabilities
I An Org provides the context for defining contracts
Munindar P. Singh (NCSU) Electronic Commerce Technologies 114 / 303
Contracts and Governance Contracts Conceptually
Governance Overview
Munindar P. Singh (NCSU) Electronic Commerce Technologies 115 / 303
Contracts and Governance Contracts Conceptually
ApproachBased on a conceptual model for governance
I Determine what attributes are subject to Identity Management
I Specify an execution architecture
I Specify interactive aspects building on the execution architecture
I Determine a core language for expressing governance structures,policies, and interactions
I Understand policy authoring needs
Munindar P. Singh (NCSU) Electronic Commerce Technologies 116 / 303
Contracts and Governance Contracts Conceptually
Governance Conceptually
Munindar P. Singh (NCSU) Electronic Commerce Technologies 117 / 303
Contracts and Governance Contracts Conceptually
Governance Operationally
Munindar P. Singh (NCSU) Electronic Commerce Technologies 118 / 303
Contracts and Governance Contracts Conceptually
Patterns for Delegate
Munindar P. Singh (NCSU) Electronic Commerce Technologies 119 / 303
Contracts and Governance Contracts Conceptually
Contextual Patterns: Penalize and RevertIntroducing an Org explicitly as the context
Munindar P. Singh (NCSU) Electronic Commerce Technologies 120 / 303
Contracts and Governance Contracts Conceptually
A Purchase Service EngagementDemonstrates incremental specification as a form of stepwise refinement
Munindar P. Singh (NCSU) Electronic Commerce Technologies 121 / 303
Contracts and Governance Contracts Conceptually
Contracts
Munindar P. Singh (NCSU) Electronic Commerce Technologies 122 / 303
Contracts and Governance Contracts Conceptually
Contracts Life CycleVazquez-Salceda et al.
Munindar P. Singh (NCSU) Electronic Commerce Technologies 123 / 303
Contracts and Governance Contracts Conceptually
Additional NormsDirectionality and context are key features
Permission versus prohibition and sanction
Munindar P. Singh (NCSU) Electronic Commerce Technologies 124 / 303
Contracts and Governance Contracts Conceptually
Norms and Facades
Normative Concept Subject’s Facade Object’s Facade
Commitment Liability PrivilegeAuthorization Privilege LiabilityPower Privilege LiabilityProhibition Liability PrivilegeSanction Liability Privilege
Munindar P. Singh (NCSU) Electronic Commerce Technologies 125 / 303
Contracts and Governance Contracts Conceptually
Norm Life Cycle: 1
Munindar P. Singh (NCSU) Electronic Commerce Technologies 126 / 303
Contracts and Governance Contracts Conceptually
Norm Life Cycle: 2Computing the substate of a terminated norm
If terminated in Thenant con Com Aut Pro San Pow
false false null null null null null
false true sat vio null null null
true false vio null sat null vio
true true sat sat vio sat sat
In the case of a power, a vio occurs upon the failure of an attempt to bringabout the consequent.
Munindar P. Singh (NCSU) Electronic Commerce Technologies 127 / 303
Contracts and Governance Contracts Conceptually
Contracts Vocabulary
Munindar P. Singh (NCSU) Electronic Commerce Technologies 128 / 303
Contracts and Governance Contracts Conceptually
States, Events, and ProcessesA Linguistic Perspective
I Verbs versus nounsI Verbs: states or events
I Can often lift events to states: maintaining maps to being in themaintenance phase
I Can identify events from state transitions
I Nouns: objectsI Count: discrete unitsI Mass: dense or continuous substances
Munindar P. Singh (NCSU) Electronic Commerce Technologies 129 / 303
Contracts and Governance Contracts Conceptually
StateHow things are
I What the user believes or knows
I What a business partner is committed to performing
I On
I Off
I Busy
I Idle
I Hibernating
Munindar P. Singh (NCSU) Electronic Commerce Technologies 130 / 303
Contracts and Governance Contracts Conceptually
TelicityFrom Greek “telos”
I TelicI Refers to goal-directed events
I AtelicI Refers to nondirected events
Munindar P. Singh (NCSU) Electronic Commerce Technologies 131 / 303
Contracts and Governance Contracts Conceptually
GradualityEvents naturally map over objects
I GradualI Proper parts of an event are of the same typeI Walking in the parkI Working on a project
Munindar P. Singh (NCSU) Electronic Commerce Technologies 132 / 303
Contracts and Governance Contracts Conceptually
AchievementImmediate and directed at a goal
I Opening a door
I Completing a phase
I Committing a database transaction
I Reaching a project milestone
Munindar P. Singh (NCSU) Electronic Commerce Technologies 133 / 303
Contracts and Governance Contracts Conceptually
ActivityOngoing (thus conceptually long-lived) and undirected
I Taking a walk
I Marketing a product
I Maintaining a software product
I Monitoring a network
I Tuning system performance
I Listening to a socket (for incoming requests)
I Searching for interesting tidbits
I Innovating
Munindar P. Singh (NCSU) Electronic Commerce Technologies 134 / 303
Contracts and Governance Contracts Conceptually
AccomplishmentLong-lived and directed toward a goal
I Building a house
I Gathering requirements
I Searching for specific information
I Finishing a project within a deadline
I Executing an algorithm
I Completing a business process
I Obtaining certification
Munindar P. Singh (NCSU) Electronic Commerce Technologies 135 / 303
Contracts and Governance Contracts Conceptually
SemelfactiveImmediate but not directed as such
I Sneeze
I Power failure
I Network failure
I Incoming bot attack
Munindar P. Singh (NCSU) Electronic Commerce Technologies 136 / 303
Contracts and Governance Policy
OutlineChallenges of Electronic Business
Architecture in IT
Contracts and GovernanceContracts Conceptually
CommitmentsOrganizational ConceptsModeling EngagementsPulling Concepts Together
PolicyCase Study: OOI
XML Concepts and Techniques
XML Modeling and Storage
Summary and DirectionsMunindar P. Singh (NCSU) Electronic Commerce Technologies 137 / 303
Contracts and Governance Policy
Challenges for Policies and Decision Making
I No unique locus: separate policies for each autonomous participant
I Dependence on business relationshipsI Complexity of modeling
I Specifying vocabulary pertinent to service engagementsI Determining where policy decisions apply
I Idea: Architecture for governance centered on interactions viewed asconversations
I Domain-specific policies: Incorporate monitoring and responding toevents
I Generic policies: Altering business relationships
Munindar P. Singh (NCSU) Electronic Commerce Technologies 138 / 303
Contracts and Governance Policy
Policy Model: TypesProvide a basis for achieving governance; hence must go beyond traditional access control
I Each policy can be understood in terms of its cause and its effectI Cause
I Reactive: triggered by a request from another stakeholderI Proactive: triggered by local observations
I EffectI Authorization of action to be taken on behalf of requesterI Enablement of action, which would otherwise not be takenI Obligation of action, which would now be performed
Munindar P. Singh (NCSU) Electronic Commerce Technologies 139 / 303
Contracts and Governance Policy
Munindar P. Singh (NCSU) Electronic Commerce Technologies 140 / 303
Contracts and Governance Policy
Policy Model: Information
Each policy relies upon certain information in order to produce a decisionI Attributes of the parties involved
I Qualifications, affiliations
I Attributes of the capabilities involvedI Interactions to be carried out upon resourcesI Collated as interaction types and resource types
I Attributes of the relationships among the parties involvedI Participations in different OrgsI Arrangements among institutions (captured as participations)I Ongoing conversations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 141 / 303
Contracts and Governance Case Study: OOI
OutlineChallenges of Electronic Business
Architecture in IT
Contracts and GovernanceContracts Conceptually
CommitmentsOrganizational ConceptsModeling EngagementsPulling Concepts Together
PolicyCase Study: OOI
XML Concepts and Techniques
XML Modeling and Storage
Summary and DirectionsMunindar P. Singh (NCSU) Electronic Commerce Technologies 142 / 303
Contracts and Governance Case Study: OOI
DoDAF and FriendsDepartment of Defense Architecture Framework
I A standardized way to organize an enterprise architecture
I Lists 26 views organized into four categoriesI Roughly, a software methodology
I How to capture requirements: user activitiesI How to develop solutions: meet performance criteriaI How to consider technical standards
I Best for large systems with lifetimes of decades
I Relates to MoDAF and TOGAF and to OMG’s MDA
Munindar P. Singh (NCSU) Electronic Commerce Technologies 143 / 303
Contracts and Governance Case Study: OOI
DoDAF View Sets
I All Views (AVs)I Two overall viewsI Include the terminology used in the remainder of the views
I Operational views (OVs)I SevenI Describe the working of the system as visible to its stakeholdersI Formalized using technical notations such as UML Activity Diagrams
and UML Message Sequence Diagrams
I Systems and services views (SVs)I SeventeenI Describe service interfaces, their performance and physical schemas,
and the functioning of the system close to an implementation level
I Technical standards views (TVs)I TwoI Relate the system to potentially relevant standards and technologies
Munindar P. Singh (NCSU) Electronic Commerce Technologies 144 / 303
Contracts and Governance Case Study: OOI
Methodology for Developing an Enterprise Architecture
I Identify stakeholdersI Understand the stakeholders’ requirements
I Begin with the customers
I Identify interactionsI Identify the business services as offered by the enterprise
I Business processesI Ongoing activities
I Identify components needed to support the above services.I OrganizationI ServicesI Data
Munindar P. Singh (NCSU) Electronic Commerce Technologies 145 / 303
Contracts and Governance Case Study: OOI
Ongoing StudiesOcean Observatories Initiative (OOI)
I Primary: Operational Activity Model (OV5) document describing theentire life cycle via several use cases
I Resources being createdI Resources being registered and publishedI Resources being commissioned and decommissionedI Several more . . .
I Secondary: OOI Concept of Operations document
Munindar P. Singh (NCSU) Electronic Commerce Technologies 146 / 303
Contracts and Governance Case Study: OOI
The OV5 Register Activity DiagramDeveloped by others
Munindar P. Singh (NCSU) Electronic Commerce Technologies 147 / 303
Contracts and Governance Case Study: OOI
What We Extract from the OV5 Register Activity
I RolesI Registrar (e.g., facility administrator)I Registrant (e.g., a researcher)
I Main interactionsI Registrant registers a new resource (e.g., a data stream) to make it
available to othersI Registrar advertises a registered resource
I Policy points for the registrarI Whether to accept the registrant’s requestI Whether to advertise a registered resource
Munindar P. Singh (NCSU) Electronic Commerce Technologies 148 / 303
Contracts and Governance Case Study: OOI
The OV5 Commission Activity DiagramDeveloped by others
Munindar P. Singh (NCSU) Electronic Commerce Technologies 149 / 303
Contracts and Governance Case Study: OOI
What We Extract from the OV5 Commission Activity
I RolesI Operator (e.g., a test facility or deployment engineer)I Provider (e.g., a researcher)
I Main interactionsI Provider requests the operator to certify a data stream from a sensorI Operator completes verification of deployment of a sensor that has
been requested for commissioning
I Policy point for the operatorI Whether to accept the provider’s request
I Policy point for the providerI Whether to proceed to validate the deployment
Munindar P. Singh (NCSU) Electronic Commerce Technologies 150 / 303
Contracts and Governance Case Study: OOI
Governance of AMQP Exchange SpaceHighlighting the business relationships
Munindar P. Singh (NCSU) Electronic Commerce Technologies 151 / 303
Contracts and Governance Case Study: OOI
Vocabulary Example for a Resource Sharing Community
Munindar P. Singh (NCSU) Electronic Commerce Technologies 152 / 303
Contracts and Governance Case Study: OOI
Governance of Community Affiliation Scenario
Munindar P. Singh (NCSU) Electronic Commerce Technologies 153 / 303
XML Concepts and Techniques
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 154 / 303
XML Concepts and Techniques XML Representation
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and TechniquesXML RepresentationXML Query and Manipulation
XPathXQueryXSLT
Programming with XML
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 155 / 303
XML Concepts and Techniques XML Representation
XML Representation
I Concepts
I Parsing and Validation
I Schemas
Munindar P. Singh (NCSU) Electronic Commerce Technologies 156 / 303
XML Concepts and Techniques XML Representation
What is Metadata?
Literally, data about dataI Description of data that captures some useful property regarding its
I Structure and meaningI Provenance: originsI Treatment as permitted or allowed: storage, representation, processing,
presentation, or sharing
I Markup is metadata pertaining to media artifacts (documents,images), generally specified for suitable parsable units
Munindar P. Singh (NCSU) Electronic Commerce Technologies 157 / 303
XML Concepts and Techniques XML Representation
Motivations for Metadata
Mediating information structure (surrogate for meaning) over time andspace
I Storage: extend life of information
I Interoperation for business
I Interoperation (and storage) for regulatory reasonsI General themes
I Make meaning of information explicitI Enable reuse across applications: repurposing (compare to
screen-scraping)I Enable better tools to improve productivity
Reduce need for detailed prior agreements
Munindar P. Singh (NCSU) Electronic Commerce Technologies 158 / 303
XML Concepts and Techniques XML Representation
Markup History
How much prior agreement do you need?
I No markup: significant prior agreement
I Comma Separated Values (CSV): no nesting
I Ad hoc tags
I SGML (Standard Generalized Markup L): complex, few reliable tools; usedfor document management
I HTML (HyperText ML): simplistic, fixed, unprincipled vocabulary that mixesstructure and display
I XML (eXtensible ML): simple, yet extensible subset of SGML to capturecustom vocabularies
I Machine processibleI Comprehensible to people: easier debugging
Munindar P. Singh (NCSU) Electronic Commerce Technologies 159 / 303
XML Concepts and Techniques XML Representation
Uses of XML
Supporting arms-length relationships
I Exchanging information across software components, even within anadministrative domain
I Storing information in nonproprietary formatI Representing semistructured descriptions:
I Products, services, catalogsI ContractsI Queries, requests, invocations, responses (as in SOAP): basis for Web
services
Munindar P. Singh (NCSU) Electronic Commerce Technologies 160 / 303
XML Concepts and Techniques XML Representation
Example XML Document
<?xml v e r s i o n =”1.0”?> <!−− p r o c e s s i n g i n s t r u c t i o n −−><tope lem a t t r 0 =”f o o”> <!−− e x a c t l y one r o o t −−><sube lem a t t r 1 =”v1 ” a t t r 2 =”v2”>
O p t i o n a l t e x t (PCDATA) <!−− p a r s e d c h a r a c t e r data −−><subsube lem a t t r 1 =”v1 ” a t t r 2 =”v2”/>
</subelem><n u l l e l e m /><s h o r t e l e m a t t r 3 =”v3”/>
</topelem>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 161 / 303
XML Concepts and Techniques XML Representation
Exercise
Produce an example XML document corresponding to a directed graph
Munindar P. Singh (NCSU) Electronic Commerce Technologies 162 / 303
XML Concepts and Techniques XML Representation
Compare with Lisp
List processing language
I S-expressions
I Cons pairs: car and cdr
I Lists as nil-terminated s-expressions
I Arbitrary structures built from few primitives
I Untyped
I Easy parsing
I Regularity of structure encourages recursion
Munindar P. Singh (NCSU) Electronic Commerce Technologies 163 / 303
XML Concepts and Techniques XML Representation
Exercise
Produce an example XML document corresponding to
I An invoice from Locke Brothers for 100 units of door locks at $19.95,each ordered on 15 January and delivered to Custom Home Builders
I Factor in certified delivery via UPS for $200.00 on 18 January
I Factor in addresses and contact info for each party
I Factor in late payments
Munindar P. Singh (NCSU) Electronic Commerce Technologies 164 / 303
XML Concepts and Techniques XML Representation
Meaning in XML
I Relational DBMSs work for highly structured information, but rely oncolumn names for meaning
I Same problem in XML (reliance on names for meaning) but betterconnections to richer meaning representations
Munindar P. Singh (NCSU) Electronic Commerce Technologies 165 / 303
XML Concepts and Techniques XML Representation
XML Namespaces: 1
I Because XML supports custom vocabularies and interoperation, thereis a high risk of name collision
I A namespace is a collection of namesI Namespaces must be identical or disjoint
I Crucial to support independent development of vocabulariesI MAC addressesI Postal and telephone codesI Vehicle identification numbersI Domains as for the InternetI On the Web, use URIs for uniqueness
Munindar P. Singh (NCSU) Electronic Commerce Technologies 166 / 303
XML Concepts and Techniques XML Representation
XML Namespaces: 2
<!−− xml∗ i s r e s e r v e d −−><?xml v e r s i o n =”1.0”?><a r b i t : top xmlns=”a URI” <!−− d e f a u l t namespace −−>
xmlns : a r b i t =”h t t p : / / w h e r e v e r . i t . might . be / a r b i t−ns ”xmlns : random=”h t t p : / / a n o t h e r . one /random−ns”>
<a r b i t : aElem a t t r 1 =”v1 ” a t t r 2 =”v2”>O p t i o n a l t e x t (PCDATA)<a r b i t : bElem a t t r 1 =”v1 ” a t t r 2 =”v2”/>
</ a r b i t : aElem><random : s i m p l e e l e m /><random : aElem a t t r 3 =”v3”/><!−− compare a r b i t : aElem −−>
</ a r b i t : top>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 167 / 303
XML Concepts and Techniques XML Representation
Uniform Resource Identifier
I URIs are abstract
I What matters is their (purported) uniqueness
I URIs have no proper syntax per seI Kinds of URIs
I URLs, as in browsing: not used in standards any moreI URNs, which leave the mapping of names to locations up in the air
I Good design: the URI resource existsI Ideally, as a description of the resource in RDDLI Use a URL or URN
Munindar P. Singh (NCSU) Electronic Commerce Technologies 168 / 303
XML Concepts and Techniques XML Representation
RDDL
Resource Directory Description Language
I Meant to solve the problem that a URI may not have any realcontent, but people expect to see some (human readable) content
I Captures namespace description for peopleI XML SchemaI Text description
Munindar P. Singh (NCSU) Electronic Commerce Technologies 169 / 303
XML Concepts and Techniques XML Representation
Well-Formedness and Parsing
I An XML document maps to a parse tree (if well-formed; otherwisenot XML)
I Each element must end (exactly once): obvious nesting structure (oneroot)
I An attribute can have at most one occurrence within an element; anattribute’s value must be a quoted string
I Well-formed XML documents can be parsed
Munindar P. Singh (NCSU) Electronic Commerce Technologies 170 / 303
XML Concepts and Techniques XML Representation
XML InfoSet
A standardization of the low-level aspects of XML
I What an element looks like
I What an attribute looks like
I What comments and namespace references look like
I Ordering of attributes is irrelevant
I Representations of strings and characters
Primarily directed at tool vendors
Munindar P. Singh (NCSU) Electronic Commerce Technologies 171 / 303
XML Concepts and Techniques XML Representation
Elements Versus Attributes: 1
I Elements are essential for XML: structure and expressivenessI Have subelements and attributesI Can be repeatedI Loosely might correspond to independently existing entitiesI Can capture all there is to attributes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 172 / 303
XML Concepts and Techniques XML Representation
Elements Versus Attributes: 2
I Attributes are not essentialI End of the road: no subelements or attributesI Like text; restricted to string valuesI Guaranteed unique for each elementI Capture adjunct information about an elementI Great as references to elements
Good idea to use in such cases to improve readability
Munindar P. Singh (NCSU) Electronic Commerce Technologies 173 / 303
XML Concepts and Techniques XML Representation
Elements Versus Attributes: 3
< i n v o i c e ><p r i c e c u r r e n c y =’USD’>
1 9 . 9 5</ p r i c e >
</ i n v o i c e >
Or< i n v o i c e amount = ’19.95 ’ c u r r e n c y =’USD’/>
Or even< i n v o i c e amount=’USD 19.95 ’/>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 174 / 303
XML Concepts and Techniques XML Query and Manipulation
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and TechniquesXML RepresentationXML Query and Manipulation
XPathXQueryXSLT
Programming with XML
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 175 / 303
XML Concepts and Techniques XML Query and Manipulation
XML Query and Manipulation
Main XML query and manipulation languages include
I XPath
I XQuery
I XSLT
I SQL/XML
Munindar P. Singh (NCSU) Electronic Commerce Technologies 176 / 303
XML Concepts and Techniques XML Query and Manipulation
Metaphors for Handling XML: 1
How we conceptualize XML documents determines our approach forhandling them
I Text: an XML document is textI Ignore any structure and perform simple pattern matches
I Tags: an XML document is text interspersed with tagsI Treat each tag as an “event” during reading a document, as in SAX
(Simple API for XML)I Construct regular expressions as in screen scraping
Munindar P. Singh (NCSU) Electronic Commerce Technologies 177 / 303
XML Concepts and Techniques XML Query and Manipulation
Metaphors for Handling XML: 2
I Tree: an XML document is a treeI Walk the tree using DOM (Document Object Model)
I Template: an XML document has regular structureI Let XPath, XSLT, XQuery do the work
I Thought: an XML document represents an information modelI Access knowledge via RDF or OWL
Munindar P. Singh (NCSU) Electronic Commerce Technologies 178 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath
Used as part of XPointer, SQL/XML, XQuery, and XSLTI Models XML documents as trees with nodes
I ElementsI AttributesI Text (PCDATA)I CommentsI Root node: above root of document
Munindar P. Singh (NCSU) Electronic Commerce Technologies 179 / 303
XML Concepts and Techniques XML Query and Manipulation
Achtung!
I Parent in XPath is like parent as traditionally in computer scienceI Child in XPath is confusing:
I An attribute is not a child of its parentI Makes a difference for recursion (e.g., in XSLT apply-templates)
I Our terminology follows computer science:I e-children, a-children, t-childrenI Sets via et-, ta-, and so on
Munindar P. Singh (NCSU) Electronic Commerce Technologies 180 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Location Paths: 1
I Relative or absoluteI Reminiscent of file system paths, but much more subtle
I Name of an element to walk downI Leading /: rootI /: indicates walking down a treeI .: currently matched (context) nodeI ..: parent node
Munindar P. Singh (NCSU) Electronic Commerce Technologies 181 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Location Paths: 2
I @attr: to check existence or access value of the given attribute
I text(): extract the text
I comment(): extract the comment
I [ ]: generalized array accessors
I Variety of axes, discussed below
Munindar P. Singh (NCSU) Electronic Commerce Technologies 182 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Navigation
I Select children according to position, e.g., [j], where j could be 1. . . last()
I Descendant-or-self operator, //I .//elem finds all elems under the current nodeI //elem finds all elems in the document
I Wildcard, *:I collects e-children (subelements) of the node where it is applied, but
omits the t-childrenI @*: finds all attribute values
Munindar P. Singh (NCSU) Electronic Commerce Technologies 183 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Queries (Selection Conditions)
I Attributes: //Song[@genre=”jazz”]
I Text: //Song[starts-with(.//group, ”Led”)]
I Existence of attribute: //Song[@genre]
I Existence of subelement: //Song[group]
I Boolean operators: and, not, or
I Set operator: union (—), analogous to choice
I Arithmetic operators: >, <, . . .
I String functions: contains(), concat(), length(), starts-with(),ends-with()
I distinct-values()
I Aggregates: sum(), count()
Munindar P. Singh (NCSU) Electronic Commerce Technologies 184 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Axes: 1
Axes are addressable node sets based on the document tree and thecurrent node
I Axes facilitate navigation of a tree
I Several are defined
I Mostly straightforward but some of them order the nodes as thereverse of others
I Some captured via special notationI current, child, parent, attribute, . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 185 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Axes: 2
I preceding: nodes that precede the start of the context node (notancestors, attributes, namespace nodes)
I following: nodes that follow the end of the context node (notdescendants, attributes, namespace nodes)
I preceding-sibling: preceding nodes that are children of the sameparent, in reverse document order
I following-sibling: following nodes that are children of the sameparent
Munindar P. Singh (NCSU) Electronic Commerce Technologies 186 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Axes: 3
I ancestor: proper ancestors, i.e., element nodes (other than thecontext node) that contain the context node, in reverse documentorder
I descendant: proper descendants
I ancestor-or-self: ancestors, including self (if it matches the nextcondition)
I descendant-or-self: descendants, including self (if it matches thenext condition)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 187 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Axes: 4
I Longer syntax: child::SongI Some captured via special notation
I self::*:I child::node(): node() matches all nodesI preceding::*I descendant::text()I ancestor::SongI descendant-or-self::node(), which abbreviates to //I Compare /descendant-or-self::Song[1] (first descendant Song) and
//Song[1] (first Songs (children of their parents))
Munindar P. Singh (NCSU) Electronic Commerce Technologies 188 / 303
XML Concepts and Techniques XML Query and Manipulation
XPath Axes: 5
I Each axis has a principal node kindI attribute: attributeI namespace: namespaceI All other axes: element
I * matches whatever is the principal node kind of the current axis
I node() matches all nodes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 189 / 303
XML Concepts and Techniques XML Query and Manipulation
XPointer
Enables pointing to specific parts of documents
I Combines XPath with URLs
I URL to get to a document; XPath to walk down the documentI Can be used to formulate queries, e.g.,
I Song-URL#xpointer(//Song[@genre=”jazz”])I The part after # is a fragment identifier
I Fine-grained addressability enhances the Web architecture
High-level “conceptual” identification of node sets
Munindar P. Singh (NCSU) Electronic Commerce Technologies 190 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery
I The official query language for XML, now a W3C recommendation, asversion 1.0
I Given a non-XML syntax, easier on the human eye than XML
I An XML rendition, XqueryX, is in the works
Munindar P. Singh (NCSU) Electronic Commerce Technologies 191 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Basic Paradigm
The basic paradigm mimics the SQL (SELECT–FROM–WHERE) clausef o r $x i n doc ( ’ q2 . xml ’ ) // Songwhere $x / @lg = ’ en ’r e t u r n<E n g l i s h−Sgr name=’{$x / Sgr /@name} ’ t i =’{$x / @ t i } ’/>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 192 / 303
XML Concepts and Techniques XML Query and Manipulation
FLWOR Expressions
Pronounced “flower”
I For: iterative binding of variables over range of values
I Let: one shot binding of variables over vector of values
I Where (optional)
I Order by (sort: optional)
I Return (required)
Need at least one of for or let
Munindar P. Singh (NCSU) Electronic Commerce Technologies 193 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery For Clause
The for clause
I Introduces one or more variables
I Generates possible bindings for each variableI Acts as a mapping functor or iterator
I In essence, all possible combinations of bindings are generated: like aCartesian product in relational algebra
I The bindings form an ordered list
Munindar P. Singh (NCSU) Electronic Commerce Technologies 194 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Where Clause
The where clause
I Selects the combinations of bindings that are desired
I Behaves like the where clause in SQL, in essence producing a joinbased on the Cartesian product
Munindar P. Singh (NCSU) Electronic Commerce Technologies 195 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Return Clause
The return clause
I Specifies what node-sets are returned based on the selectedcombinations of bindings
Munindar P. Singh (NCSU) Electronic Commerce Technologies 196 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Let Clause
The let clause
I Like for, introduces one or more variables
I Like for, generates possible bindings for each variable
I Unlike for, generates the bindings as a list in one shot (no iteration)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 197 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Order By Clause
The order by clause
I Specifies how the vector of variable bindings is to be sorted before thereturn clause
I Sorting expressions can be nested by separating them with commasI Variants allow specifying
I descending or ascending (default)I empty greatest or empty least to accommodate empty elementsI stable sorts: stable order byI collations: order by $t collation collation-URI: (obscure, so skip)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 198 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Positional Variables
The for clause can be enhanced with a positional variable
I A positional variable captures the position of the main variable in thegiven for clause with respect to the expression from which the mainvariable is generated
I Introduce a positional variable via the at $var construct
Munindar P. Singh (NCSU) Electronic Commerce Technologies 199 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Declarations
The declare clause specifies things likeI Namespaces: declare namespace pref=’value’
I Predefined prefixes include XML, XML Schema, XMLSchema-Instance, XPath, and local
I Settings: declare boundary-space preserve (or strip)
I Default collation: a URI to be used for collation when no collation isspecified
Munindar P. Singh (NCSU) Electronic Commerce Technologies 200 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Quantification: 1
I Two quantifiers some and every
I Each quantifier expression evaluates to true or false
I Each quantifier introduces a bound variable, analogous to for
f o r $x i n . . .where some $y i n . . .s a t i s f i e s $y . . . $xr e t u r n . . .
Here the second $x refers to the same variable as the first
Munindar P. Singh (NCSU) Electronic Commerce Technologies 201 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Quantification: 2
A typical useful quantified expression would use variables that wereintroduced outside of its scope
I The order of evaluation is implementation-dependent: enablesoptimization
I If some bindings produce errors, this can matter
I some: trivially false if no variable bindings are found that satisfy it
I every: trivially true if no variable bindings are found
Munindar P. Singh (NCSU) Electronic Commerce Technologies 202 / 303
XML Concepts and Techniques XML Query and Manipulation
Variables: Scoping, Bound, and Free
for, let, some, and every introduce variables
I The visibility variable follows typical scoping rulesI A variable referenced within a scope is
I Bound if it is declared within the scopeI Free if it not declared within the scope
f o r $x i n . . .where some $x i n . . .s a t i s f i e s . . .r e t u r n . . .
Here the two $x refer to different variables
Munindar P. Singh (NCSU) Electronic Commerce Technologies 203 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Conditionals
Like a classical if-then-else clause
I The else is not optional
I Empty sequences or node sets, written ( ), indicate that nothing isreturned
Munindar P. Singh (NCSU) Electronic Commerce Technologies 204 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Constructors
Braces { } to delimit expressions that are evaluated to generate thecontent to be included; analogous to macros
I document { }: to create a document node with the specified contentsI element { } { }: to create an element
I element foo { ’bar’ }: creates ¡foo¿Bar¡/foo¿I element { ’foo’ } { ’bar’ }: also evaluates the name expression
I attribute { } { }: likewise
I text { body}: simpler, because anonymous
Munindar P. Singh (NCSU) Electronic Commerce Technologies 205 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery Effective Boolean Value
Analogous to Lisp, a general value can be treated as if it were a Boolean
I A xs:boolean value maps to itself
I An empty sequence maps to false
I A sequence whose first member is a node maps to true
I A numeric that is 0 or NaN maps to false, else to true
I An empty string maps to false, others to true
Munindar P. Singh (NCSU) Electronic Commerce Technologies 206 / 303
XML Concepts and Techniques XML Query and Manipulation
Defining Functions
d e c l a r e f u n c t i o n l o c a l : i t e m f t o p ( $t ){
l o c a l : i t e m f ( $t , ( ) )} ;
I Here local: is the namespace of the query
I The arguments are specified in parentheses
I All of XQuery may be used within the defining braces
I Such functions can be used in place of XPath expressions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 207 / 303
XML Concepts and Techniques XML Query and Manipulation
Functions with Types
d e c l a r e f u n c t i o n l o c a l : i t e m f t o p ( $t as e l e m en t ( ) )as e l e m e n t ( ) ∗
{l o c a l : i t e m f ( $t , ( ) )
} ;
I Return types as above
I Also possible for parameters, but ignore such for this course
Munindar P. Singh (NCSU) Electronic Commerce Technologies 208 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT
A programming language with a functional flavor
I Specifies (stylesheet) transforms from documents to documents
I Can be included in a document (best not to)<?xml v e r s i o n =”1.0”?><?xml−s t y l e s h e e t t y p e=” t e x t / x s l ”
h r e f =”URL−to−x s l−s h e e t ”?><main−e lement>
. . .</main−e lement>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 209 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery versus XSLT: 1
Competitors in some ways, but
I Share a basis in XPath
I Consequently share the same data model
I Same type systems (in the type-sensitive versions)
I XSLT got out first and has a sizable following, but XQuery has strongbacking among vendors and researchers
Munindar P. Singh (NCSU) Electronic Commerce Technologies 210 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery versus XSLT: 2
I XQuery is geared for querying databasesI Supported by major relational DBMS vendors in their XML offeringsI Supported by native XML DBMSsI Offers superior coverage of processing joinsI Is more logical (like SQL) and potentially more optimizable
I XSLT is geared for transforming documentsI Is functional rather than declarativeI Based on template matching
Munindar P. Singh (NCSU) Electronic Commerce Technologies 211 / 303
XML Concepts and Techniques XML Query and Manipulation
XQuery versus XSLT: 3
There is a bit of an arms race between themI Types
I XSLT 1.0 didn’t support typesI XQuery 1.0 doesI XSLT 2.0 does too
I XQuery presumably will be enhanced with capabilities to makeupdates, but XSLT could too
Munindar P. Singh (NCSU) Electronic Commerce Technologies 212 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Stylesheets
A programming language that follows XML syntax
I Use the XSLT namespace (conventionally abbreviated xsl)I Includes a large number of primitives, especially:
I (deep copy)I (shallow copy)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 213 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Templates: 1A pattern to specify where the given transform should apply: an XPath expression
IIIII Match only the root<x s l : t e m p l a t e match=”/”>
. . .</ x s l : template>
I Example: Duplicate text in an element<x s l : t e m p l a t e match=” t e x t ( ) ”><x s l : v a l u e−o f s e l e c t = ’. ’/><x s l : v a l u e−o f s e l e c t = ’. ’/>
</ x s l : template>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 214 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Templates: 2
I If no pattern is specified, apply recursively on et-children via<x s l : app ly−t e m p l a t e s/>
I By default, if no other template matches, recursively apply toet-children of current node (ignores attributes) and to root:<x s l : t e m p l a t e match=”∗|/”><x s l : app ly−t e m p l a t e s/>
</ x s l : template>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 215 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Templates: 3
I Copy text node by default
I Use an empty template to override the default:<x s l : t e m p l a t e match=”X”/><!−− X = d e s i r e d p a t t e r n −−>
Confine ourselves to the examples discussed in class (ignore explicitpriorities, for example)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 216 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Templates: 4
I Templates can be namedI Templates can have parameters
I Values for parameters are supplied at invocationI Empty node sets by defaultI Additional parameters are ignored
Munindar P. Singh (NCSU) Electronic Commerce Technologies 217 / 303
XML Concepts and Techniques XML Query and Manipulation
XSLT Variables
I Explicitly declared
I Values are node sets
I Convenient way to document templates
Munindar P. Singh (NCSU) Electronic Commerce Technologies 218 / 303
XML Concepts and Techniques Programming with XML
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and TechniquesXML RepresentationXML Query and Manipulation
XPathXQueryXSLT
Programming with XML
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 219 / 303
XML Concepts and Techniques Programming with XML
Document Object Model (DOM)
Basis for parsing XML, which provides a node-labeled tree in its API
I Conceptually simple: traverse by requesting element, its attributevalues, and its children
I Processing program reflects document structure, as in recursivedescent
I Can edit documents
I Inefficient for large documents: parses them first entirely even if atiny part is needed
I Can validate with respect to a schema
Munindar P. Singh (NCSU) Electronic Commerce Technologies 220 / 303
XML Concepts and Techniques Programming with XML
DOM Example
DOMParser p = new DOMParser ( ) ;p . p a r s e (” f i l e n a m e ”) ;Document d = p . getDocument ( )Element s = d . getDocumentElement ( ) ;N o d e L i s t l = s . getElementsByTagName (” member ”) ;Element m = ( Element ) l . i tem ( 0 ) ;i n t code = m. g e t A t t r i b u t e (” code ”) ;N o d e L i s t k i d s = m. g e t C h i l d N o d e s ( ) ;Node k i d = k i d s . i tem ( 0 ) ;S t r i n g elemName = ( ( Element ) k i d ) . getTagName ( ) ; . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 221 / 303
XML Concepts and Techniques Programming with XML
Simple API for XML (SAX)
I Parser generates a sequence of events:I startElement, endElement, . . .
I Programmer implements these as callbacksI More control for the programmer
I Processing program does not necessarily reflect document structure
Munindar P. Singh (NCSU) Electronic Commerce Technologies 222 / 303
XML Concepts and Techniques Programming with XML
SAX Example: 1
c l a s s MemberProcess e x t e n d s D e f a u l t H a n d l e r {p u b l i c v o i d s t a r t E l e m e n t ( S t r i n g u r i , S t r i n g n ,
S t r i n g qName , A t t r i b u t e s a t t r s ) {i f ( n . e q u a l s (” member ”) ) code = a t t r s . g e t V a l u e (” code ”) ;i f ( n . e q u a l s (” p r o j e c t ”) ) i n P r o j e c t = t r u e ;b u f f e r . r e s e t ( ) ;}
. . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 223 / 303
XML Concepts and Techniques Programming with XML
SAX Example: 2
. . .
p u b l i c v o i d endElement ( S t r i n g u r i , S t r i n g n ,S t r i n g qName) {
i f ( n . e q u a l s (” p r o j e c t ”) ) i n P r o j e c t = f a l s e ;i f ( n . e q u a l s (” member ”) && ! i n P r o j e c t )
. . . do someth ing . . .}
}
Munindar P. Singh (NCSU) Electronic Commerce Technologies 224 / 303
XML Concepts and Techniques Programming with XML
SAX Filters
A component that mediates between an XMLReader (parser) and a client
I A filter would present a modified set of events to the clientI Typical uses:
I Make minor modifications to the structureI Search for patterns efficiently
I What kinds of patterns, though?
I Ideally modularize treatment of different event patterns
I In general, a filter can alter the structure of the document
Munindar P. Singh (NCSU) Electronic Commerce Technologies 225 / 303
XML Concepts and Techniques Programming with XML
Programming with XML
I LimitationsI Difficult to construct and maintain documentsI Internal structures are cumbersome; hence the criticisms of DOM
parsers
I Emerging approaches provide superior binding from XML toI Programming languagesI Relational databases
I Check pull-based versus push-based parsers
Munindar P. Singh (NCSU) Electronic Commerce Technologies 226 / 303
XML Modeling and Storage
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 227 / 303
XML Modeling and Storage
XML Modeling and Storage
The major aspects of storing XML include
I Concepts: Data and Document Centrism
I Storage
I Mapping to relational schemas
I SQL/XML
Munindar P. Singh (NCSU) Electronic Commerce Technologies 228 / 303
XML Modeling and Storage
Modern Information Systems
I Three legs of modern software systemsI Documents: as in XMLI Tuples: as in the information stored in relational databasesI Objects: as in programming languages
I A lot of effort goes into managing translations among these at thelevel of programming
I But deeper challenges remain . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 229 / 303
XML Modeling and Storage
Data-Centric View: 1
< r e l a t i o n name=’ Student ’><t u p l e><a t t r 1 >V11</a t t r 1 >
. . .<a t t r n>V1n</a t t r n>
</t u p l e>. . .
</ r e l a t i o n >
I Extract and store via mapping to DB model
I Regular, homogeneous structure
Munindar P. Singh (NCSU) Electronic Commerce Technologies 230 / 303
XML Modeling and Storage
Data-Centric View: 2
I Ideally, no mixed content: an element contains text or subelements,not both
I Any mixed content would be templatic, i.e.,I Generated from a database via suitable transformationsI Generated via a form that a user or an application fills out
I Order among siblings likely irrelevant (as is order among relationalcolumns)
Expensive if documents are repeatedly parsed and instantiated
Munindar P. Singh (NCSU) Electronic Commerce Technologies 231 / 303
XML Modeling and Storage
Document-Centric View
I Irregular: doesn’t map well to a relation
I Heterogeneous data
I Depending on entire doc for application-specific meaning
Munindar P. Singh (NCSU) Electronic Commerce Technologies 232 / 303
XML Modeling and Storage
Data- vs Document-Centric Views
I Data-centric: data is the main thingI XML simply renders the data for transportI Store as dataI Convert to/from XML as neededI The structure is important
I Document-centric: documents are the main thingI Documents are complex (e.g., design documents) and irregularI Store documents whereverI Use DBMS where it facilitates performing important searches
Munindar P. Singh (NCSU) Electronic Commerce Technologies 233 / 303
XML Modeling and Storage
Storing Documents in Databases
I Use character large objects (CLOBs) within DB: searchable only astext
I Store paths to external files containing docsI Simple, but no support for integrity
I Use some structured elements for easy search as well as unstructuredclobs or files
I Heterogeneity complicates mappings to typed OO programminglanguages
Storing documents in their entirety may sometimes be necessary forexternal reasons, such as regulatory compliance
Munindar P. Singh (NCSU) Electronic Commerce Technologies 234 / 303
XML Modeling and Storage
Database Features
I Storage: schema definition language
I Querying: query language
I Transactions: concurrency
I Recovery
Munindar P. Singh (NCSU) Electronic Commerce Technologies 235 / 303
XML Modeling and Storage
Potential DBMS Types for XML: 1
I Object-orientedI Nice structureI Intellectual basis of many XML concepts, including schema
representations and path expressionsI Not highly popular in standalone products
I RelationalI Limited structuring ability (1NF: each cell is atomic)I Extremely popularI Well optimized for flat queries
Munindar P. Singh (NCSU) Electronic Commerce Technologies 236 / 303
XML Modeling and Storage
Potential DBMS Types for XML: 2
I Object relational: hybrids of aboveI Not highly popular in standalone products
I Custom XML stores or native XML databasesI Emerging ideas: may lack core database features (e.g., recovery, . . . )I Enable fancier content management systemsI Leading open source products:
I Apache Xindice (server; XPath)I Berkeley DB XML (libraries; XQuery)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 237 / 303
XML Modeling and Storage SQL/XML
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and StorageSQL/XMLXML and Relational DatabasesXML SchemaXML Keys
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 238 / 303
XML Modeling and Storage SQL/XML
Quick Look at SQLStructured Query Language
I Data Definition Language: CREATE TABLE
I Data Manipulation Language: SELECT, INSERT, DELETE, UPDATE
I Basic paradigm for SELECTSELECT t1 . column−1, t1 . column−2 . . . tm . column−nFROM t a b l e −1 t1 , t a b l e−m tmWHERE t1 . column−3=t4 . column−4 AND . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 239 / 303
XML Modeling and Storage SQL/XML
SQL 2003Standardized by ANSI/ISO; next version after SQL 1999
I Includes SQL/XML: SQL extensions for XML (other aspects of SQL2003 are not relevant here)
I Distinct from Microsoft’s SQLXMLI SQL/XML is included in products
I By DBMS vendors, sometimes with different low-level details (MINUSversus EXCEPT)
I DBMS-independent products
Munindar P. Singh (NCSU) Electronic Commerce Technologies 240 / 303
XML Modeling and Storage SQL/XML
XML Type in SQL/XML
I A specialized data type for XML content; distinct from text
I Usable wherever an SQL data type is allowed: type of column,variable, tuple cell, and so on . . .
I Value rooted on the XML Root information item (described next)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 241 / 303
XML Modeling and Storage SQL/XML
XML Root Information Item: 1
Based on the XML InfoSet document information item, this can be an
I XML root (as in SQL/XML)
I XML element
I XML attribute
I XML parsed character data (text; aka PCDATA)
I XML namespace declaration
I XML processing instruction
I XML comment
And some more possibilities from the InfoSet . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 242 / 303
XML Modeling and Storage SQL/XML
XML Root Information Item: 2
I Unlike the XML InfoSet root (which allows exactly one childelement), this allows zero or more children
I Partial results need not be documents
I IS DOCUMENT: a predicate that checks if the argument XML valuehas a single root
I An XML value can beI NULL, as usual for SQLI An XML root item, including whatever it includes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 243 / 303
XML Modeling and Storage SQL/XML
SQL/XML Builtin Operators
I xmlparse(): maps a string (char, varchar, clob) to a value of typeXML (stripping whitespace by default)
I xmlserialize(): maps a value of type XML to a string
I xmlconcat(): combines values into a forest
I xmlroot(): create or modify the root node of an XML value
Munindar P. Singh (NCSU) Electronic Commerce Technologies 244 / 303
XML Modeling and Storage SQL/XML
SQL/XML Publishing Functions: 1
These are templates that go into a SELECT query; all with names thatbegin “xml”
I xmlelement(name ’Song’, ·)I Needs a value: an SQL column or expression or an attribute or an
elementI Yields a value (an element)I Can be nested, of course
I xmlattributes(column [AS cname], column [AS cname],. . . )I Creates XML attributes from the columnsI Inserts into the surrounding XML element
Munindar P. Singh (NCSU) Electronic Commerce Technologies 245 / 303
XML Modeling and Storage SQL/XML
SQL/XML Publishing Functions: 2
I xmlforest()I Creates XML elements from columnsI Analogous to a node-set in XPathI Must be placed within an element; otherwise not well-formed XML
I xmlagg(): combines a collection of rows, each with a single XMLvalue into a single forest
I xmlnamespaces()
I xmlcomment(): comment
I xmlpi(): processing instruction
Munindar P. Singh (NCSU) Electronic Commerce Technologies 246 / 303
XML Modeling and Storage SQL/XML
SQL/XML Example: 1
SELECT xmle lement (Name ’ Sgr ’ ,x m l a t t r i b u t e s ( z . s g r I d AS s t u d e n t−ID ) ,z . sgrName )
FROM S i n g e r zWHERE . . .
yields something like<Sgr s t u d e n t−ID=’s1 ’>
E a g l e s</Sgr>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 247 / 303
XML Modeling and Storage SQL/XML
SQL/XML Example: 2
SELECT xmle lement (Name ’ Sgr ’ ,x m l a t t r i b u t e s ( z . s g r I d AS s t u d e n t−ID ) ,z . sgrName ,xmle lement (Name ’ Song ’ , ’ Hote l ’ ) )
FROM S i n g e r zWHERE . . .
yields something like<Sgr s t u d e n t−ID=’s1 ’>
E a g l e s<Song>Hote l </Song>
</Sgr>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 248 / 303
XML Modeling and Storage SQL/XML
SQL/XML Mapping Rules
A number of low-level matters, which are conceptually trivial butcomplicate combining SQL and XML effectively; captured as mappingrules
I Lexical encodings in names and content
I Mapping datatypes in each direction, e.g., SQL date and XMLSchema date
I Mapping SQL tables, schemas, catalogs to and from XML
Munindar P. Singh (NCSU) Electronic Commerce Technologies 249 / 303
XML Modeling and Storage SQL/XML
Tool Support for SQL 2003
I Oracle 10g, IBM DB2, Sybase support it
I Apparently, Microsoft doesn’t or won’t [not sure]
I Oracle 9i release 2 supports similar constructs, but in proprietarysyntax
Munindar P. Singh (NCSU) Electronic Commerce Technologies 250 / 303
XML Modeling and Storage SQL/XML
Oracle 9i SQL/XML: 1
CREATE TABLE s i n g e r ( s g r I d VARCHAR2( 9 ) NOT NULL ,sgrName VARCHAR2( 1 5 ) NOT NULL ,s g r I n f o SYS .XMLTYPE NULL ,CONSTRAINT s i n g e r k e y
PRIMARY KEY ( s g r I d ) ) ;
Munindar P. Singh (NCSU) Electronic Commerce Technologies 251 / 303
XML Modeling and Storage SQL/XML
Oracle 9i SQL/XML: 2
INSERT INTO s i n g e r VALUES ( ’ Sgr −01 ’ , ’ Ea g le s ’ ,SYS .XMLTYPE. createXML ( ’< genre>rock </genre > ’) ) ;
INSERT INTO s i n g e r VALUES ( ’ Sgr −04 ’ , ’ B e a t l e s ’ ,SYS .XMLTYPE. createXML (
’< t r i v i a ><c o n v i c t i o n s >freedom</ c o n v i c t i o n s ><genre>rock </genre></ t r i v i a > ’) ) ;
SELECT z . sgrName , z . s g r I n f o . e x t r a c t ( ’/ g e n r e / t e x t ( ) ’ ). g e t C l o b V a l ( )
FROM s i n g e r z ;
Munindar P. Singh (NCSU) Electronic Commerce Technologies 252 / 303
XML Modeling and Storage SQL/XML
Oracle 9i SQL/XML: 3
SELECT z . sgrName , z . s g r I n f o . e x t r a c t ( ’// g e n r e / t e x t ( ) ’ ). g e t C l o b V a l ( )
FROM s i n g e r zWHERE z . s g r I n f o . e x t r a c t (
’// g e n r e / t e x t ( ) ’ ) . g e t S t r i n g V a l ( ) l i k e ’ r % ’;
SELECT z . sgrName , z . s g r I n f o . e x t r a c t ( ’/ g e n r e / t e x t ( ) ’ ). g e t C l o b V a l ( )
FROM s i n g e r zWHERE z . s g r I n f o . e x i s t s N o d e ( ’ / / genre ’ ) = 1 ;
Munindar P. Singh (NCSU) Electronic Commerce Technologies 253 / 303
XML Modeling and Storage SQL/XML
Oracle 9i SQL/XML: 4
SELECT SYS XMLAGG(SYS XMLGEN( z . sgrname ) ,SYS .XMLGENFORMATTYPE. c r e a t e f o r m a t ( ’ F o o L i s t ’ ) )
. g e t C l o b V a l ( )FROM s i n g e r zWHERE z . s g r I d I S NOT NULLGROUP BY z . sgrname ;
Munindar P. Singh (NCSU) Electronic Commerce Technologies 254 / 303
XML Modeling and Storage XML and Relational Databases
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and StorageSQL/XMLXML and Relational DatabasesXML SchemaXML Keys
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 255 / 303
XML Modeling and Storage XML and Relational Databases
XML to Relational Databases
I Using large objects
I Flatten XML structures
I Referring to external files
Recall that for a relational schema, its entire set of attributes is necessarilya superkey
Munindar P. Singh (NCSU) Electronic Commerce Technologies 256 / 303
XML Modeling and Storage XML and Relational Databases
Artificial Representation: Repetitious
Capturing an object hierarchy in a relation
I Imagine an artificial identifier for each nodeI Construct a relation with three main relational attributes or columns
I One column for the identifierI One column for the name of an attribute (i.e., element name)I One column for the value (assumes the value would fit into the same
relational type: potentially this could be CLOB or BLOB)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 257 / 303
XML Modeling and Storage XML and Relational Databases
Artificial Representation: Graph
Use four generic relations to represent a graphI Vertices:
I Element ID, Name
I ContentsI Element ID, Text, number (to allow multiple text nodes)
I AttributesI ID, Attribute name, Attribute value
I EdgesI Source ID, Target ID
Better typed than repetitious style because this has no nulls
Munindar P. Singh (NCSU) Electronic Commerce Technologies 258 / 303
XML Modeling and Storage XML and Relational Databases
Shallow Representation: 1
The “natural” approaches are based on tuple-generating elements (TGEs)I Choose one XML element type as the TGE
I TGE corresponds to a tupleI The key is based on an ID attribute or text of the TGE
I A relational attribute (column) for each subelement or attribute
I Easiest if there is an attribute for IDs and there are no other attributes
Munindar P. Singh (NCSU) Electronic Commerce Technologies 259 / 303
XML Modeling and Storage XML and Relational Databases
Shallow Representation: 2
I ConsequencesI Nulls for missing subelements can proliferateI Subelements with structure (subelements or attributes) aren’t
represented wellI Ancestors cannot be searched for
Munindar P. Singh (NCSU) Electronic Commerce Technologies 260 / 303
XML Modeling and Storage XML and Relational Databases
Deep Representation
Also called shredding an XML document
I Choose a TGE as beforeI A column for each descendant, except that
I Can skip wrapper elements (no text, only subelements), but mustreconstruct them to create an XML document
I ConsequencesI Nulls for missing subelementsI Lots of columns in a relationI Ancestors cannot be searched forI Loses structural information
Munindar P. Singh (NCSU) Electronic Commerce Technologies 261 / 303
XML Modeling and Storage XML and Relational Databases
Representing Ancestors
Ancestors are the elements that are above the scope of the given TGE
I Choose a TGE as before
I A column for each descendant as beforeI A column for each ancestor (that needs to be searched)
I Appropriate attributes or text fields to make the search worthwhile
I ConsequencesI Nulls for missing subelementsI Lots of columns in a relation
Munindar P. Singh (NCSU) Electronic Commerce Technologies 262 / 303
XML Modeling and Storage XML and Relational Databases
Generalized TGE
I Each element is a TGE, yielding a different relation
I A column for each terminal child: attribute or textI A column for each ancestor to capture the entire path from root to
this nodeI Must promote uniquifying content so that each TGE yields unique
tuples
I ConsequencesI Nulls for missing subelementsI Lots of relationsI Lots of columns in a relation
Munindar P. Singh (NCSU) Electronic Commerce Technologies 263 / 303
XML Modeling and Storage XML and Relational Databases
Variations in Structure
I Create separate relations for each variantI Consequences
I Lots of possible structures to storeI Queries would not be succinctI Acceptable only if we know in advance that the number of variants is
small and the data in each is substantial
Munindar P. Singh (NCSU) Electronic Commerce Technologies 264 / 303
XML Modeling and Storage XML and Relational Databases
Semistructured Representation
Create two (sets of) relations
I Specific part: one (or more) relations based on one of the naturalapproaches
I Generic part: one relation based on an artificial approach
Munindar P. Singh (NCSU) Electronic Commerce Technologies 265 / 303
XML Modeling and Storage XML and Relational Databases
Thoughtful Design
I The above approaches are not sensitive to the meaning andmotivation behind the XML structure
I Understand the XML structure via a conceptual model (in terms ofentities and relationships)
I Avoid unnecessary nesting in the XML structure, if possible
I Design a corresponding relational schema by hand
This is not always possible, though
Munindar P. Singh (NCSU) Electronic Commerce Technologies 266 / 303
XML Modeling and Storage XML and Relational Databases
Evaluation
How does the above work for data-centric and document-centric views?I Compare with respect to
I Document structureI Document “roundtripping” (compare &, &, #a39)I Normalization
I Are the documents unique?
I Are the documents unique up to “isomorphism”?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 267 / 303
XML Modeling and Storage XML and Relational Databases
Schema Evolution
A big problem for databases in practical settings
I For relational schemas, certain kinds of updates are simpler thanothers
I Can have consequences on optimization
I XML schemas can be evolved by using XSLT to map old data to newschema
Munindar P. Singh (NCSU) Electronic Commerce Technologies 268 / 303
XML Modeling and Storage XML and Relational Databases
From Relations to XML
Mapping a relation schema (set of relations plus functional dependencies)to an XML document
I Map relation R to an element RE with key or unique constraints
I Map column C of R to an attribute of RE or equivalently a childelement with just text
I Map relation S with a foreign key to R toI A child element SE of RE (omit foreign key content from SE ): works if
only one such RE for SE ; ORI An element SE that includes the foreign key content, and includes a
keyref to RE
XML key, unique, keyref as introduced in the XML Keys section
Munindar P. Singh (NCSU) Electronic Commerce Technologies 269 / 303
XML Modeling and Storage XML and Relational Databases
Creating XML from Legacy Sources
Often need to read in information from non-XML sourcesI From relational databases
I Easier because of structureI Supported by vendor tools
I From flat files, CSV documents, HTML Web pagesI Bit of a black art: lots of heuristicsI Tools based on regular expressions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 270 / 303
XML Modeling and Storage XML Schema
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and StorageSQL/XMLXML and Relational DatabasesXML SchemaXML Keys
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 271 / 303
XML Modeling and Storage XML Schema
XML Schema Motivated
Validation means verifying whether a document matches a given grammar(assumes well-formedness)
I Applications have an explicit or implicit syntax (i.e., grammar) fortheir particular elements and attributes
I Explicit is better have definitionsI Best to refer to definitions in separate documents
I When docs are produced by external software components or byhuman intervention, they should be validated
Munindar P. Singh (NCSU) Electronic Commerce Technologies 272 / 303
XML Modeling and Storage XML Schema
Specifying Document Grammars
Verifying whether a document matches a given grammarI Implicitly in the application
I Worst possible solution, because it is difficult to develop and maintain
I Explicit in a formal document; languages includeI Document Type Definition (DTD): in essence obsoleteI XML Schema: good and prevalentI Relax NG: (supposedly) better but not as prevalent
Munindar P. Singh (NCSU) Electronic Commerce Technologies 273 / 303
XML Modeling and Storage XML Schema
XML Schema
I Same syntax as regular XML documents
I Local scoping of subelement names
I Incorporates namespacesI (Data) Types
I Primitive (built-in): string, integer, float, date, ID (key), IDREF(foreign key), . . .
I simpleType constructors: list, unionI Restrictions: intervals, lengths, enumerations, regex patterns,I Flexible ordering of elements
I Key and referential integrity constraints
Munindar P. Singh (NCSU) Electronic Commerce Technologies 274 / 303
XML Modeling and Storage XML Schema
XML Schema: complexType
I Specifies types of elements with structure:I Must use a compositor if ≥ 1 subelementsI Subelements with typesI Min and max occurrences (default 1) of subelements
I Elements with text content are easyI EMPTY elements: easy
I Example?I Compare to nulls, later
Munindar P. Singh (NCSU) Electronic Commerce Technologies 275 / 303
XML Modeling and Storage XML Schema
XML Schema: Compositors
I Sequence: ordered list of subelementsI Can occur within other compositorsI Allows varying min and max occurrence
I All: unordered subelementsI Must occur directly below root elementI Max occurrence of each element is 1
I Choice: exclusive or: include one subelementI Can occur within other compositors
Munindar P. Singh (NCSU) Electronic Commerce Technologies 276 / 303
XML Modeling and Storage XML Schema
XML Schema: Main Namespaces
Part of the standardI xsd: http://www.w3.org/2001/XMLSchema
I Terms for defining schemas: schema, element, attribute, . . .I The schema element has an attribute targetNamespace
I xsi: http://www.w3.org/2001/XMLSchema-instanceI Terms for use in instances: schemaLocation,
noNamespaceSchemaLocation, nil, type
I targetNamespace: user-defined
Munindar P. Singh (NCSU) Electronic Commerce Technologies 277 / 303
XML Modeling and Storage XML Schema
XML Schema Instance Doc
<!−− Comment −−><Music xmlns=”h t t p : / / a . b . c /Muse”
xmlns : x s i =”t he s t a n d a r d−x s i ”x s i : schemaLocat ion=”schema−URI schema−l o c a t i o n−URL”>
<!−− N o t i c e s p a c e c h a r a c t e r i n above s t r i n g −−>. . .
</Music>
Define null values as<aElem x s i : n i l =” t r u e ”/>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 278 / 303
XML Modeling and Storage XML Schema
Null Value: 1
A special value, not in any domain, but combinable with any domain
I Need?I Possible meanings
I Not applicableI Unknown: missingI Questionable existenceI Absent (known but absent)
I Hazards of null values?
Munindar P. Singh (NCSU) Electronic Commerce Technologies 279 / 303
XML Modeling and Storage XML Schema
Null Value: 2
XML Schema enables developing custom null values for each domainI Create an arbitrary value that
I Matches the given data typeI Is not a valid value of the domain, however
I Design applications to understand specific restricted type
Munindar P. Singh (NCSU) Electronic Commerce Technologies 280 / 303
XML Modeling and Storage XML Schema
XML Schema Null
I ¡elem/¿ (equivalently ¡elem¿¡/elem¿) means that the elementcontains the empty string
I This is not null
I xsi defines the attribute nilI Used as ¡elem xsi:nil=”true”/¿ if elem is declared nillable (via
nillable=”true”)
Munindar P. Singh (NCSU) Electronic Commerce Technologies 281 / 303
XML Modeling and Storage XML Schema
XML Schema: Nillable
I An xsd:element declaration may state nillable=’true’I An instance of the element might state xsi:nil=”true”I The instance would be valid even if no content is present, even if
content is required by default
Munindar P. Singh (NCSU) Electronic Commerce Technologies 282 / 303
XML Modeling and Storage XML Schema
Placing Keys in Schemas
I Keys are associated with elements, not with types
I Thus the . in a key selector expression is bound
I Could have been (but are not) associated with types where the .could be bound to whichever element was an instance of the type
Munindar P. Singh (NCSU) Electronic Commerce Technologies 283 / 303
XML Modeling and Storage XML Schema
Creating XML Schema Docs: 1
Included into the same namespace as the including doc<xsd : schema xmlns : xsd=”the−s t a n d a r d−xsd ”
xsd : targetNamespace=”the−t a r g e t ”>< i n c l u d e xsd : schemaLocat ion=”part−one . xsd”/>< i n c l u d e xsd : schemaLocat ion=”part−two . xsd”/><!−− schemaLocat ion as i n xsd , not x s i −−>
</xsd : schema>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 284 / 303
XML Modeling and Storage XML Schema
Creating XML Schema Docs: 2
I Use import instead of includeI Imports may have different targetsI Included schemas have the same targetI Specify namespaces from which schemas are to be importedI Location of schemas not required and may be ignored if provided
Munindar P. Singh (NCSU) Electronic Commerce Technologies 285 / 303
XML Modeling and Storage XML Schema
Foreign Attributes in XML Schema
XML Schema elements allow attributes that are foreign, i.e., with anamespace other than the xsd namespace
I Must have an explicit namespace
I Can be used to insert any additional information, not interpreted by aprocessor
I Specific usage is with attributes from the xlink: namespace
<xsd : schema><xsd : e l e m e nt name=’ c o u r s e ’ t y p e =’cT ’
x l i n k : r o l e =’work ’ ncsu : o f f e r i n g =’ t r u e ’></xsd : schema>
Munindar P. Singh (NCSU) Electronic Commerce Technologies 286 / 303
XML Modeling and Storage XML Schema
XML Schema Style Guidelines: 1
I Flatten the structure of the schemaI Don’t nest declarations as you would a desired instance documentI Make sure that element names are not reusedI Unqualified attributes cannot be globalI If dealing with legacy documents with the same element names having
different meanings, place them in different namespaces where possible
I Use named types where appropriate
Munindar P. Singh (NCSU) Electronic Commerce Technologies 287 / 303
XML Modeling and Storage XML Schema
XML Schema Style Guidelines: 2
I Don’t have elements with mixed content
I Don’t have attribute values that need parsing
I Add unique IDs for information that may repeat
I Group information that may repeatI Emphasize commonalities and reuse
I Derive types from related typesI Create attribute groups
Munindar P. Singh (NCSU) Electronic Commerce Technologies 288 / 303
XML Modeling and Storage XML Schema
XML Schema Documentation
xsd:annotation
I Should be the first subelement, except for the whole schemaI Container for two mixed-content subelements
I xsd:documentation: for humansI xsd:appinfo: for machine-processible data
I Such as application-specific metadataI Possibly using the Dublin Core vocabulary, which describes library
content and other media
Munindar P. Singh (NCSU) Electronic Commerce Technologies 289 / 303
XML Modeling and Storage XML Keys
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and StorageSQL/XMLXML and Relational DatabasesXML SchemaXML Keys
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 290 / 303
XML Modeling and Storage XML Keys
Integrity Constraints in XML
I Entity: xsd:unique and xsd:key
I Referential: xsd:keyref
I Data type: XML Schema specifications
I Value: Solve custom queries using XPath or XQuery
Entity and referential constraints are based on XPath
Munindar P. Singh (NCSU) Electronic Commerce Technologies 291 / 303
XML Modeling and Storage XML Keys
XML Constraints: 1
Keys serve as generalized identifiers, and are captured via XML Schemaelements:
I Unique: candidate keyI The selected elements yield unique field tuples
I Key: primary key, which means candidate key plusI The tuples exist for each selected element
I Keyref: foreign keyI Each tuple of fields of a selected element corresponds to an element in
the referenced key
Munindar P. Singh (NCSU) Electronic Commerce Technologies 292 / 303
XML Modeling and Storage XML Keys
XML Constraints: 2
Two subelements built using restricted application of XPath from withinXML Schema
I Selector: specify a set of objects: this is the scope over whichuniqueness applies
I Field: specify what is unique for each member of the above set: thisis the identifier within the targeted scope
I Multiple fields are treated as ordered to produce a tuple of values foreach member of the set
I The order matters for matching keyref to key
Munindar P. Singh (NCSU) Electronic Commerce Technologies 293 / 303
XML Modeling and Storage XML Keys
Selector XPath Expression
A selector finds descendant elements of the context nodeI The sublanguage of XPath used allows
I Children via ./child or ./* or childI Descendants via .// (not within a path)I Choice via —
I The subset of XPath used does not allowI Parents or ancestorsI text()I AttributesI Fancy axes such as preceding, preceding-sibling, . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 294 / 303
XML Modeling and Storage XML Keys
Field XPath Expression
A field finds a unique descendant element (simple type only) or attributeof the context node
I The subset of XPath used allowsI Children via ./child or ./*I Descendants via .// (not within a path)I Choice via —I Attributes via @attribute or @*
I The subset of XPath used does not allowI Parents or ancestorsI text()I Fancy axes such as preceding, . . .
An element yields its text()
Munindar P. Singh (NCSU) Electronic Commerce Technologies 295 / 303
XML Modeling and Storage XML Keys
XML Foreign Keys
<k e y r e f name = ” . . . ” r e f e r =”pr imary−key−name”>< s e l e c t o r xpath =”...”/ >< f i e l d name =”...”/ >
</ k e y r e f >
I Relational requirement: foreign keys don’t have to be unique ornon-null, but if one component is null, then all components must benull.
Munindar P. Singh (NCSU) Electronic Commerce Technologies 296 / 303
Summary and Directions
Outline
Challenges of Electronic Business
Architecture in IT
Contracts and Governance
XML Concepts and Techniques
XML Modeling and Storage
Summary and Directions
Munindar P. Singh (NCSU) Electronic Commerce Technologies 297 / 303
Summary and Directions
Summary and Directions
Collective concept map
Munindar P. Singh (NCSU) Electronic Commerce Technologies 298 / 303
Summary and Directions
Key Ideas
I Information system interoperation
I Architecture conceptually
I Importance of metadata
I XML technologies
I Elements of rational resource allocation
Munindar P. Singh (NCSU) Electronic Commerce Technologies 299 / 303
Summary and Directions
Business Environments
Theme of this course: How is computer science different for openenvironments?
I AutonomyI Messaging, not APIsI Markets
I HeterogeneityI Capturing structure of informationI Transforming structures
I DynamismI Partially addressed through above
Support flexibility and arms-length relationships
Munindar P. Singh (NCSU) Electronic Commerce Technologies 300 / 303
Summary and Directions
Limitations of XML
I Doesn’t represent meaning
I Doesn’t represent conceptual structureI Enables multiple representations for the same information
I Give an example
Transforms can be robustly specified and accurately documented only ifmodels are known, but usually the models are not known
Munindar P. Singh (NCSU) Electronic Commerce Technologies 301 / 303
Summary and Directions
Directions in XML
Trends: sophisticated approaches for
I Querying and manipulating XML, e.g., XSLT and XQuery
I Sophisticated storage and access techniques in traditional relationaldatabases
I Tools that shield programmers from low-level details
I Semantics, e.g., RDF, OWL, . . .
Munindar P. Singh (NCSU) Electronic Commerce Technologies 302 / 303
Summary and Directions
Course: Service-Oriented Computing
I Takes the ideas of this course closer to their natural conclusionsI For autonomous interacting computations
I Basic standards that build on XMLI Descriptions through richer representations of meaningI Engagement of parties in extended transactions and processesI Collaboration among partiesI Selecting the right parties
How to develop and maintain flexible, arms-length relationships
Munindar P. Singh (NCSU) Electronic Commerce Technologies 303 / 303