locate specific sections of your xml documents with xpath ...€¦ · locate specific sections of...

Locate specific sections of your XML documentswith XPath, Part 2Refine XPath results using predicate matching

Skill Level: Intermediate

Brett D. McLaughlin, Sr. ([email protected])Author and EditorO'Reilly Media, Inc.

17 Jun 2008

Part 1 of this tutorial gave you a foundational understanding of XPath. Using slashnotation, wildcards, unions, and simple text, you learned how to locate elements andattributes anywhere within an XML document. However, sometimes you need morethan just matching based on the name of a node. Predicates give you advanced andrefined searching capabilities, allowing you to evaluate the values of attributes andthe parent and child nodes of a targeted element. Rather than find a wider node setand refine or filter that set programmatically, you can add predicates to your XPathsto find exactly the nodes you want.

Section 1. Before you start

Learn what to expect from this tutorial and how to get the most out of it.

About this tutorial

This tutorial details the XPath specification, which allows you to specify particularsections of an XML document using a directory-like syntax. You'll learn the syntax ofXPath, and you'll work with tools that let you experiment with XPath. By the time youcomplete this tutorial, you'll be well beyond the basics of XPath, and you'llunderstand predicates, more complex matching, and how to use XPath in your own

Locate specific sections of your XML documents with XPath, Part 2© Copyright IBM Corporation 1994, 2008. All rights reserved. of 63

mailto:[email protected]

http://www.ibm.com/legal/copytrade.shtml

applications.

Objectives

Frequently used acronyms

• API: application programming interface

• HTML: Hypertext Markup Language

• URI: Uniform Resource Identifier

• W3C: World Wide Web Consortium

• XHTML: Extensible Hypertext Markup Language

• XML: Extensible Markup Language

• XSL: Extensible Stylesheet Language

• XSLT: XSL Transformations

Part 2 of this tutorial focuses on using predicates and predicate matching in yourXPaths. It covers the various selectors XPath provides for being as specific orgeneric in a search as you like. It also shows you how to build complex predicatesinto a SQL-like set of queries, all using only standard XPath syntax.

You'll begin to understand the ideal applications of XPath, and you'll know whenXPath is best used alongside another XML technology such as XQuery. Finally,you'll see how XPath deals with data types, particularly when you performcomparisons of attribute values to numeric values.

Prerequisites

Other tutorials in this series

• Part 1: Use XPath to navigate and select portions of anXML document

The most important prerequisite to this tutorial is that you thoroughly read and workthrough Part 1 (see Resources). That tutorial covers all the basics of XPath, and itincludes a detailed discussion of nodes and the piece-by-piece evaluation thatXPaths are handled with. Make sure you're familiar with all of that before you startPart 2.

Additionally, this tutorial is written for XML document authors and programmers. Youshould be familiar with and comfortable reading, writing, and manipulating XML. Youshould also be familiar with XML concepts, including the following:

developerWorks® ibm.com/developerWorks

Locate specific sections of your XML documents with XPath, Part 2 of 63 © Copyright IBM Corporation 1994, 2008. All rights reserved.

http://www.ibm.com/developerworks/edu/x-dw-x-introxpath1.html



• Elements

• Attributes

• Text

• The root element

Familiarity with the Document Object Model (DOM) is helpful for an understanding ofnodes, but is not required. If you want to read up on the DOM, visit Resources to findseveral relevant links. You should also make sure you're comfortable with thesection on nodes from Part 1 of this tutorial.

This tutorial will mention several other APIs and specifications as well, includingXSL, XSLT, and XPath. Knowledge of any of these is helpful, but not required. Formore information on any of these, consult Resources in the tutorial.

Section 2. Set up your environment for the examples

The document and setup for this tutorial are identical to Part 1, so if you've alreadyworked through Part 1, you're ready to start Part 2.

You'll work with an XML document throughout this tutorial. You need to have thisdocument accessible on your machine, and to be familiar with the basic structure ofthe document. Additionally, you'll need a tool that executes your XPath locations andgives you feedback on what you've selected. This section explains how to get all ofthis working on your personal environment so you can follow along with the tutorialexamples.

Unfortunately, the tools to evaluate XPath locations are still specific to eachoperating system. Some of the nicer tools that download as EXE files and run onMicrosoft® Windows® won't work on Mac OS X. Similarly, the tools to work withXPath on Mac OS X won't run on Windows. While you can use Java™ programsand classes to create a system-independent means to work with XPath, this tutorialfocuses on XPath rather than any particular programming language.

The following sections detail how to get a tool to work with XPath on both Windowsand Mac OS X. Choose the section you're interested in. Once you have a tool, allthe syntax and examples shown throughout the rest of this tutorial will work on anyplatform; you'll just use your tool to evaluate the XPaths.

Evaluate XPaths on Windows

ibm.com/developerWorks developerWorks®



One of the better tools to work with XPath on Windows is Stylus Studio (seeResources for links to the Stylus Studio Web site and downloads page). Downloadany of the Stylus Studio trial versions—Enterprise Suite, Professional Suite, orHome Edition—and install them on your platform.

Once you install Stylus Studio, make sure you can evaluate the XPaths from the firstpart of the tutorial; then you know that you're ready to go. Your screen should looksimilar to Figure 1.

Figure 1. Stylus Studio allows you to evaluate XPath locations

Evaluate XPaths on Mac OS X




The tools to work with XPath on Mac OS X—without using Java classes—are lessrefined and polished than their Windows counterparts. The most useful tool forlearning XPath is AquaPath, which is open source and free for download (visit thedownload links in Resources). Download AquaPath as a disk image, and simplydrag the AquaPath application from the mounted image into your Applications folder.

Double-click on the newly installed AquaPath application, and you should see aspartan-looking screen, like the one in Figure 2.

Figure 2. AquaPath provides for XPath evaluation on Mac OS X

It doesn't look like much, but as you start to load XML documents and type inXPaths, this tool—like Stylus Studio—is worth its weight in gold.




An XML test document

XPath is more about XML than any other programming language. So while mostprogrammers eventually move to using XPath through an API from Java or C#programming, this tutorial focuses purely on evaluating XPaths (through a tool suchas AquaPath or Stylus Studio) against XML documents. It should go without saying,then, that you'll need an XML document to write XPaths against. And, for the sake ofteaching, you'll need a document that's fairly lengthy (searching a 50-line documentisn't very impressive) and has lots of elements and attributes with data values.

Listing 1 shows just a portion of this XML document, which is the Ant build file fromthe Apache Xerces2 Java Parser. This document includes a lot more than is shown,but it would take pages just to display. However, you can download the entire XMLdocument from Resources.

Listing 1. Sample XML document for tutorial

<?xml version="1.0"?><project default="usage" basedir=".">

<property name="build.dir" value="./build"/><property name="data.dir" value="./data"/><property name="docs.dir" value="./docs"/><property name="samples.dir" value="./samples"/><property name="src.dir" value="./src"/><property name="tests.dir" value="./tests"/><property name="tools.dir" value="./tools"/>

</project>

Make sure you have xerces-build.xml somewhere that's easily accessible, and thatyour XPath tool is installed and ready to use. With those two things in place, you'reready to start.

Section 3. Be more selective with predicates

All of the XPaths you wrote in Part 1 of this tutorial focused on selecting certainnodes one step at a time until you arrived at a destination node. For example, in theXPath //target/copy/fileset, the destination nodes are fileset nodesnested within a copy element, nested within a target element, located anywherein the source document. This works well when you look for a node set that's definedby the elements or attributes (or both, if you use the | union operator) it contains,and nothing more.

However, XPaths like this have a limitation to. They are useful—and in many casesjust what you want—but they don't take advantage of a lot of the information in thesource XML document. For instance, you've no tools (yet!) to make any use of thevalues of all the attributes in the source XML document. You also have no way touse what you've learned so far to return, for example, elements with only certainattributes, or certain child elements. You can return the attributes or child elementsthemselves by using XPaths such as //target/copy/*, which returns childelements of copy elements nested within target elements. But what if you don'twant those child elements? What if you want all the copy elements that have childelements?

When you want to restrict or filter your results based on criteria that goes beyond thenames of elements and attributes and the path to those elements and attributes, youneed predicates. A predicate is a short expression, embedded in square brackets ([and ]), that affects one particular node set in your XPath expression. A predicateallows you to further refine your XPaths and add a tremendous amount of flexibilityand power to your XPath toolbox.

Add a check for a particular attribute or element

The most basic form of predicate matching is to take a node set and check to see ifthe nodes in that set have particular attributes or child elements.

Add a check for a particular attribute

Suppose you want to find all copy elements that are nested within targetelements. That's simple enough; you use an XPath like //target/copy. However,in Ant build files—like xerces-build.xml, the example file for this tutorial— copyelements come in two basic forms:

• An empty-element form: There's usually a file attribute and a tofileattribute, indicating what file to copy and the new name to use. This formlooks like this:

<copy file="${src.dir}/org/apache/xerces/parsers/org.xml.sax.driver"

tofile="${build.dest}/META-INF/services/org.xml.sax.driver"/>

• A form with child elements: A todir attribute is on the copy element,but the other copy-related information is in child elements. This form lookslike this:

<copy todir="${build.src}"><fileset

dir="${src.dir}"includes="org/apache/**

org/w3c/dom/html/HTMLDOMImplementation.java"excludes="**/classfiles_updated **/CVS* **/.#* **/XMLMessages.java

**/DatatypeContentModel.java **/ComplexTypeInfo.java**/v1/** **/v2/**javax.xml.parsers.ConvertToURI.java">

</fileset></copy>

The XPath //target/copy returns both of these forms. Further, if you want toretrieve the actual fileset element nested within a copy element, you could usean XPath like //target/copy/fileset. You can also easily retrieve the todiror file attributes using XPaths like //target/copy/@todir or//target/copy/@file. In those cases, you implicitly select either the form ofcopy with nested children (by selecting the todir attribute) or the form that's empty(by selecting the file attribute). In both cases, the resulting node sets still containthe attributes, not the copy elements themselves.

But what if you want to only return the copy elements with a file attribute? That'swhere you need to specify a match but have that match separate from the targetnodes. Listing 2 shows you the syntax to use a predicate to match an attribute.

Listing 2. Select elements based on the existence of an attribute

//target/copy[@file]

Go ahead and type this XPath into your XPath evaluator. You should get results thatlook like Figure 3.

Figure 3. This predicate selects only copy elements with a file attribute

The first part of this XPath should look familiar: //target/copy. The new part is inbrackets: [@file]. The brackets indicate that this is a predicate. That means thatit's evaluated, but the evaluation occurs against each node in the node set returnedby the preceding XPath. So for every node that's returned by //target/copy, that




node is evaluated against the predicate [@file]. When you drop the brackets,which are just syntax to indicate a predicate, you're left with something that shouldalso look familiar: @file. This is just an XPath that indicates an attribute namedfile.

Putting this all together, then, each node returned from //target/copy is tested tosee if it has an attribute named file. If so, the predicate returns true, and thatparticular node is included in the node set. Any nodes that do not have a fileattribute cause the predicate to return false, and the node is rejected. So theresulting node set, as illustrated in Figure 3, has only the copy elements you want,which are the ones with file attributes on them.

It's similarly easy to get the copy elements with a different attribute, such as todir.Listing 3 takes care of that task.

Listing 3. Select elements based on the existence of a different attribute

//target/copy[@todir]

Add a check for a particular child element

Checking for child elements is as simple as checking for attributes. Suppose youwant all the copy elements with child fileset elements. However, it's risky toassume that a copy element has child elements because it has a todir attribute.Rather than checking for an attribute, you want to select copy elements based onthe existence of a particular child element. You could use an XPath like that inListing 4.

Listing 4. Select elements based on the existence of a child element

//target/copy[fileset]

You'll get the results in Figure 4. Note that not all copy elements are selected; onlythe ones with the indicated child element are accepted.

Figure 4. This predicate selects only copy elements with a fileset child element




There's not much to say here; this is pretty straightforward. A node set is generatedfrom the XPath //target/copy, then the predicate [fileset] is applied. With no@ character, fileset is assumed to be an element and a child of the current nodeset. So all copy elements with child elements named fileset cause a true returnvalue, and those elements are returned by the XPath. copy elements without thatchild element are not returned.

Predicates can examine multiple levels of nesting

You're not limited to just the attributes of the node set that you're currentlyexamining, or even just the child elements of that element. Suppose that instead ofwanting the copy elements that have a nested fileset element, you actually wantthe target elements that contain those copy elements. So effectively, you say thatyou want to find the target elements with nested copy elements, which in turnhave a specific nested child element (fileset). The predicate you need here has anesting in it as in Listing 5.

Listing 5. Select elements based on the existence of a child element's child




element

//target[copy/fileset]

This time, your results should look like Figure 5.

Figure 5. This predicate selects only copy elements with a particular nesting ofelements

You can take this as far as you want; the predicates in your XPaths are reallyXPaths themselves (plus a few extra wrinkles that I'll show you later). You couldbuild even more complex XPath predicates, such as those shown in Listing 6.

Listing 6. Predicates can have several layers of nesting

//target[java/classpath/pathelement/@path]

Figure 6 shows the resulting node set from this. In this case, there's only onematching node in the source document.




Figure 6. Predicates can combine element and attribute paths

XPaths like this can appear a little daunting, especially when the predicate is aslong—or longer than—the nonpredicate part of the XPath. However, if you workthrough them piece by piece, you'll have little trouble working with them. Ultimately,the flexibility is being able to return any set of nodes along a much lengthier XPath.With predicates and element and attribute matching, you're no longer limited toending your XPath at the level of node you want to return.

XPaths can have multiple predicates

One of the mantras of these tutorials is that XPaths are evaluated piece by piece.




With each forward slash you see, a node set is evaluated, and then the rest of theXPath is evaluated relative to those nodes. When you add predicates to the mix,nothing changes. Because of that, you can actually include multiple predicates aspart of one XPath. You simply add each predicate to the node set you want it toapply to. Listing 7 shows an XPath that is fairly simple but involves more than onepredicate.

Listing 7. Predicates can have several layers of nesting

//target[@depends]/jar[metainf]

The resulting node set is shown in Figure 7. Notice that there's nothing special aboutthis node set; it's just like all the rest you've generated. The only difference is thatyou've applied filters at several levels of the XPath.

Figure 7. An XPath can have more than one predicate

Walk through this XPath step by step:




1. The evaluator locates all target elements, regardless of where in thedocument they appear.

2. For each node in that set, the evaluator applies the predicate[@depends]. If the node has an attribute named depends, the evaluatoradds that node to the result set. If not, it drops the node.

3. For each remaining node, the evaluator checks to see if there are anychild jar elements. If so, the evaluator keeps the node in the result set.

4. For each jar element in the result set, the evaluator applies the predicate[metainf]. So if the jar element has a child element named metainf,the evaluator keeps that node in the result set.

5. Since the XPath is complete, the evaluator returns the remaining nodesas part of a result set.

Taken as a set of steps and evaluations, the evaluation process is pretty simple. It'salmost like taking two XPaths, each with a predicate, and combining the two XPathsinto a single XPath. That's the beauty of the step-by-step (or piece-by-piece, if youlike) processing of XPath: You can apply any predicate and any path at any stage,and you get predictable results without worrying about extra or unusual syntax.

Section 4. Select nodes by their position

Once you add predicates to your XPath toolbox, you open up a world of additionalselection possibilities. So far, you saw how to refine your node sets based on thosenodes' attributes and child elements. That's pretty handy, but sometimes you justwant the first node in a list, the last node, or some combination of nodes. In thesecases, you look to select nodes not based on the XML document structure (otherthan the XPath that gets you your working node set in the first place), but ratherbased on a node's position.

For programmers, this is especially apropos. In any language that supports lists andloops, it's trivial to grab an item in a list by its position, whether by an array index of amethod like firstChild() or by get(list.size()-1). Thankfully, XPathprovides for positional matching in predicates, and that means you can get at justthe nodes you want in your own XPaths.

Select the first or last node




The simplest positional operations are selecting one single node from a set—eitherthe first node or the last. Because you already know that XPaths are evaluated andthe result is a set of nodes, you probably already know how to get at nodes by theirpositions: using a syntax that is one part array notation, one part list.

Select the first node in a set

Node sets are essentially index-based lists. You can access each item in a node setby its numeric position within a predicate. To get the first item in a node set, you'dapply the predicate [1]. It doesn't get much simpler than that. Listing 8 retrieves thefirst node in a fairly interesting XPath.

Internet Explorer doesn't always handle [1]correctlyInternet Explorer, from version 5 on, has been incorrectlyimplementing the XPath specification. In that browser, 0 is allowed,and it points to the first node in a set. This is blatantly wrong, andthe hope is that newer XPath implementations in their browsers willcorrect this mistake. Be aware that if you're evaluating XPathsdirectly in Internet Explorer (which really isn't a common situation),[1] selects the second node; [0] selects the first.

Listing 8. The predicate [1] selects the first node in a set

//target[@depends]/java[1]

Evaluate this XPath in your favorite tool; you'll get results similar to those in Figure 8.

Figure 8. The [1] predicate selects the first node in a set




But wait, something odd is going on here. This XPath returned three nodes!Remember, XPath evaluates multiple actual nodes at once. Walk through thisexpression, and consider carefully what actually happens:

1. The evaluator locates all of the target elements in the document. Anode set is composed that has each of these elements, regardless ofposition.

2. The evaluator applies the first predicate, [@depends]. A new node setcontaining only target elements with depends attributes is created.

3. The next part of the XPath is from the preceding slash (at the end of//target[@depends]/) to the next slash, or the end of the XPath.That's java[1]. For each element in the current set, the evaluatorcreates a set of java elements. But remember, a set of these javaelements occurs for each target element from the previous part of theXPath. If three elements matched //target[@depends], then threesets of nodes are in play right now.




4. For each of those sets, the evaluator applies the predicate [1]. So thefirst item in each set is returned. That means that if three node sets areevaluated, there are three first elements.

5. The evaluator unions all of the nodes from all sets that are still viable intoa single result set. The result, in this case, is three java elements, all ofwhich are the first java element under an element named target thathas a depends attribute.

This is where understanding exactly how XPaths are evaluated is critical. Even inXML circles, many authors and developers have no idea that the predicate [1] in anXPath—even at the end of the XPath—doesn't guarantee a single node is returned.

Select the first node in a result set

You're probably wondering exactly how you select the first node in the final resultset. That's simple, because XPath allows you to specify the order of operationsusing parentheses, just like in math. Listing 9 shows an XPath that is almostidentical to Listing 8 but generates a very different result.

Listing 9. The predicate [1] is applied to an entire XPath

(//target[@depends]/java)[1]

Try this, and you'll see the results in Figure 9 that you probably were expecting backin Figure 8.

Figure 9. You can use the [1] predicate to select the first node in an overallresult set




This isn't too complex. Any time you surround a part of your XPath in parentheses,that portion is evaluated completely before applying a predicate. Note that you couldrestate Listing 9 as (//target[@depends]/java[1])[1]. However, the first[1] predicate is redundant here.

Also, keep in mind that the first index in a set for an XPath node set is always 1, not0.

Select the last node in a set

When you want to select the last node in a set, you can't resort to a numericoperator; what's the last index of a node set of indeterminate length? To get the lastnode, you need to use one of several XPath functions this tutorial introduces:last(). The last() function does just what you'd expect: It returns the last nodein the set it's applied on. Listing 10 is a similar example to Listing 8.

Listing 10. The predicate [last()] selects the last node in a set

//target[@depends]/java[last()]




Like Listing 8 —and what you saw in Figure 8 —this returns the last node from threedifferent node sets. Check out Figure 10 to see these results visually.

Figure 10. The [last()] function predicate selects the last node in a set

By now, this shouldn't be a surprise. In fact, you can simply take the steps shownpreviously and replace the word "first" in the second-to-last and last step with "last":

1. The evaluator locates all of the target elements in the document,regardless of position.

2. The evaluator applies the first predicate, [@depends]. The evaluatorcreates a new node set containing only target elements with dependsattributes.

3. The evaluator processes the next part of the XPath, java[1]. For eachelement in the current set, the evaluators form a set of java elements. Sothree sets of nodes are in play at this point.

4. For each of those sets, the evaluator applies the predicate [last()]. So




the last item in each set is returned. This results in three last elements.

5. The evaluator unions all nodes from all three sets into a single result set.The result is three java elements, all of which are the last java elementunder an element named target that has a depends attribute.

Similar to the use of [1], you can apply last() to an entire XPath usingparentheses, as Listing 11 demonstrates.

Listing 11. The predicate [last()] is applied to an entire XPath

(//target[@depends]/java)[last()]

Figure 11 shows the single node that this expression returns.

Figure 11. You can use the [last()] predicate to select the last node in anoverall result set

Remember, when you use [1] or [last()] on an entire XPath like this, your resultset will always have either a single node in it or no nodes.




There is no first() function in XPath

There is a last() function in XPath, but there is not a corresponding first()function. If there were, that function could conveniently solve the Internet Explorerproblem mentioned in the Internet Explorer doesn't always handle [1] correctlysidebar. However, first() is probably left out of the XPath specification becauseit's completely redundant. In a spec-compliant implementation of XPath, a predicatelike [first()] would always be equivalent to [1], so there's no purpose in addingthe function to the language.

In any case, this warning—that first() isn't an XPath function—is particularlyimportant because some XPath evaluators, including AquaPath on Mac OS X, don'treport an error when you use this function. So an XPath such as(//target[@depends]/java)[first()] simply returns an empty node set.This is particularly irresponsible, as it masks a syntax problem and can be tricky todebug.

Node sets can be empty or single-valued

When you use the positional operators in your predicates, always remember thatthere's no guarantee that a node set has more than one member node. In fact,there's no guarantee that a node set has any member nodes at all. For instance,take the XPath /project. That path selects the root node of the exampledocument. By definition, a document has only one root node. So the XPaths/project[1] and /project[last()] return exactly the same thing: the rootelement named project.

Things get a little more confusing—at least if you're not careful—when you apply apositional operator to an empty node set. Take the XPath /root, applied to thesample document. The result is an empty node set; that's because the root elementis named project, not root. If you apply a positional predicate, you're applyingeither an index like 1 or a function like last() to an empty set. The result is not anerror, but simply another empty set. The idea here is that in XPaths, and particularlyXPaths that have multiple predicates, an error is only generated for extremesyntactical problems. In this case, there's no syntax issue; it's just that an emptynode set has no first node and no last node. The XPath /root[1] simply returns anempty node set.

The reason this is potentially confusing—again, especially in longer XPaths—is thatyou have to track down exactly where the problem occurred. For instance, here's alonger XPath://target[@depends]/java[@fork]/classpath[@path]/pathelement[last()].If this XPath returns an empty node set, how do you know where the problemoccurred? Any portion of the XPath might return an empty node set. If no targetelements have a depends attribute, you get an empty node set. If there are, butthere are no child elements named java, you'd get an empty node set. If there are




child java elements, but none with fork attributes, you get an empty node set, andso on. At each stage of evaluation, if the node set comes up empty, all the remainingparts of the XPath will operate on empty node sets and therefore will return emptynode sets as well. (In this example, the empty node set occurs whenclasspath[@path] is evaluated; there are no classpath elements with anattribute named path in the entire document.)

Select several nodes by their position

You need to know one more critical function to use the position completely in yourXPaths: the position() function. position() returns the index of a particularnode in the current set, and allows you to use that index in your XPaths. Forinstance, you can select all the nodes at a greater position than a certain index, or allthe nodes except for one at a certain position.

position() returns a numeric value

Consider the XPath in Listing 12, which not only shows the use of position(), butalso introduces simple comparisons in a predicate.

Listing 12. Use position() to select only a portion of a node set, based on eachnode's index

//target/property[position() < 4]

This returns all property elements, where those elements are children of atarget element, and whose position is less than 4 in their node set. Figure 12shows the results.

Figure 12. Compare the position() function against a numeric limit

The example in Listing 12 masks the possible situation that happens when using[1] and last(): The predicate acts on the current node set, not the overall XPath.In this case, //target/property[position() < 4] happens to be the sameas (//target/property)[position() < 4] (note the parentheses in thesecond XPath), so the returned nodes are the same. However, going back to theexample XPath from Listing 8 and Listing 10, check out Listing 13.

Listing 13. position() applies to the current node set, not the overall XPath

//target[@depends]/java[position() < 3]

To a novice, it would seem there's no way this XPath can return more than twonodes: the nodes at positions 1 and 2. By now, though, you should know that thepredicate at the end of Listing 13 applies to each node set being worked on. In thiscase, that's not identical to the overall XPath. In fact, Listing 13 returns five nodes,not two (try it out for yourself). It should also be unsurprising that you can apply apredicate with position() in it—as well as any other predicate—to an entireXPath with parentheses. In that case, you'd want something like Listing 14.

Listing 14. You can also apply position() to an overall XPath

(//target[@depends]/java)[position() < 3]

By definition, this XPath can return at most two nodes: the first and second nodes inthe overall result node set.

Other uses of position() are pretty intuitive. Listing 15, for example, selects allthe nodes in a set except for the first node.

Listing 15. Select all but the first node in the current set

(//target[@depends]/java)[position() > 1]

Listing 16 selects just the second node in a set.

Listing 16. You can select a node at a particular position

//target/property[position() = 2]

Of course, Listing 16 is identical in function to the XPath //target/property[2],which is a lot simpler to type and read.

Listing 17 is a useful XPath; it demonstrates a predicate that returns every nodeexcept for the last in the current set.

Listing 17. Compare position() against other functions

//target[@depends]/java[position() < last()]

This also gives you a hint of what's to come: Notice that there are two functions usedin the predicate here. The first, position(), gets the position of the current nodebeing evaluated; this is no big deal. However, the second, last(), gets the positionof the last node in the current set, which has nothing to do with the node beingevaluated, other than the same set is in use. You'll use this a lot later when I explain

SQL-like joins in XPath.

Finally, I cannot overstate that these positional functions apply to the current nodeset being evaluated, not to the overall XPath (unless parentheses are in play). Youmust realize that several different node sets can all be open and in evaluation, sopredicates such as [1] and [last()] can return more than a single node, andpredicates such as [position() < 2] and [position < last()] can returntwo and three times as many nodes as you expect.

position() returns a numeric value

One last cautionary word before you leave positional predicates: The position()function returns a number that is the current node's index. A predicate that has onlythe current node's index in it returns true. That means that the predicate[position()] always returns true. The upshot of this is that an XPath like//target[@depends]/java[position()] returns every node in the result setbeing evaluated, and is equivalent to the same XPath with the last predicatedropped off: //target[@depends]/java. The position() function, then, reallyonly has value when compared with a position or against a position.

Section 5. Compare and filter nodes based on the valuesof attribute nodes

To some degree, everything you saw in predicates so far is just a warm-up for whatamounts to the main event in predicates: comparing attributes to values. Whilechecking for attributes, nested elements, and the position of nodes is important,about 90% of predicates are comparisons of attributes to specific values.

Most common in that category is comparing an attribute or element to a literal value,such as 32 or init. I'll talk about more complex comparisons later, but for now, lookat basic comparisons in a predicate.

Filter based on the value of an attribute

You already know how to select an attribute with an XPath. You'd use something like//target/@name. You also know how to select an element based on it having anattribute: //target[@name]. Both of these XPaths become exponentially morepowerful when you can compare that name attribute to a value.

Compare attributes based on equality

Suppose you want to select the target element that has a name attribute namedinit. The XPath for this is just what you'd expect; that XPath is in Listing 18.

Listing 18. You can compare an attribute to a string literal

//target[@name='init']

This predicate does two things, one of which is implicit and one of which is explicit:

1. It selects only target elements with a name attribute. Without thatattribute, the comparison short-circuits and any node that doesn't havethat name attribute is disqualified from being in the result node set.

2. The explicit (and more obvious) thing happens: The XPath evaluatorcompares the value of the name attribute to the string literal init. If thevalues match, the target element with that attribute is added to theresult set.

Try out this XPath; you should see something like Figure 13.

Figure 13. Compare attribute values to string literals




The values you compare to should be within single quotes. Double quotes actuallywork, but the XPath specification is often used with Web browsers, and just likeJavaScript expressions, XPaths often end up nested in double quotes. By usingsingle quotes for your comparison values, you protect yourself againstshort-circuiting an expression that beings with double quotes, such as the value foran HTML or XHTML attribute.

Compare attributes based on inequality

You've already seen several of the inequality operators in the section on positionalpredicates. You can use these same operators to compare attribute values. Themost common operators here are less than (<), greater than (>), less than or equalto (<=), and greater than or equal to (>=). These usually are applied to attributes withnumeric versions, as in Listing 19.

Listing 19. You can compare an attribute to numeric values

//book[@pageCount > 500]

Note that the XPath in Listing 19 is a sample only and will return an empty node setif evaluated against xerces-build.xml. The example document in this tutorial has noattributes with purely numeric values, so Listing 19 is just an example of what'spossible.

XPath evaluations automatically convert values of attributes to numbers whenpossible in these situations. The rest is pretty basic: The attribute is checked for (theimplicit step already mentioned), then the value of that attribute is compared againsta literal value of 500. If the attribute value is less than 500, the indicated element ispart of the result set.

Another implicit check happens with numeric predicates: An attribute must beconvertible to a number. So an element must have the indicated attribute, and thatattribute must convert to a number; then, the value itself is checked. If the attributecan't be converted to a number, you'll get an empty result set [no matter whether youuse equal (=), less than (<), or greater than (>)].

Not all strings are created equal

Generally, inequality operators such as less than (<) and greater than (>) areassociated with numeric values in your document's attributes. However, that's notthe only type of equality you can work with. You can use those same operators tocompare string values. For example, try out the XPath in Listing 20.

Listing 20. Compare a string-valued attribute to a string literal

//target[@name > 'start']

This XPath compares the value of each target's name attribute to the string'start'. If the name is greater than 'start' —which in XPath means that laterletters in the alphabet are used—then the corresponding target node is returned.So the names usage, tests, and test all return true in the example document.Figure 14 shows a partial result set.

Figure 14. Compare attribute string values to string literals with inequalityoperators

The rules you'd expect are in play here; the first letter is compared, and then thesecond. You get what amounts to alphabetization when you use these stringcomparisons. However, there are some wrinkles, first and foremost withcapitalization. All capital letters are considered less than all lowercase letters. Inother words, "S" is not only less than 's,' as you'd expect, but also is less than "a,""b," "f," and so forth. So if all your attribute names are lowercase, an XPath such as//target[@name > 'Test'] will return all target elements. That's becauseevery lowercase name, no matter what lowercase letter it starts with, is consideredgreater than the "T" in "Test."

Comparisons can occur anywhere within your XPath




Like with any other predicate, you can match and compare attributes anywherewithin your XPath. Listing 21 is an example of several predicates, all with attributevalue matching.

Listing 21. Any predicate can appear anywhere in an XPath

//target[@name='init']/property[starts-with(@name, 'parser')]

A couple of new wrinkles are thrown into this XPath, although they should be notrouble for you to handle. Here's what happens step by step:

1. The evaluator finds all the target elements, regardless of location.

2. The evaluator applies the predicate [@name='init']. So all targetelements with a name attribute are candidates. Then the evaluatorcompares the value of the name attribute to init. In the sampledocument, since targets all must have unique names, the evaluatorreturns a single target node.

3. The evaluator locates all of the property elements of that node directlynested within the target element.

4. The evaluator applies another predicate: [starts-with(@name,'parser')]. starts-with() is a new function that takes twoarguments:

• The node to compare, which can be an element, attribute, or text; thenode is located relative to the current node set

• The string literal that the node's value should start withFor each property element from the previous step, the evaluator checksthe value of the name attribute to see if it starts with parser. If so, theevaluator adds that node to the result set.

Try out this XPath, and you should get results like those in Figure 15.

Figure 15. Check to see if an attribute value starts with a certain literal




Notice that even with new functions in this XPath, and increased complexity, it's stilleasy to break down and understand. That's because you've two key concepts underyour belt:

• XPaths are evaluated piece by piece, with location/navigational steps first,then predicates. Each slash indicates another piece of the overall XPath.

• Each successive piece is evaluated relative to the nodes returned by theprevious piece.

The only slightly tricky part of using a function like starts-with() occurs when




you want to evaluate the current node (rather than one of its attributes). Forexample, looking again at Listing 21, if you want to return the actual attribute nodesrather than the target elements those attributes are on, Listing 22 would be whatyou want.

Listing 22. You can test current nodes in functions like starts-with()

//target[@name='init']/property/@name[starts-with(., 'parser')]

Here, you don't want to evaluate a node relative to the last portion of the XPath, butrather the possible return nodes themselves. You use a single period (.). For UNIX®and shell users, this is just like the current directory syntax, so it should look familiar.This XPath returns the name attributes themselves in the result set.

A quick rundown of partial string-matching functions

You can use a lot more functions in your predicates to do string matching. The mostcommon are starts-with(), which you've seen, and ends-with(), whichshould be pretty intuitive at this point. Listing 23 shows a sample of ends-with().

Listing 23. ends-with() is the opposite of starts-with()

//@*[ends-with(., 'version')]

This XPath selects all attribute nodes (@*) anywhere in the document (indicated bythe double slash). Then, for each of those nodes, it checks to see if the attribute'svalue ends with the string 'version'. There are nine cases of this in the samedocument; try out the XPath on AquaPath or Stylus Studio to see the results foryourself.

Another useful string-matching function is substring(). substring() takesthree arguments:

1. The node to evaluate: This is the same as the first argument to bothstarts-with() and ends-with().

2. The index to begin to figure out a substring.

3. The index to end to figure out a substring.

So substring('name', 1, 3) would return the string value 'nam'. Of course,you usually use this in an XPath, as in Listing 24.

Listing 24. Using a substring() along with the name of a node, through the




name() function

(//* | //@*)[substring(name(), 1, 5) = 'class']

This is another pretty useful and interesting XPath. First, all element and attributenodes, anywhere in the source document, are selected (//* | //@*). That partialXPath alone is useful: It makes selecting everything but text in a document trivial.Then, a predicate is applied to the overall result set (through the parentheses). Thatpredicate first uses the substring() function: substring(name(), 1, 5).name() is a new function; it returns the name of the current node being evaluated.So a target element would return "target"; a classpath attribute would return"classpath." Finally, the returned name is substringed; only the first five charactersare returned (from index 1 to index 5). For "target," this would result in "targe"; for"classpath," "class." Then, that value is compared to the literal "class". Any nodesthat match are added to the result set.

Figure 16 shows the result; basically, all the elements and attributes in the documentthat begin with "class" are returned by this XPath.

Figure 16. Check to see if the name of an element or attribute begins with"class"




By now, you get the gist of functions. For more on functions, check out the onlinereference (see Resources), which has all the XPath functions, along with theirsyntax. They should all be fairly self-explanatory once you complete this section andthe rest of this tutorial.

Section 6. Compare the text of element nodes to values




So far, I've focused mostly on attribute values, but XML has a lot more textual valuesthan just attributes. Elements obviously have text in them in many cases, and youcan use that text in comparisons within predicates, just like you did with attributes.

A new sample document

To play with the textual values of elements—as well as some more advancedpredicates discussed in the latter sections of this tutorial—you'll need a differentsample document. The xerces-build.xml that you've used has no textual valueswithin its elements; they're all empty elements, with all the data in attributes.

Listing 25 is a portion of Shakespeare's play Macbeth in XML. As withxerces-build.xml, the actual file is quite long, so Listing 25 is just an excerpt. You candownload the complete file,http://www.cafeconleche.org/examples/shakespeare/macbeth.xml, from theResources of the tutorial.

Listing 25. Shakespeare's Macbeth in XML format

<?xmlversion="1.0"?><!DOCTYPEPLAY SYSTEM"play.dtd">

<PLAY><TITLE>TheTragedy ofMacbeth</TITLE>

<FM>Textplaced inthe publicdomain byMobyLexicalTools,1992.SGMLmarkup byJon Bosak,1992-1994.XMLversion byJon Bosak,1996-1998.Thiswork may befreelycopied anddistributedworldwide.</FM>

<PERSONAE>

<TITLE>DramatisPersonae</TITLE>

<PERSONA>DUNCAN,king ofScotland.</PERSONA>

<PGROUP><PERSONA>MALCOLM</PERSONA><PERSONA>DONALBAIN</PERSONA><GRPDESCR>hissons.</GRPDESCR></PGROUP>

<PGROUP><PERSONA>MACBETH</PERSONA><PERSONA>BANQUO</PERSONA><GRPDESCR>generalsof theking'sarmy.</GRPDESCR></PGROUP>

<PGROUP><PERSONA>MACDUFF</PERSONA><PERSONA>LENNOX</PERSONA><PERSONA>ROSS</PERSONA><PERSONA>MENTEITH</PERSONA><PERSONA>ANGUS</PERSONA><PERSONA>CAITHNESS</PERSONA><GRPDESCR>noblemenofScotland.</GRPDESCR></PGROUP>

<PERSONA>FLEANCE,son toBanquo.</PERSONA><PERSONA>SIWARD,Earl ofNorthumberland,general ofthe

Englishforces.</PERSONA><PERSONA>YOUNGSIWARD, hisson.</PERSONA><PERSONA>SEYTON,an officerattendingonMacbeth.</PERSONA><PERSONA>Boy,son toMacduff.</PERSONA><PERSONA>AnEnglishDoctor.</PERSONA><PERSONA>AScotchDoctor.</PERSONA><PERSONA>ASoldier.</PERSONA><PERSONA>APorter.</PERSONA>

<PERSONA>AnOldMan.</PERSONA><PERSONA>LADYMACBETH</PERSONA><PERSONA>LADYMACDUFF</PERSONA><PERSONA>Gentlewomanattendingon LadyMacbeth.</PERSONA><PERSONA>HECATE</PERSONA><PERSONA>ThreeWitches.</PERSONA><PERSONA>Apparitions.</PERSONA><PERSONA>Lords,Gentlemen,Officers,Soldiers,Murderers,Attendants,andMessengers.</PERSONA></PERSONAE>

<SCNDESCR>SCENEScotland:England.</SCNDESCR>

<PLAYSUBT>MACBETH</PLAYSUBT>

<ACT><TITLE>ACTI</TITLE>

<SCENE><TITLE>SCENEI. Adesertplace.</TITLE><STAGEDIR>Thunderandlightning.Enter threeWitches</STAGEDIR>

<SPEECH><SPEAKER>FirstWitch</SPEAKER><LINE>Whenshall wethree meetagain</LINE><LINE>Inthunder,lightning,or inrain?</LINE></SPEECH>



</SCENE></ACT></PLAY>

Make sure you have this document available. After you load it up in AquaPath orStylus Studio, you're ready to continue in the tutorial. Note that unlessxerces-build.xml is mentioned specifically, the sample document referred to in therest of this tutorial is macbeth.xml. Additionally, take note that all the element namesin macbeth.xml are uppercase. Because XPath is case-sensitive, make sure thatyou pay attention to capitalization throughout the listings in this tutorial and in yourown experimentation.

Filter based on the nested text of an element

To compare an element's text to anything, you need to get at the text of an element.That's something you've not done before, but it's as simple as grabbing an attribute.The text() function returns the text of an element. Listing 26 returns the title of theplay—in this case, The Tragedy of Macbeth.

Listing 26. Getting the text within an element

/PLAY/TITLE/text()

Make sure you select the window or tab to get textual results. You should seesomething like Figure 17.

Figure 17. You can get an element's textual content using the text() function




Once you know how to get an element's textual content, you can use that withpredicates and other matching functions, as in Listing 27.

Listing 27. Compare textual content to a literal value

//*[contains(text(), 'Scotland')]




This XPath introduces another new function, contains(). contains() checks tosee if the first argument (which is or evaluates to a string) contains within it thesecond value (which is another string). In this case, the XPath in Listing 27 checksall elements to see if any of those elements contain the string 'Scotland' astextual content text. Figure 18 shows the results.

Figure 18. The text() function, which works in predicates, works well alongsidecontains()




If you use Stylus Studio, this XPath will not work; you'll get an error. Go ahead andread the next few paragraphs, and then I'll explain what's going on in detail.

Note that this XPath returns the element nodes that contain that text. If you wantedthe text itself, you'd slightly adjust the XPath, as shown in Listing 28.

Listing 28. Refer to a text node with the . operator

//*/text()[contains(., 'Scotland')]

As with attributes, the . operator refers to the current node; in this case, that's thetext of each element in the document.

This is also a good reminder that text in a document is itself a node, or a collection ofnodes. The XPath /PLAY/TITLE/text() still evaluates to a set of nodes. In thiscase, it evaluates to a single node with the value "The Tragedy of Macbeth."However, the text is contained within a node, just as elements and attributes are.

XPath 1.0, XPath 2.0, and Stylus Studio

If you use Stylus Studio on Windows and you try to evaluate the XPath in Listing 28,you'll probably got an error similar to that in Figure 19.

Figure 19. Stylus Studio reports an error in certain uses of contains()




The problem here really isn't a problem at all. Instead, it's a discrepancy between theversion of XPath that AquaPath implements and the version that Stylus Studioimplements. This article is written largely based on XPath 1.0, because most toolsfully implement XPath 1.0. However, Stylus Studio is ahead of the game (in thisrespect) and implements XPath 2.0.

In XPath 2.0, you must apply contains() to a singular item. That's not to say youcan't apply it across several nodes, but you can't use contains() to evaluateevery node that's a child of a target node, for example. That's a multinode request,and you can't have XPath 2.0's version of contains() do that.

XPath 2.0 does have ways to accomplish this sort of thing, but they involve anentirely new syntax: one that will have to wait for another article or tutorial. If this isof interest to you, let me know by e-mailing me directly or participating in thediscussion forum (the link is in Resources).

Most of the remaining XPaths in this tutorial work in both XPath 1.0 and 2.0, though,so you aren't stuck unable to continue on, no matter what platform or tool you use.




Get the current element nodes' text children

Things begin to get really interesting when you start to apply predicates moreliberally and combine more complex predicates with text matching. For example, theMacbeth play details who the speaker is for every speech. It would be trivial to get allthe speeches where the speaker is, say, one of the witches. For that, you'd use anXPath like the one shown in Listing 29.

Listing 29. Refer to a text node with the . operator

//*/text()[contains(., 'Witch')]

That's interesting, but it's not all that useful. What would be useful, though, is to findall the acts that have a speech by one of the witches in them. That XPath is a bitmore complex; it's shown in Listing 30.

Listing 30. Using multiple predicates with lots of nested children

//ACT[SCENE/SPEECH/SPEAKER[contains(text(), 'Witch')]]/TITLE/text()

Look at the results in Figure 20. Then, I'll walk you through exactly what happenedhere.

Figure 20. Combine nested predicates with contains() and the text() function




This is one of the most complicated XPaths you've seen yet, but if you take thingsstep by step, this shouldn't be a problem:

1. The evaluator locates all ACT elements (//ACT).

2. For each ACT, the evaluator applies a lengthy predicate([SCENE/SPEECH/SPEAKER[contains(text(), 'Witch')]]).




Several steps are involved here:

1. Navigate through a nested SCENE element, then to any SPEECHchild elements, and then to any SPEAKER child elements ofSPEECH (SCENE/SPEECH/SPEAKER).

2. For each SPEAKER element, see if the textual value contains thestring 'Witch' ([contains(text(), 'Witch')]).

3. If the value 'Witch' is found, return true for the predicate.

3. Each node returned in the previous step is still an ACT element. TheXPath evaluator continues by selecting the TITLE of that act.

4. The evaluator returns the textual value of the TITLE node through thetext() function.

None of this sounds too difficult when deconstructed like this. Just break thingsdown into smaller, manageable steps, and you'll evaluate these complex XPathsmentally in no time.

Despite the ability to break this down, though, note one interesting thing: Predicatescan be nested within other predicates. In Listing 30, a predicate on the ACT elementselects a nesting several layers deep. However, there's also a predicate on theelement that's selected in the first predicate. The result is a slightly odd-lookingdouble end bracket (]]), which is something you'll see more and more as yourXPaths get increasingly complex. The rule is simply this: For every node set, youcan apply a predicate. That means that if you have a predicate that selects a set ofnodes, you can apply another node set to that set. Often that allows for twopredicates, and in some cases, three or four.

What is an element's value?

Before you dive into even more advanced uses of XPath, you'll want to deal with theissue of mixed content. Mixed content is when an element has both textual contentand nested elements. This occurs most often in XHTML, and it's a real pain forspecifications like XPath. Because XPath likes to either deal with nested elements ordeal with textual content, many of your XPaths and predicates will get tricky, if notdownright unpredictable, when you start to work with mixed content.

The problem comes down to nodes. Remember, everything in your document istreated as a node: elements, attributes, and text. However, there is no such thing asa "mixed node" that contains both elements and text. So in the case of mixedcontent, you'll end up with multiple nodes under the overall parent element: a nodefor each child element, as well as a node for each piece of text. If an element




appears between two bits of text, you'll have more text nodes than just one. You'llhave a node for the text before the nested element, an element node for the nestedelement, and then another text node for the rest of the text.

The short and succinct solution when you work with mixed content is always writeyour XPaths as specifically as you can, and be careful using—and particularlyrelying on—functions such as text(). It's not a true solution, but you'll discover thatin many cases, you can either avoid working with mixed content (by selecting orusing good constraints on your XML) or live with it and avoid some of thetext-specific functions. Another useful option is to make selections using XPath, butthen use another API or a programming language to handle most of the processingof that text.

Section 7. Use the XPath axes to navigate

At some point in your XPath travels, you'll run across a snooty XML documentauthor who will turn up her nose at your fancy predicates, sniff loudly, and ask howmuch you use the ancestor axis, or perhaps mention that a descendant is a betterchoice. Those terms— axis, ancestor, descendant, and a whole host ofothers—refer to what are called document axes in XPath. They're an important partof XPath, and in truth, not everyone who mentions or uses them is as conceited asthis introduction might make them sound. However, in many cases, you simply don'tneed the axes in XPath.

All that said, just as it's important to understand what a node is even if you never useone explicitly, it's important to understand the document axes in XPath. While youmight rarely use them explicitly, they underpin everything you've done and will do inXPath.

So what's an axis -- or is it axes -- anyway?

A document axis defines a node set relative to the current node. That's the formaldefinition, anyway. The plural of an axis (which is always singular) is axes; so youcan have a single document axis, or you can talk about multiple document axes.And, you've used these axes all along.

For instance, take the XPath /PLAY/TITLE. You could rewrite that XPath as shownin Listing 31.

Listing 31. Use the child axis




/PLAY/child::node()[name()='TITLE']

Here, the XPath uses the child axis. child::node() returns all nodes of the childof the current node set (in this case, the node set selected by /PLAY). Then, theXPath uses a predicate to select the names of each of those nodes and comparethem to the string 'TITLE'. Listing 32 shows a simpler version of Listing 31, stillusing the child axis.

Listing 32. Use the child axis (in a simpler fashion)

/PLAY/child::TITLE

Of course, neither of these are as simple as /PLAY/TITLE, which is why you don'tsee document axes in use as often as you might expect.

Document axes are helping out invisibly

Even though you probably won't use the child axis much, it is how XPath constructsits statements and how XPath evaluators handle their business. Every single XPathyou write boils down to a collection of statements like this:

axis::node-test[predicate]

Most of the time, you leave out the axis, and XPath figures out the axis implicitly.That implicit axis usually turns out to be child, as /PLAY/TITLE was really/PLAY/child::TITLE. In fact, you could spell this out even more clearly:/child::PLAY/child::TITLE. The initial slash moves to the root, and thenthere's a child named PLAY, and then a child of that named TITLE.

When you work with attributes, the axis that's applied is not child but rather anattribute. So, returning briefly to the xerces-build.xml sample, take the XPath/project/target/@name. You could represent this using the attribute axis as inListing 33.

Listing 33. Use the attribute axis

/project/target/attribute::name

Of course, you can spell that out even further; Listing 34 uses the child and attributeaxes.

Listing 34. Use the child axis and the attribute axis




/child::project/child::target/attribute::name

Again, while XPath can do this internally, there's no reason for you to think in theseterms. /project/target/@name is much easier.

What axes are available?

Table 1 lists all the available axes, even though you'll never use most of thesedirectly.

Table 1. XPath axesDocument axis Purpose

self Selects the current node

parent Selects the parent node of the currentnode

child Selects all the children of the currentnode

attribute Selects all the attributes of the currentnode

ancestor Selects all the ancestors—parents,parents of parents, and so on—of thecurrent node

ancestor-or-self Selects the ancestors of the current nodeas well as the current node itself

descendant Selects all the children, children of thechildren, and so on, of the current node

descendant-or-self Selects the descendants of the currentnode as well as the current node itself

preceding Selects all the nodes in the entiredocument that occur before the currentnode

preceding-sibling Selects all the preceding nodes in thedocument that are siblings—at the samelevel—of the current node

following Selects all the nodes in the entiredocument that occur after the currentnode

following-sibling Selects all the following nodes in thedocument that are siblings—at the samelevel—of the current node

namespace Selects all namespace nodes of thecurrent node




Most of these are pretty self-explanatory, and many are axes that are good to knowabout but you won't use in practice in your own XPaths.

Some axes are pretty useful

The question, then, is when are axes useful? The application that often causes thedocument axes to shine is when you want to do more than just check a childelement, or an attribute, or even just a parent node, for a certain condition. Suppose,for example, you want to find all the acts that have a witch involved. That's prettydifficult; you'd have to check all the ACT elements for SPEECH elements where theSPEAKER contains "Witch." Then, you'd need to check all the STAGEDIR elements inthose speeches to see if a witch came on the scene but didn't speak. Finally, you'dwant to check the LINE elements themselves to see if someone mentioned a witch.That's three disparate XPaths, and it's worse yet: STAGEDIR can appear both nextto and outside of a SPEECH element. So you've got to account for both of thosepossibilities; that's two XPaths just for the STAGEDIR element, not just one. Listing35 is actually what you'd need.

Listing 35. A complex XPath to select values from different nested elements

//ACT[SCENE/SPEECH/SPEAKER[contains(., 'Witch')]]/TITLE/text() |//ACT[SCENE/SPEECH/LINE[contains(., 'Witch')]]/TITLE/text() |//ACT[SCENE/SPEECH/STAGEDIR[contains(., 'Witch')]]/TITLE/text() |//ACT[SCENE/STAGEDIR[contains(., 'Witch')]]/TITLE/text()

With axes, though, there's a much easier way. What you really want to do is checkeach ACT element to see if any of the child elements had the word "Witch" in them.That's an example of using the descendant axis, and you do that with the XPath inListing 36.

Listing 36. The descendant axis grabs all children, grandchildren, and so on,of an element

//ACT[descendant::*[contains(., 'Witch')]]/TITLE/text()

It goes without saying that Listing 36 is much simpler, easier to read, and just aseffective as Listing 35. The results of Listing 36 are in Figure 21.

Figure 21. Check several descendant elements for a string value




It's in these cases—where you want to look several layers deep within an element,possibly for disparate elements—that axes shine.

Section 8. Build SQL-like joins with predicates




So far, all the predicates that you used take the current node set and determinewhich nodes to include in the result set by comparing some attribute or child elementof that particular node to a literal value, or seeing if that node has a particular child orattribute. That's pretty common and certainly comprises the majority of the workyou'll do with XPath.

However, at times, the same piece of data is represented in multiple places in anXML document, or perhaps two related pieces of data appear in different parts of thesame document. In those cases, you might want to relate those two pieces of datatogether in an XPath. In other words, you want to make a selection in one part of adocument based on related data in a completely different part of the document.Here's where XPath starts to, but doesn't quite, reach its limits. With some cleverXPaths, you can achieve SQL-like joins and connect related data, even when thepieces of data you try to relate are completely different parts of your XML document.

Predicates can navigate up and down the XML hierarchy

To get a handle on these more complex XPaths and the concept of joining twodifferent sections of a document, realize that while a predicate is always relative tothe preceding node set, you already know how to move from a relative node to anabsolute one: with a single slash (/). You also know how to locate an elementanywhere within a document: with the double slash (//). Both slashes can appear atthe beginning of your predicates.

Consider Listing 37, which is a sort of silly XPath, but a perfectly legal one.

Listing 37. Use a predicate with the double slash

/PLAY/ACT[TITLE = //ACT/TITLE]

This is basically a self-referential XPath, or an identity. Look closely to see whatoccurs here:

1. The evaluator locates all the ACT elements nested within the root PLAYelement.

2. The TITLE element of each ACT is referenced in the predicate. Theevaluator compares that TITLE element to any TITLE elements nestedwithin ACT elements, located anywhere within the document.

3. Since each node returned by /PLAY/ACT/TITLE always matches anode from //ACT/TITLE, the predicate always returns true.




You can state Listing 37 more simply as /PLAY/ACT.

You can do something similar with the XPath in Listing 38.

Listing 38. Matching based on a value in another XPath

/PLAY/ACT[TITLE = //TITLE[contains(., 'IV')]]/TITLE/text()

This predicate selects a TITLE that contains "IV" (matched only by the title "ACTIV") and uses that value to compare against the current node set's TITLEs. Again,this is a silly and useless example; it would be much easier to simply use an XPathsuch as /PLAY/ACT/TITLE[contains(., 'IV')]/text() and have yourXPath be more readable. However, when you start to think about more creativepredicates like this, it leads to some interesting possibilities.

Compare two unknown values with predicates

Take a look at the top of the macbeth.xml sample document. There are a number ofgroups of people, in PGROUP elements, each of which has several PERSONAelements and, optionally, a GRPDESCR element with a group description. Supposeyou want all speeches where the speaker is in a group where the descriptioncontains "Scotland." First, figure out the XPath to get all the speakers without thepredicate. That would be //SPEECH/SPEAKER. Just to make things interesting, let'sfind the speeches in Act II and then return the SPEECH element itself rather than theSPEAKER. That would leave you with an XPath like this://ACT[TITLE[contains(., 'ACT II')]]/SCENE/SPEECH. This leaves outthe SPEAKER element, but you'll need that when you get to the predicate.

Next, figure out the XPath you need to select SPEAKERs with the group you want.That involves grabbing a PGROUP where the GRPDESCR contains "Scotland," andthen for that PGROUP, returning all the PERSONAs. You can see the XPath toaccomplish that in Listing 39.

Listing 39. Relate data through two unknowns in a predicate

//PGROUP[contains(GRPDESCR, 'Scotland')]/PERSONA

Try out this XPath, and you should get the nodes shown in Figure 22.

Figure 22. Get all the PERSONAs that are in a certain PGROUP




Now, you can use this as the right-hand side of an equality. In other words, you wantto compare the text of all the SPEAKER elements from the first XPath to the text of allthe PERSONA elements returned previously. Listing 40 is the XPath you'd use.

Listing 40. Matching based on a value in another XPath

//ACT[TITLE[contains(., 'ACT II')]]/SCENE/SPEECH[SPEAKER/text() =//PGROUP[contains(GRPDESCR, 'Scotland')]/PERSONA/text()]




When you evaluate this expression, be patient. AquaPath took quite a while on mysystem, and this is about as complicated as your XPaths will get. It's also a prettycool XPath: You're selecting nodes in one part of a document based on relatedinformation in another part of the document. The cool part is that the nodes aren'trelated by their hierarchy or place in the document, but rather by a logicalrelationship. In fact, in terms of the XML, there's no relationship at all. There aren'tshared elements or attributes, and even the names of the elements are different ineach section. However, you know that this data is related, and that's what'simportant.

Try this XPath out yourself, and you should see something like Figure 23.

Figure 23. Find all the SPEECHes by SPEAKERs that are in a certain PGROUP

While your XPaths will seldom need to be this tricky, if you ever have related data,being able to connect those pieces of information through an XPath is worth itsweight in gold.




Section 9. Conclusion

Like any specification, XPath isn't of much value unless you know how to use it andyou know when to use it. XPath can do a lot of things, but you'll really only enjoy andget benefit from XPath if you're judicious with it. You must use it when appropriateand not use it when it's not the best tool.

When is XPath ideal?

XPath is a great tool when you want to find specific information within an XMLdocument. Whether you look for elements, attributes, or textual data, nothing is asquick, intuitive, and helpful as a good XPath to get you to the data you seek. XPathalso makes it simple to filter out elements and attributes that you don't want.

XPath is also ideal when you have a solid understanding of an XML document'sstructure. There's a huge difference in performance between an XPath like/descendant-or-self::*[contains(., 'Fred')] and an XPath like//ACTOR[NAME[contains(., 'Fred')]]. The first is terribly imprecise, ispotentially going to take a while to evaluate, and doesn't use any foreknowledge tospeed things up. The second presumes a good knowledge of the source document'sstructure and is pretty easy to evaluate and process.

When is XPath not ideal?

XPath reaches the fringes of its value when you get into SQL-like joins and lookups,as discussed in the last section of this tutorial. Joins are possible, and there'snothing wrong with them, but when you reach that level of XPath sophistication,you're better off examining XQuery. XQuery builds on XPath to give you ordering,even more refined selection criteria, and the ability to process each returned valuemore completely than XPath does.

XPath is also not a transformation and processing specification. It's not great atcombining text, inserting data into a string, adding node values together, or doinganything beyond raw selection. XSLT builds on XPath, and even XQuery can dosome formatting for you. Beyond that, you're best off using a Java or C# (or pickyour language of choice) API to pull in XPath results and then process them in aprogramming language. Look for future articles on IBM developerWorks that detailthat sort of situation.

Where do I go next?




The best thing you can do is use XPath in your own applications, right away. Asgreat as it is to work through 35 or 40 examples in a build file or marked-upShakespeare play, nothing is as effective as trying out XPath on your owndocuments. You know what those documents have in them, and you'll get a lot ofsatisfaction out of selecting hard-to-reach nodes in your own programmingenvironment.

You should also look over existing applications and see if they contain places whereXPath might naturally fit into your application work flow. Look for opportunities to usewhat you've learned from this two-part tutorial.

Then, once you have XPath under your fingertips, you can follow either (or evenboth) of these two paths:

1. Learn how to use XPath from your favorite programming language:Upcoming articles are planned on IBM developerWorks for using XPathfrom Java, as well as XQuery from Java. Both are great places to takeyour XPath knowledge to the next level and incorporate XPath into aprogramming environment.

2. Learn XQuery and/or XSLT: These two XML specifications both buildfirmly upon XPath and give you additional capabilities. You'll reinforcewhat you already know about XPath, and you'll add some new tricks andwrinkles to that knowledge.

Whatever you do, try to get XPath into the fabric of your current applications. You'llbecome more familiar with predicates, the axes, and what XPath can really do. And,hopefully, you'll find that your apps improve and are more flexible, and that XML as adocument format is more accessible than ever before.




Downloads

Description Name Size Download method

Example source code x-introxpath2.zip 58KB HTTP

Information about download methods



http://download.boulder.ibm.com/ibmdl/pub/software/dw/xml/x-introxpath2/x-introxpath2.zip

http://www.ibm.com/developerworks/library/whichmethod.html


Resources

Learn

• Locate specific sections of your XML documents with XPath, Part 1: Navigateand select portions of an XML document (Brett McLaughlin, developerWorks,June 2008): Explore the basics of XPath to easily locate and refer to specificdata in a document, including its various selectors and semantics, in anexample-driven and hands-on manner.

• XPath 1.0: Read the formal definition of XPath in the original specification.

• XPath 2.0: Read the online specification for the most current version of XPath.

• Tutorial on XPath: Understand how XPath is fundamental to much advancedXML usage with this useful but brief tutorial from the W3C.

• Online function reference for XPath, XQuery, and XSLT: Once you understandhow predicates and functions work, visit this great resource to look up syntaxand find functions that aren't commonly discussed.

• Document Object Model (DOM): Explore the concept of nodes, usedextensively in XPath, more fully in this article on the W3C site.

• Understanding DOM (Nicholas Chase, developerWorks, March 2007): Digdeeper into manipulating XML from a node-based API in an excellent tutorial.

• Stylus Studio: Learn and use a robust set of XML-related tools, including anXPath evaluator. Numerous white papers on XML are also available from theStylus Studio Web site.

• Apache Xerces2 Java Parser: The sample XML document used in this tutorial isa build file from this XML parser. Get the file from Download.

• macbeth.xml: Get the file from Download.

• IBM XML certification: Find out how you can become an IBM-CertifiedDeveloper in XML and related technologies.

• XML technical library: See the developerWorks XML Zone for a wide range oftechnical articles and tips, tutorials, standards, and IBM Redbooks.

• developerWorks technical events and webcasts: Stay current with technology inthese sessions.

• The technology bookstore: Browse for books on these and other technicaltopics.

• developerWorks podcasts: Listen to interesting interviews and discussions forsoftware developers.

Get products and technologies





http://www.w3.org/TR/xpath

http://www.w3.org/TR/xpath20

http://www.w3schools.com/Xpath/default.asp

http://www.w3schools.com/Xpath/xpath_functions.asp

http://www.w3.org/DOM/

http://www.ibm.com/developerworks/edu/x-dw-xudom-i.html

http://www.stylusstudio.com/

http://xerces.apache.org/xerces2-j/

http://www.ibm.com/certify/certs/01001503.shtml

http://www.ibm.com/developerworks/views/xml/library.jsp

http://www.ibm.com/developerworks/offers/techbriefings/?S_TACT=105AGX06&S_CMP=art

http://www.ibm.com/developerworks/apps/SendTo?bookstore=safari

http://www.ibm.com/developerworks/podcast/?S_TACT=105AGX06&S_CMP=ART


• Stylus Studio 2008 XML: Download to get started with XPath and XMLdocuments on the Windows platform.

• AquaPath: Download to enable easy XPath location evaluation on Mac OS X.

• Scandalous Software: Check out a solid set of XML-related tools, all targeted atthe Mac OS X platform.

• Java & XML, Third Edition (Brett McLaughlin and Justin Edelson, O'ReillyMedia, 2006): Cover XML from start to finish, including extensive information onvarious XML vocabularies.

• IBM trial software for product evaluation: Build your next project with trialsoftware available for download directly from developerWorks, includingapplication development tools and middleware products from DB2®, Lotus®,Rational®, Tivoli®, and WebSphere®.

Discuss

• Participate in the discussion forum for this content.

• XML zone discussion forums: Participate in any of several XML-relateddiscussions.

• developerWorks XML zone: Share your thoughts: After you read this article,post your comments and thoughts in this forum. The XML zone editorsmoderate the forum and welcome your input.

• developerWorks blogs: Check out these blogs and get involved in thedeveloperWorks community.

About the author

Brett D. McLaughlin, Sr.Brett McLaughlin is a bestselling and award-winning non-fiction author. His books oncomputer programming, home theater, and analysis and design have sold in excessof 100,000 copies. He has been writing, editing, and producing technical books fornearly a decade, and is as comfortable in front of a word processor as he is behind aguitar, chasing his two sons around the house, or laughing at reruns of ArrestedDevelopment with his wife. His last book, Head First Object Oriented Analysis andDesign, won the 2007 Jolt Technical Book award. His classic Java and XML remainsone of the definitive works on using XML technologies in the Java language.

Trademarks



http://www.stylusstudio.com/xml_download.html

http://www.ditchnet.org/aquapath/

http://scan.dalo.us/

http://www.amazon.com/Java-XML-Brett-McLaughlin/dp/059610149X/newinstance-20

http://www.ibm.com/developerworks/downloads/?S_TACT=105AGX06&S_CMP=art

http://www.ibm.com/developerworks/forums/forum.jspa?forumID=262

http://www.ibm.com/developerworks/forums/dw_xforums.jsp

http://www.ibm.com/developerworks/forums/dw_forum.jsp?forum=1080&cat=11

http://www.ibm.com/developerworks/blogs/

http://www.ibm.com/developerworks/community

http://www.headfirstlabs.com/books/hfooad/

http://www.headfirstlabs.com/books/hfooad/

http://www.oreilly.com/catalog/9780596101497/index.html


Adobe, the Adobe logo, PostScript, and the PostScript logo are either registeredtrademarks or trademarks of Adobe Systems Incorporated in the United States,and/or other countries.IBM, the IBM logo, ibm.com, DB2, developerWorks, Lotus, Rational, Tivoli,WebSphere, and pureXML are trademarks of IBM Corporation in the United States,other countries, or both.Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in theUnited States, other countries, or both.UNIX is a registered trademark of The Open Group in the United States and othercountries.Other company, product, or service names may be trademarks or service marks ofothers.




locate specific sections of your xml documents with xpath ...€¦ · locate specific sections of...

Documents