technical software news

ISSUE ONE 2005

in this issue

A P U B L I C AT I O N O F W O L F R A M R E S E A R C H

Mathematica 5.1 introduces powerful handling of new kinds of data and hundreds of other enhancements and extensions g details inside

Announcing Mathematica® 5.1 The Data Functions Release

A Conversation with Wolfram Research’s Director of Kernel Technology, Tom Wickham-Jones

gridMathematica™ 2 The Gold Standard Environment for Parallel Computing

Also Inside:Applicaton Updates

In the News

New Online

Recent Book Releases

User Profile

Upcoming Events

Announcing Mathematica 5.1 The Data Functions Release

Adding unmatched performance for handling data to unparalleled speed, scope, and scalability

Accessing Data

For access to structured large-scale data, users will benefit from DatabaseLink, the industrial-strength solution for integrating Mathematica with any standard SQL data-base. Compatible with Oracle, Microsoft

“In Mathematica 5.1, I can easily search the DNA sequence for an entire human chromosome—which contains a few hun-dred million base pairs—for sequence pat-terns in less than a second,” says Oyvind Tafjord, lead developer for Mathematica’s string manipulation.

Mathematica 5.1 is the latest version of Wolfram Research’s world-renowned sci-ence and technology software system. Building on the dramatic speed, scope, and scalability improvements of Mathematica 5, Version 5.1 adds a host of new capabilities, especially for working with large-scale, diverse types of data.

Working with Data

“We’ve introduced innovative algorithms to deliver unmatched performance for all steps in the data handling process—importing, analyzing, manipulating, or plotting,” says Tom Wickham-Jones, Director of Kernel Technology. “And when we say data, we encompass textual and network as well as numerical data.” Version 5.1’s comprehensive string manipulation, matching, and searching capabilities allow users to handle string data in the same way as general expressions. Text-intensive applications—such as website manipulation, data mining, and DNA pro-cessing—can now be quickly and effec-tively managed, with performance scalable to gigabyte sequences.

String ManipulationMathematica can parse large-scale string data quickly and easily, allowing for detailed textual analysis in a single step. In the image on the left, Mathematica searches through the entire SARS genome sequence for long repeats. On the right, Mathematica finds and returns every palindromic word from its built-in 90,000-word English dictionary.

g Industrial-strength string manipulationg Built-in universal database connectivityg Highly optimized binary data I/Og Additional import-export formats, including XLS and AVIg Integrated web services supportg Cluster analysis capabilities addedg 2D and 3D automated network visualizationg High-performance array visualizationg Numerical linear algebra performance enhancements

g Fully integrated piecewise functionsg Integration over implicitly defined regionsg Event handling in numerical differential equationsg New algorithms for symbolic differential equationsg Interactive exploration tool for differential equationsg MathematicaMark™ benchmarking toolg GUIKit interface and application builder—now built ing Many more exciting innovations

There are more than 50 new functions, toolkits, and performance improvements in Mathematica 5.1. Highlights include:

2

Access, and all other major databases, DatabaseLink gives users immediate access to the data they need, which can then be analyzed in Mathematica. DatabaseLink also supports advanced features, such as multi-ple connections, transactions, and rollbacks,

DatabaseLinkThe new built-in DatabaseLink now provides an industrial-strenth, ready-made solution for integrating Mathematica with any standard SQL database. In addition to a Mathematica language interface, DatabaseLink also includes GUIKit-based graphical interfaces. Here is an example of how Database Explorer lets you do a quick query on a connected database.

with interactive point-and-click tools for making connections and querying data. High-speed binary file I/O has been added for maximizing import and export performance of data sets. This increases Mathematica’s potential data throughput by a factor of up to 70, sufficient for real-time utilization of new high-performance com-putational capabilities. Existing I/O formats handled by Import and Export have been

Binary Data I/OThese two graphs compare the speed of data import between Mathematica 5.0 and 5.1 for a number of different formats.

sped up too, including the loading of tabular data—now 400 to 700% faster. “Data analysis is a top priority in the development of this and future versions of Mathematica,” says Conrad Wolfram, Direc-tor of Strategic Development. “Version 5.1 addresses the performance issues in binary data I/O head on, systematically eliminating any slowdowns and making Mathematica the standard for complex data analysis.”

continued on next page

Learn How to Use the Latest Features in Mathematica 5.1 by Taking an Online Training Course

Wolfram Education Group offers beginning, intermediate, and advanced courses throughout the world—even from the comfort of your own home or office. New online mini-courses are generally 3–4 hours in length and provide tightly focused, highly applied, and very practical training experiences, at very affordable prices.

For a complete schedule of classes, check our training calendar at:

wolfram.com/r/7013/weg

“ Mathematica 5.1 ... delivers

improvements over Version

5.0 that are vastly out of

proportion for a .1 upgrade.”

Peter Coffee, eWeek

Array PlottingAll of the above images were produced directly by ArrayPlot.

Visualizing Data

Mathematica 5.1 also incorporates many new visualization tools to enhance and extend its data analysis capabilities with new state-of-the-art functions for large-scale graph layout and array plotting. GraphPlot and GraphPlot3D apply a range of algorithms and strategies to find aesthetically optimal 2D and 3D layouts for graphs and networks. Apart from graphing theoretical and pure mathematical appli-cations, these functions can be used to analyze, display, understand, and optimize sophisticated structures.

“Visualizing and interpreting complex information is becoming increasingly impor-tant in the internet age,” says Yifan Hu, Optimization Developer of Kernel Technol-ogy. “We have innovated graph drawing algorithms that will visualize relational data with tens of thousands of nodes in just a few seconds.” ArrayPlot applies similarly dynamic visualization algorithms to arrays, bitmaps, and systems like cellular automata. With

Mathematica 5.1 g continued

and handling added to NDSolve. Instead of just solving over a predefined range, Mathematica can now solve differential equations until a particular function of the solution is satisfied. Complementing these new tools and extensions for data analysis and computa-tion are enhancements to Mathematica’s unique automatic algorithm selection—the inherent intelligence that automati-cally applies the best algorithm to any given task.

data reduction for optimum visibility of large numbers of points, the ability to handle sparse or dense arrays automatically, and a wide range of coloring and other graphics options, ArrayPlot delivers an ideal time-saving mechanism for identifying the structure of matrices and choosing the best way to deal with them.

Algorithmic Enhancements

Mathematica 5.1 introduces unique system-wide support for piecewise functions. These functions, which hold different definitions in different regions, can now be manipulated and computed automatically with more than 100 enhancements to solving, calculus, and other capabilities. Version 5.1 provides the first-ever unified system for general handling of arbitrary-dimension, arbitrarily nested piecewise functions. Differential equations can also be solved over different regions with the unique-ly flexible framework for event detection

Large-Scale Graph LayoutThe example above shows one method GraphPlot uses to optimize graph layout. At first the nodes of a graph are arranged randomly. The example shows progressive rearrangement with a physics-based algorithm in which nodes interact through idealized spring-like forces. GraphPlot gives the final result automatically, always picking the appropriate algorithm to use.

Expanding Piecewise DefinitionsPiecewiseExpand expands nested piecewise func-tions and converts other functions for simplified output automatically or over a specified domain.

4

1 2000 4000 6000 8000 10000

1

2000

4000

6000

8000

10000

1 2000 4000 6000 8000 100001

2000

4000

6000

8000

10000

Rule 30 cellular automaton A 3-color cellular automaton

A 10,000 x 10,000 sparse matrixA processed image

MathematicaMarkThe MathematicaMark2004 benchmark suite includes representative Mathematica programs that sample different aspects of computer system performance. The benchmark package includes an extensible database of benchmarking results, to which new results are automatically compared. Running the function BenchmarkReport on a typical mid-speed computer system generates the notebook above.

inside wolfram research

A Conversation with Tom Wickham-Jones, Director of Kernel Technology

Q: How many Mathematica releases have you been involved in?

A: 5.1 is the ninth—starting with 2.0.

Q: Are they getting easier?

A: The internal complexity of everything that has to get integrated has vastly increased. But one of our big achievements over the years has been developing very sophisticated and streamlined software engineering practices that more than compensate. And the result is that we can deliver versions with more new features in a much shorter time.

Q: From a development point of view, how has Mathematica changed over the years?

A: There’s more and more depth inside. Functions like NDSolve have now really become extensible interfaces into pretty complete sets of state-of-the-art algorithms. Of course, one of the hallmarks of Mathematica is the tremendous effort we put into automatic algorithm selection. So as a user you just have to call a function like NDSolve—and then we use all sorts of sophisti-cated functionality to determine how to actually solve your problem. One of the things that’s made this kind of thing possible is the coherent way that Mathematica is designed—which is what lets us use more and more of Mathematica’s functionality inside each new compo-nent that we add to the system. In early versions of Mathematica, most of the inter-nal code was written directly in an extension of C. Now a lot of the powerful things we’re adding are written in the much higher-level language of Mathematica itself. And that’s part of what’s let us progressively accelerate our development process.

Q: Do you have a favorite development achievement?

A: From its foundation, Mathematica has always been a very general system, applicable in many areas. A key challenge has been to maintain this generality, while also optimizing computational performance. I’ve been work-ing on that for nearly 10 years. It has required a long stream of software architecture innovations. But start-ing with Mathematica 5.0, I believe we’ve now reached the point where there’s no compromise between speed and generality—and in fact Mathematica now runs faster than specialized software systems.

Q: So where does Mathematica go from here?

A: We’ve got a tremendous number of new directions that we’re pursuing. The techniques we’ve developed allow us to interleave ambitious long-term projects with more specific shorter-term ones. We’ve got a terrific platform in Mathematica, and we’re continually seeing new ways that we can use it to go in interesting and important new directions. I’m very excited about the things we’ll be delivering over the next few years.

Dr. Wickham-Jones joined Wolfram Research in 1990 and has worked on the Mathematica kernel ever since. He is also the architect of webMathematica™, the author of Mathematica Graphics, and is a well-known figure in the technical computing community.

EquationTrekkerBuilt with NDSolve and the new GUIKit technology, EquationTrekker is an interactive tool for exploring differential equations. By simply moving the mouse, users can change parameters and see how a system behaves in real time.

Add-on Functionality

In addition to DatabaseLink, several other add-ons and links have been made standard in Mathematica 5.1. The GUIKit interface and application builder—formerly available as an independent toolkit—is now built in, as is web services support for exchanging data over the World Wide Web.

The MathematicaMark2004 benchmark suite is also included in Mathematica 5.1. This new add-on contains a representative sample of programs that stress different aspects of Mathematica’s capabilities. With MathematicaMark, users can instantly esti-mate if a hardware upgrade would be a worthwhile investment. It creates quick benchmark reports that show how a par-ticular computer compares to other machines for overall Mathematica perfor-mance as well as typical computations and processes. Equipped to tackle the most data-intensive tasks with innovative tools for communications and computation, Version 5.1 is a serious extension to the unparal-leled scientific and technological advances of Mathematica 5.

For more details about the new and enhanced features in Mathematica 5.1: wolfram.com/r/7013/newin5.1

Phot

o cr

edit:

Chr

is Br

own/

Wol

fram

Res

earc

h, In

c.

in the news

Wolfram Research will soon release Version 2 of its widely used gridMathematica soft-ware, the gold standard environment for parallel computing. gridMathematica will be optimized for virtually all supercomputers, heterogeneous grids, and personal work-stations. With gridMathematica, scientists and engineers can easily develop parallel applications using the uniquely productive and interactive Mathematica environment and deploy them seamlessly on any parallel computer configuration. Features of gridMathematica 2 include significant scheduling and performance enhancements, automatic parallelization of wider classes of computations, programma-ble scheduling for problem-specific tuning, support for speculative parallelism, improved failure recovery and user configurability, and full optimization for all 64-bit architectures. gridMathematica 2 also capitalizes on the recent improvements in Mathematica, letting users take advantage of new features such as industrial-strength string manipulation,

universal database connectivity, web services support, cluster anal-ysis capabilities, and high-speed binary I/O. gridMathematica gives users immediate access to cutting-edge technologies and the full power of modern supercomputer systems. Build-ing on the rich Mathematica foundation, gridMathematica is perfectly suited for numerical supercomputing applications as well as parallel processing of complex non-numeric data such as images, 2D and 3D vector graphics, geospatial information, genomic sequences, algebraic formulas, logical and program constructs, and gener-al symbolic data. It also gives users access to the world’s largest algorithm collection—in one integrated system. Mathematica has been in long-term use on a number of installations of the Top 500 supercomputers, dating back to its original release in 1988. The introduction of grid-Mathematica in 2002 greatly expanded that

gridMathematica 2The Gold Standard Environment for Parallel Computing

Orion Cluster Workstations Now Include gridMathematicaA recent partnership with Orion Multisystems unites Wolfram Research’s gridMathematica with Orion’s new high-perfor-mance Cluster Workstations for technical computing. With gridMathematica installed, the Orion Cluster Workstation provides a quick way to set up and run large calculations on the most cost-effective cluster architec-ture on the market. Together, gridMathematica and Orion Cluster Workstations provide supercomputer performance with the ease-of-use benefits of desktop and deskside personal computers.wolfram.com/r/7013/orion

user profileGrowing a Grid at Skidmore CollegeFlip Phillips found an “insightful” way to accelerate his research this summer at Skidmore College. Phillips and his colleagues are studying human vision, primarily 3D shape percep-tion, or “how we see what we see.” Phillips’ focus is on empirical studies and actual laboratory testing. When the studies do not match up to existing theoretical models, the onus is on Phillips to prove or disprove their validity, and he is using gridMathematica as one of his primary tools for doing so. Testing the existing theoretical models requires computations—and lots of them. Phillips took advantage of Skidmore’s Unlimited License Program for all Wolfram Research products and used gridMathematica to reduce computation time for each dataset from 1.5–2 months down to only 4–6 hours by “scavenging” for idle Macintosh computers throughout the campus network. When a free machine is found, the program loads gridMathematica and any needed data and starts to calculate. With some adjustments, such as reviewing intermediate results and “pruning off” undesirable branches, Phillips says the process is made even faster.wolfram.com/r/7013/skidmoregrid

audience, and gridMathematica has since been successfully deployed by many major corporations, government organizations, and universities. With gridMathematica 2, Wolfram Research further improves upon this optimized parallel Mathematica envi-ronment for modern supercomputers.

For more information: wolfram.com/r/7013/gridmathematica

“ [gridMathematica] is a fantastic product! What else can get networked computers

to cooperate using a very natural, interpreted language implementation? It is

one thing to construct and benchmark a cluster or a Beowulf project and entirely

another to actually use it for applications, particularly applications that have

already been developed .… I was doing parallel computations the same day that

[gridMathematica] came out of the box.”

Chris French, Thomson Consumer Electronics

6

application updates

Visit the New Documentation Center The Wolfram Research Documentation Center has been redesigned for easier searching and retrieval of information and has been updated to include the complete documentation for Mathematica 5.1. The Documentation Center also features comprehensive online references for all other Wolfram research products and applications.wolfram.com/r/7013/documents

Order Your Own Unique Mathematica Gallery ArtworkSeveral vivid, full-color images can now be purchased as posters from the Mathematica Gallery Collection. These graphics by Michael Trott, author of the recently published book, The Mathematica GuideBook for Graphics, bring together art and science to produce powerful, eye-catching visualizations of interesting mathematical surfaces. These affordable new prints are available in many different sizes with options for matte, gloss, and even custom framing.wolfram.com/r/7013/gallery

Mathematica in Education and Research (MiER)Back in a new quarterly format, MiER is now available in both electronic and print versions. The journal publishes articles, notes, and special features on the use of Mathematica technology in all aspects of education and research. Article submission guidelines are available on the website. wolfram.com/r/7013/mier

Wolfram Technology Conference ProceedingsThe 2004 Wolfram Technology Conference offered more than 80 presentations, classes, and workshops on Wolfram technologies and their applications. Select presentation notebooks are available in the Mathematica Information Center. wolfram.com/r/7013/techconf2004

upcoming eventsfebruary 7–18, 2005The 2005 Wolfram Research Tour of India—the third major leg of a transcontinental lecture pro-

gram—will focus on Mathematica as a tool for exploration in mathematics and science. Lectures by longtime Mathematica kernel architect Dr. Rob Knapp will demonstrate how Mathematica’s design and key technologies mesh to make it the technical software of choice for a wide variety of applications.wolfram.com/r/7013/india2005

march 21–25, 2005Look for Wolfram Research at the Annual APS March Meeting in Los Angeles, California. Wolfram develop-ers will be on hand to demonstrate and discuss the latest Wolfram technologies and how they relate to the research and application of physics. wolfram.com/r/7013/aps

june 20–july 8, 2005Apply now for NKS Summer School 2005, a unique opportunity to get involved with the ideas and dis-coveries of A New Kind of Science. Hosted at Brown University in Providence, Rhode Island and led by Stephen Wolfram, the program will give students the opportunity to explore and develop original research at the frontiers of science.wolfram.com/r/7013/summerschool

july 10–15, 2005Rocky Mountain Mathematica is conducted by certi-fied Mathematica trainers and consultants. Held in Frisco, Colorado, it features courses aimed at teachers of mathematics and other scientific subjects who wish to enhance their use of Mathematica for presentations, projects, teaching, and research. wolfram.com/r/7013/rrm

august 5–8, 2005The Seventh International Mathematica Symposium (IMS 2005) in Perth, Australia, is an interdisciplinary conference for users

of Mathematica in mathematics, natural and life sci-ences, social sciences and law, engineering, graphics and design, arts and music, education, industry, and commerce. The closing date for abstracts is February 28, 2005.wolfram.com/r/7013/ims2005

october 6–8, 2005The 2005 Wolfram Technology Conference will take place at the Wolfram Research headquarters in Champaign, Illinois. The

conference will offer a variety of sessions targeted to technical computing users of all levels and fields, as well as to software administrators, developers, authors, trainers, and consultants.wolfram.com/r/7013/techconf2005

For a complete listing of upcoming events, go to:wolfram.com/r/7013/calendar

new online

experience exchangeSubmit your nonproprietary journal or conference articles, papers, web links, or other Mathematica-related success stories to experience@wolfram.com. If we feature your work on our website, we’ll send you a Mathematica T-shirt. wolfram.com/r/7013/experience

Download the Latest Applications for Mathematica 5.1Many Mathematica application packages have been updated for compatibility with Version 5.1. The updates are free to registered users of the applications and can be downloaded immediately from the Wolfram website. A current list of updates is available online.wolfram.com/r/7013/apps

A Definitive, Four-Volume Series by Michael TrottExplore Mathematica’s application to numerous problems across mathematics and the sciences with these long-awaited books by a well-known and respected member of the Mathematica community. The Mathematica GuideBooks by Michael Trott provide new insights into almost every function in Mathematicawith a wide range of incisive examples and many engaging exercises. The Programming and Graphics volumes, the first two titles in a four-volume set, guide novice and expert readers to gaining fluency in Mathematica pro-gramming constructs and visualization capabilities. Each volume has over 1000 pages and includes a DVD-ROM complete with executable Mathematica code and programs, rendered color graphics, and animations. The Numerics and Symbolics volumes are scheduled to be published in spring 2005. More information—including sample chapters and exercises—can be found on the website.wolfram.com/r/7013/guidebooks

Now AvailableThe Mathematica GuideBook for Programming wolfram.com/r/7013/trott1

The Mathematica GuideBook for Graphics wolfram.com/r/7013/trott2

Wolfram Research, Inc. 100 Trade Center Drive, Champaign, IL 61820-7237, USAinfo@wolfram.com

Wolfram Research Europe Ltd. 10 Blenheim Offi ce Park, Lower Road, Long Hanborough,Oxfordshire OX29 8RY, UNITED KINGDOMinfo@wolfram.co.uk

Wolfram Research Asia Ltd. Oak Ochanomizu Building 5F, 3-8 Kanda Ogawa-machi,Chiyoda-ku, Tokyo 101-0052, JAPANinfo@wolfram.co.jp(Reseller support only.)

New Mathematica-Related Books

Other New Books in the Wolfram StoreComputational Geosciences with Mathematica by William C. Hanebergwolfram.com/r/7013/haneberg

Differential Equations: An Introduction with Mathematica, Second Edition, by Clay C. Rosswolfram.com/r/7013/ross

The SIAM 100-Digit Challenge: A Study in High-Accuracy Numerical Computing by Folkmar Bornemann, Dirk Laurie, Stan Wagon, and Jörg Waldvogelwolfram.com/r/7013/bornemann

© 2005 Wolfram Research, Inc. Mathematica is a registered trademark of Wolfram Research, Inc. gridMathematica, MathematicaMark, webMathematica, and MATHwire are trademarks of Wolfram Research, Inc. All other trademarks used herein are the property of their respective owners. Mathematica is not associated with Mathematica Policy Research, Inc. or MathTech, Inc. MKT7013 388256 1204.jas

Moving? Please let us know so that we can deliver this newsletter to your new address. Update your information at: www.wolfram.com/mathwire/subscribe.cgi

Keep Up to Date with MATHwire™ To have the latest news and updates from Wolfram Research delivered directly to your inbox, subscribe to MATHwire, our email newsletter, at: wolfram.com/r/7013/mathwire