Preserving Command Line Workflow for a Package ManagementSystem using ASCII DAG Visualization

Katherine E. Isaacs and Todd Gamblin

Fig. 1: ASCII depiction of the package dependency graph of dia (40 nodes), as specified in the Spack package management tool.The dependency cairo and its direct neighbors have been interactively highlighted.

Abstract— Package managers provide ease of access to applications by removing the time-consuming and sometimes completelyprohibitive barrier of successfully building, installing, and maintaining the software for a system. A package dependency containsdependencies between all packages required to build and run the target software. Package management system developers, packagemaintainers, and users may consult the dependency graph when a simple listing is insufficient for their analyses. However, usersworking in a remote command line environment must disrupt their workflow to visualize dependency graphs in graphical programs,possibly needing to move files between devices or incur forwarding lag. Such is the case for users of Spack, an open source packagemanagement system originally developed to ease the complex builds required by supercomputing environments. To preserve thecommand line workflow of Spack, we develop an interactive ASCII visualization for its dependency graphs. Through interviewswith Spack maintainers, we identify user goals and corresponding visual tasks for dependency graphs. We evaluate the use of ourvisualization through a command line-centered study, comparing it to the system’s two existing approaches. We observe that despitethe limitations of the ASCII representation, our visualization is preferred by participants when approached from a command lineinterface workflow.

Index Terms—Software visualization, information visualization, command line interface

1 INTRODUCTION

Building and executing software can be a complicated and frustratingprocess due to complex requirements in terms of dependencies, theirversions and install locations, and how they were compiled. Packagemanagement systems, such as Homebrew [22] for OSX systems andAPT [41] for Ubuntu systems, have been developed to handle thesecomplexities automatically for most users. Use of these package man-agement systems can thus expand the usability of software by greatlydecreasing the burden on users attempting to get software to run.

• Katherine E. Isaacs is with the University of ArizonaE-mail: kisaacs@cs.arizona.edu.

• Todd Gamblin is with Lawrence Livermore National LaboratoryE-mail: tgamblin@llnl.gov

Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publicationxx xxx. 201x; date of current version xx xxx. 201x. For information onobtaining reprints of this article, please send e-mail to: reprints@ieee.org.Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx

For users of a particular software application to benefit, someonemust first package the software for the system. This is often done bythe engineer or scientist who developed the application. Furthermore,the developers of the package management system itself expand andmaintain functionality as build systems and architectures evolve. Addi-tionally, power users with unique needs may want to specify additionalrequirements when building the software. In all of these scenarios,understanding the dependencies of a particular software package can bebeneficial. While a list or tree of packages is sufficient for some tasks,sometimes consulting the full set of package relationships is of inter-est. Generally the relationships among a package and all of its directand indirect dependencies form a directed acyclic graph (DAG). Asthese graphs generally are limited to tens of nodes and developers areparticularly interested in the dependency paths, the graph is frequentlyrepresented as a node-link diagram.

Many package management systems are command line tools. Wefocus on one such open source tool, Spack [13, 14], which was devel-oped to meet the needs of supercomputing applications. In this context,

(a) radare2 (b) graphterm

Fig. 2: Package dependency graph of python rendered through (a) radare2’s call graph feature, and (b) our approach.

developers may only be able to access the supercomputer or other re-mote system via a terminal interface. The most expedient method ofvisualizing a small DAG often requires several extra steps in transfer-ring the data to a local machine and running a separate program insteadof the terminal in which the user was working. This step can representa significant overhead in terms of the entire analysis needed. Thesedifficulties with accessing plotted data from command line workflowshave led to the creation of ASCII charting support in tools such asgnuplot [51] and Octave [12]. To better match the workflow of com-mand line-focused users when viewing a DAG, we propose bringinginteractive DAG visualization to the terminal. Our goal is to “meetcommand line users where they are” when possible.

We developed an interactive, terminal-based DAG visualization tosupport the analysis of package dependency graphs. The limitations ofthe terminal pose many challenges. To ensure portability, marks arelimited to ASCII characters that cannot be overlaid. Available colorsare limited as well. Relatively few marks can be shown simultaneouslydue to the row and column size of the terminal. With these limita-tions in mind, we convert a graphical layered graph layout to ASCII.Search and highlight capabilities are available via keyboard commands.We conducted a study comparing the efficacy of the command lineworkflow with our visualization to the workflows using the existingGraphViz-rendered dot [15, 16] layout and static ASCII representa-tions. We find that despite the limitations of the ASCII visualization, itcan be effective in a command line workflow and our study participantspreferred it to the existing workflows.

In summary, our contributions are:

1. A domain and task characterization for package dependencygraphs (Sect. 3),

2. an interactive visualization of directed acyclic graphs using onlythe characters { |, , /, \, X, o } (Sect. 4),

3. a study comparing the use of our interactive terminal-based DAGvisualization in a command line workflow to that of the existinggraphical and ASCII solutions (Sect. 5 and Sect. 6), and

4. a survey of dependency visualizations provided by Github projects(Sect. 2 and supplementary materials).

2 RELATED WORK

Several graphical visualizations, such as SHriMP Views [43], Ex-TraVis [10], SolidSX [37], the layouts available through Roassal’sGRAPH feature [6], and the work of Noack and Lewerentz [31], havebeen proposed for software dependencies, but most focus on the soft-ware creation and maintenance aspect—how the dependencies relate todeveloping the software—rather than the packaging and building pro-cess. One difference is that software module dependencies frequentlyexhibit a containment structure not present in our package dependencydata but emphasized by the existing visualizations. Another is thatsome visualizations make design decisions to accommodate scale ortemporal behavior that our package dependency data does not exhibit.Kula et al. [26] developed a system to show changes in a software’s li-brary dependencies over time, but did not directly show the relationshipbetween the libraries, only with the target software. The visualizationwas for maintainers of the software, not people trying to install thesoftware or develop installation management tools. For a survey ofsoftware dependency visualization, see Caserta and Zendra [8].

Dependency Visualization in Github Repositories. Many opensource software tools offer some form of dependency visualization.We surveyed Github repositories retrieved in a search for “visualizedependencies” 1 to determine what visualizations are available andwhich are commonly employed by open source developers. We an-alyzed a broad set of projects including many types of dependencyvisualizations in computing, not only those used by package managers.Some projects offer multiple visualization features. Of the projectswith visualization features (224 total of 483 surveyed), 57.6% use someform of hierarchical layout node-link diagram. Other common represen-tations were force-directed node-link diagrams (23.2%) and indentedor node-link trees (15.2%). We also found a few instances of othervisualizations showing tree structure (sunburst, treemap, flamegraph,CodeCity), as well as a Sankey diagram, an arc diagram, and a chorddiagram. Force-directed layouts and chord diagrams do not capturethe DAG structure of package dependency graphs. Trees are useful forsome package manager tasks (Sect. 3), but our work is motivated bysupporting graph topology tasks (Sect. 3.2) that are not well supportedby the existing indented trees.

Of the projects in our Github survey, 48.2% use GraphViz in someform, with the most common usage (65.7%, or 31.7% of all projectswith visualization features) being to output a dot format file withdocumentation suggesting rendering with GraphViz’s dot algorithm.The next most common tool was d3js, in use by 21.0% of the projects.ASCII indented trees were found in 9.8% of the projects. Aside fromone ASCII graph tool (“ascii-graph”) described below and a singleview based on git similar to the one described in Sect. 3.1, only theASCII indented tree and the non-rendered graph description files (e.g.,dot, graphML, GEXF) can be viewed from the command line without aseparate application, but these do not explicitly show the topologicalgraph features of interest.

Graph Drawing in ASCII. ASCII flow chart tools were initially cre-ated to aid print documentation of computer programs. In these depic-tions, nodes take the form of outlined boxes with labels inset. FlowChar-ter [18] broke programs into chunks of six to seven nodes at variouslevels of abstraction in creating flow charts. Knuth [24] printed eachbox in sequence vertically and routed orthogonal edges along the side.

There are many tools to assist manual creation of ASCII node-link diagrams for electronic documentation, such as Emacs Artist [1],ASCII Flow [20], and AsciiO [19], but fewer that do automatic layout.Graph::Easy [48], vijual [4], and ascii-graphs [38] are generaltools for automatically producing static ASCII layouts with similarnode styles to FlowCharter and the diagrams of Knuth. As one ofour design goals is to represent the graph topology compactly, wedid not want to use a boxed node style. Furthermore, of the three,only ascii-graphs offers a hierarchical layout that would match thecharacter of our package dependency graphs.

The software reverse engineering framework radare2 [2] can dis-play code branching graphs and call graphs in ASCII. Similar to ourapproach, it is based on a layered graph algorithm and provides inter-activity. It was the only other ASCII graph representation we foundthat did so. However, we determined the depictions and interactivity

1See supplemental materials for methodology, charts describing all views,tools, and formats observed, and a complete list of projects surveyed.

Table 1: Summary of Support for Desired Features

Feature graphterm dot git log ascii-graphs radarePDFs graph

displays in Yes No Yes Yes Yesterminal

matches graph Good Good Poor Good Poorsemantics

compact Good N/A Good Fair Fair

interactive Yes No No No Yes

were not suitable for the character of our package dependency graphsor the tasks of interest on them. Fig. 2a shows the radare2 call graphof a program we wrote to mimic the dependency graph of python aspackaged by Spack (Sect. 3). Fig. 2b shows our depiction of the samegraph for comparison.

General image to ASCII converters [29, 32, 45] focus on emulatingtone differences across an image and thus are a poor fit for node-linkdiagrams. Xu et al. [53] present an algorithm for converting vector lineart to ASCII, but their method requires several minutes to generate animage, which is unacceptable in our workflow. Furthermore, none of thegeneral ASCII conversion methods preserve the structural meaning ofthe elements, and thus would require a post-processing step to identifythe vertices and edges to support interactivity.

In Table 1, we summarize the most suitable approaches and theirsupport for desired features in representing package dependency graphs.

General Graphical Layout Approaches. Many layered layout algo-rithms [44] and orthogonal layout algorithms [46] use graphical marks.For a more in depth discussion of these algorithms, see Tamassia [47].We select a layered layout for conversion to ASCII as described inSect. 4, finding the structure matches both the tasks and expectationsof our users. For example, layered layouts make clear the directionof dependencies by position. Flow direction is not a priority in manyorthogonal layouts. The optimally compact grid layout of Yoghourdjianet al. [54] balances directional flow requirements with other orthog-onal aesthetic criteria and typographical content constraints by usingmultiple directions for flow. We chose to not relax flow direction asthey do in order to preserve user expectations (i.e., vertical dependencydirection), as an evaluation of graph drawing aesthetic criteria for UMLdiagram comprehension [34] suggests domain semantics play an im-portant role. Dwyer et al. [11] use grouping techniques to decreaseedge clutter in directed graphs, making them more compact. However,these techniques use marks to denote group containment and we areconstrained in the number of marks we can legibly fit in a terminalwindow when using ASCII.

3 THE SPACK PACKAGE MANAGEMENT SYSTEM

We provide background on package management systems and in partic-ular, Spack [14], the package management system used in this work.We describe the current visualization support in Spack. Finally, weperform a task analysis on package dependency graphs that leads to ourvisualization design.

Package management systems (or package managers) are softwaretools that aim to ease software installation and maintenance. Theterm package refers to a particular software application, its relateddigital artifacts, and the information necessary to automatically install,update, and configure it2. Such information includes the other softwarewhich must already be installed. We refer to these required softwarepackages as the target package’s dependencies. When installing apackage, the package management system will traverse the dependencyinformation and install any dependencies required to build and run thetarget software.

2The term package is also used in languages such as Java to describe classorganization. We do not address that type of package in this work.

Fig. 4: ASCII indented tree showing python dependencies in Spack.Each line includes the hash of the package configuration, the depen-dency type, and the package name, version, compiler, configuration,and architecture.

A package’s dependencies may themselves have dependencies,

A B

C

target

Fig. 3: Depen-dency graphof target.

some of which may overlap with each other. Thus,the relations among a package and its dependenciesform a directed acyclic graph (DAG). Fig. 3 showsa small example in which the target package directlydepends on packages A and B. Packages A and Bin turn both depend on package C. This is knownas a diamond dependency. In the simple case, thepackage manager need only to ensure packages A,B, and C are installed before installing the targetpackage. However, complications arise when packages A and B requiredifferent versions of package C.

Spack is a package management system designed to streamlinethe process of building software when multiple versions of depen-dencies may be needed. It is motivated by scientific software onsupercomputers—shared systems where different users have differ-ent requirements not only in terms of software versions, but also in thecompilers and build options used. Spack makes it easier to concurrentlybuild and maintain packages that depend on different versions and buildconfigurations of the same software.

Dependencies in Spack may be fulfilled by one of multiple otherpackages. For example, many supercomputing programs depend on themessage passing interface (MPI) [36]. Several implementations of thisinterface exist. Spack may choose one or the user may specify whichone Spack should build against. System-dependent options, such asMPI availability, along with updates to the package database requireusers to regenerate dependency graphs they wish to analyze to ensureup-to-date versions that reflects the state of their system.

Access to supercomputing resources is typically via shell, acommand-line interface. Developing and testing on supercomputersis essential as supercomputers are the primary targets of large-scalescientific software. Thus, package creation, maintenance, debugging,and distribution is done entirely at the command line, as described bythe Spack package creation tutorial 3.

3.1 Visualizing Dependencies in Spack

The Spack spec command shows dependencies of a package as anindented ASCII tree. When more information about the build, such asversions and compilers, are known, these are shown in the tree as well(Fig. 4). The tree view emphasizes what a package depends on (i.e., thepackage’s dependencies). In many situations this is satisfactory. How-ever, some tasks (described in the next section) benefit from viewing theother direction—which packages are affected by a package (i.e. thosepackages which depend on the target packages). Sometimes an even amore general overview of the dependency structure is desired. In thesecases, Spack augments the dependency tree by providing two utilitiesfor inspecting dependency graphs: a dot format description of thegraph and an ASCII representation based on the git log --graphcommand. We briefly describe the git log --graph command andthe pre-existing Spack ASCII representation.

The git command git log --graph provides an ASCII represen-tation of repository commits and their branching behavior. It placesno more than one commit per row with the most recent commit (of thecurrent branch) at the top. The sequential connections between twocommits are shown using edges drawn using |, , /, and \. Fig. 5a

3http://spack.readthedocs.io/en/latest/packaging guide.html

(a) (b)

Fig. 5: (a) Result of git log --graph --online --all and (b)git-style python dependency graph.

shows an example. The left-most vertical line shows commits to themaster (initial) branch. Three more branches are created and merged.

Spack’s graph command adapts the git log --graph algo-rithm [42] to show the package dependency graph. Unlike commits,package dependencies do not have a temporal order, so a topologicalsort is used instead. This places the package of interest on the first rowwith its direct dependencies on the following rows. Fig. 5b shows theSpack dependency graph of python drawn with this git-like approach.

The edge colors denote which dependency the edge leads to. Thisencoding helps users track an edge across several rows. The sixteenANSI colors (the eight original and eight high intensity versions) areassigned to the packages in a round-robin fashion.

There are two major advantages to this git-like layout. First, thelayout conserves columns, thus fitting in the 80 column limits preferredby some command line users. The conservative use of columns canalso be a downside as the resulting graph is visually dense. Second, thegit-like layout is an unambiguous representation: when one edge isrouted into another, they both exit at the same terminus.

While this heavily vertical style matches well with the pure text gitlog command, the temporal nature of git commits, and the propensityfor long chains in the resultant graph, it obscures the layered nature ofdependency graphs. Furthermore, the git-like graphs require manyrows, requiring users to scroll even for relatively small graphs. TheSpack maintainers we collaborated with (see Sect. 3.2) wanted a morecompact representation that used fewer rows.

Fig. 6 shows the git-like dependency graph for dia, a packagewith 39 dependencies (also depicted using our approach in Fig. 1).Visually tracking some edges can require several page-up operations.While the edge coloring can help users keep their place, as these areassigned before edge layout, sometimes the same or similar colors crossor appear side-by-side as with the two long red edges in Fig. 6.

3.2 Task Analysis and Abstraction

We seek to improve the dependency graph visualization for Spack. First,we consider how dependency graph visualizations are used. Througha series of interviews with two Spack maintainers (one via a video-conference, four text chats, and an informal in-person discussion, theother via an informal in-person discussion), we identified three usergroups and their tasks with respect to dependency graphs. We focusspecifically on graph-related tasks, as other tasks, such as simply view-ing the set of package dependencies or viewing only what a particularpackage depends on (rather than what depends on it), are already wellsupported by Spack’s indented tree listing (Fig. 4).Audience and Their Tasks. There are three classes of people who con-sult Spack dependency graphs: Spack power users, package developers,and Spack maintainers/contributors. We discuss their goals below andrelate them to the task taxonomy for graphs of Lee et al. [27]. Wesummarize our classification in Table 2. This analysis was reviewed

Fig. 6: git-style package dependency graph of dia (also shown inFig. 1). The freetype node has been duplicated to show alignmentbetween the two halves.

after development by the Spack maintainer we had the most contactwith, who is among the authors of this paper.

Spack users may refer to a package dependency graph to understandrelationships between the other packages their target software dependson and thus what optional constraints they may want to specify, suchas a particular parallel runtime library. Knowing which other packagesmay be affected can be useful in their decision-making process. Con-sider a power user ‘Yulia’ trying to install a scientific package withwhich she has a passing familiarity. Yulia favors the parallel runtime im-plementation provided by GroupX because it has yielded performanceimprovements for her on a previous project. However, she understandsother packages are known to perform better with the application suiteof GroupY. She wants to examine the graph to assess the potentialtrade-offs in choosing particular implementations. Recognizing thesesituations requires identifying direct and indirect connections and gain-

Table 2: Tasks Abstraction for Spack Dependency Graphs

Graph Task Spack Task Spack Role

topology – · Determine packages affected by · users,accessibility a package developers

· Identify diamond dependency · maintainers

browsing – · Find source(s) of a dependency · developersfollow path

overview · Assess trade-offs among options · users· Assess complexity to judge · maintainers

performance

attribute-based · Identify dependency type or install · all(all) configuration

ing a sense of how all the connections work together, in other words,“topology – accessibility” tasks, “overview” tasks, and “attribute-based”tasks in Lee et al.’s taxonomy.

In addition to performing the power user’s tasks for debugging,package developers may examine dependency graphs to verify theyhave included all the necessary build information in their package.Unexpected dependencies in the graph may indicate an inadequatespecification. Tracing a path from the dependency up to its sources,as motivated by a particular error message, can help resolve an error.Consider a package developer ‘Devon’ who is testing his package on asupercomputer accessible by many of his target users. He tries multiplepossible configurations and is surprised by some of the ways in whichSpack resolves the dependencies. Devon consults the dependency graphto understand why some configuration choices affect other packagesin the graph. These tasks utilize both the identification of direct andindirect connections (task: topology – accessibility) as well as followinga path (task: browsing – follow path).

Spack maintainers analyze package dependency graphs when debug-ging, adding features, or testing. Consider a Spack maintainer ‘Mabel’who is investigating a report of the Spack system failing to build apackage due to choices it made while resolving dependencies. Mabelwants to check that the dependencies truly exist and if they conflict witheach other in a way Spack was unable to determine. She consults thedependency graph, looking in particular for diamond dependencies andother instances of multiple dependencies as these present difficultiesto the dependency resolution algorithm. This is again a task aboutidentifying connections (task: topology – accessibility).

In addition to investigating bugs, Mabel wants to evaluate the per-formance of the package management system. Should she notice aparticular package takes a long time, she can look at the dependencygraph to gain a sense of the complexity of any package installation(task: overview). Information about the version, compiler, and optionsselected for each package, as well as the type of dependency (e.g.,required for build only, called by the target, or linked by the target) isalso of interest (tasks: attribute-based (all)).

As all users perform topology-based tasks on dependency graphs,node-link diagrams are an appropriate choice to represent them. Wenote that indented trees may be more suitable for some Spack userscenarios where all packages depending on a particular choice neednot be found and indirect connections need not be evaluated. Thesescenarios are already served by Spack’s indented tree feature, whichalso includes rich attribute information. The node-link diagram aug-ments this functionality allowing the exploration of more complicatedtopology-based tasks.Data. There are over 2,100 packages in the Spack library as of June2018. The majority of them have dependency graphs with fewer than 50nodes. Fig. 7 shows the cumulative distribution of graph sizes by nodecount. Node-link diagrams are effective for representing graphs thatare relatively small in size and emphasize topology-based tasks [17,23]and thus we use them to represent package dependency graphs.

0 20 40 60 80 100Number of Nodes

0.0

0.2

0.4

0.6

0.8

1.0

Cumulative Distribution of Spack Dependency Graph Sizes

Fig. 7: Most Spack dependency graphs have fewer than 50 nodes.

Workflow. Spack is a command line tool. Spack users install packagesthrough the command line. Package developers use the Spack com-mands spack create and spack edit at the terminal to create andmaintain their packages and then test them on their target supercom-puters via remote login. Similarly Spack maintainers test and debug

the system on the command line via the same interface. As one of themotivations for Spack was streamlining the build process on supercom-puting systems, much of this command line access is to a remote, andoften secure, system that may not have graphical applications such as agraphical web browser installed. Furthermore, users may have limitedprivileges and be restricted to command line access.

Typically, to view a graphical visualization, users must shift focusback to their local machine, copy the file from the supercomputer,and launch a local viewer. This adds several steps to the process andtakes the user away the rest of their analysis. Utilities like rsynccan streamline the file copy process when many packages need tobe analyzed in a workflow session. While this may be the case forSpack maintainers, it is not so for users and package developers. Inthe case that graphical applications are available, the user may viewthem via X11 forwarding if they had the foresight to login with thatoption enabled. Depending on the system and their location, this optioncan induce significant lag. For example, we launched a PDF viewerwith a graph through X11 forwarding on one of our users’ systemsand incurred a penalty of greater than 10 seconds to launch the viewerand then on the order of 1 second for operations such as redrawing forpanning, expanding a drop-down menu, and re-sizing.

During our interviews, the users expressed the desire for an ASCII-based visualization for use on the command line that was more ‘dot’-like than their current one (3.1). We discussed the possibilities of aninteractive, browser-renderable tool, especially to support the multi-variate attribute-based tasks, but reception to that proposal was luke-warm. The users were more interested in a console-based tool thatwould help the majority of their smaller analysis tasks first. Given thecommand line workflow, the overhead of one-off copying of files, thebarriers to using graphical tools, the small size of the graphs, and theneed to support topological operations, we agreed that initial request ofthe users was viable and proceeded to design an improved visualizationof package dependencies as ASCII node-link diagrams.

4 graphterm

To support the command-line workflow for Spack community members,we developed graphterm 4, a Python tool for interactive ASCII depen-dency graph visualization. graphterm adapts a layered graph layoutthat uses graphical marks to create a layout using ASCII characters.Unlike the git-like layout, graphterm bundles edges, resulting inambiguity. Users can resolve the ambiguity via interactive highlighting.We describe the design and implementation of the layout of graphtermin Sect. 4.1. We then discuss the supported interactivity in Sect. 4.2.

4.1 graphterm Layout

From our discussions with Spack maintainers (Sect. 3.2) we concludedthat the users conceptualize dependency graphs in a fashion similar to alayered graph layout [44]. One Spack maintainer specifically requesteda “dot-like” layout in ASCII. Furthermore, we concluded that such alayout supports their tasks. Users can quickly determine the directionof dependencies by vertical order, unlike in an orthogonal layout whichmay be more adaptable to ASCII, but does not maintain a vertical orhorizontal ordering of nodes.

The layout algorithm (Algorithm 1) starts by obtaining a graphicallayered layout. Based on the mark placement and induced crossingstherein, it generates a corresponding set of node and edge positions ona grid (Algorithm 2, Algorithm 3). Finally, ASCII characters from theset { |, , /, \, o, X } are placed to represent those nodes and edges(Algorithm 4). We then place node labels to fit (Algorithm 5).

Converting graphic layout to an ASCII grid. We considered severalexisting hierarchical and layered graph layouts to adapt to ASCII. Theconsidered layouts included dot, dagre [33], and the hierarchicallayouts included in Tulip [3] and OGDF [9]. We chose the Tuliphierarchical layout [3] as a basis because it is a straight-edge layoutthat uses mostly vertical and diagonal edges with a propensity to re-usevertical edges in our package dependency graphs. This resulted in less

4http://github.com/kisaacs/graphterm

(a) mkfontdir graph with graphical marks. (b) mkfontdir graph with ASCII marks.

Fig. 8: The incoming edges of util-macros and pkg-config cross several others in the original layout. We re-route those that cross thevertical edge of zlib to be horizontal but vertically offset from each other by package. util-macros and its neighbors are highlighted in thegraphterm depiction, demonstrating the depiction of the incoming edge from mkfontdir.

Algorithm 1: GraphTerm Layout Overviewgraphterm (G);

positions = Tulip hierarchical(G);crossings = get edge crossings(positions);positions += get bundle positions(G, crossings); // Algorithm 2

xset = [], yset = [];for x, y in positions do xset.insert(x), yset.insert(y) ;rows, columns = to grid points(G, xset, yset); // Algorithm 3

grid = place on grid(G, columns, rows); // Algorithm 4

grid = place labels(G, grid); // Algorithm 5

clutter in comparison to other layouts as well as marks that were easierto adapt to ASCII.

We run the Tulip hierarchical layout5 on the package dependencygraph. From the graphical layout, we obtain initial positions for allnodes as well as line segment end points for all edges.

First, we determine sets of x and y positions of note in the layout—the positions of the nodes, the end points of the line segments, and thepositions of the crossings. The layered nature of the layout induces asmall set of y values. In the Tulip hierarchical layout, dummy nodesare inserted into the graph at the pre-existing layers to aide in edgerouting and within-layer node ordering. The location of the dummynodes correspond to the non-node endpoints of the line segments thatcompose each edge. Combined with the true nodes, these induce asmall set of x values. We will ultimately convert these floating point xand y values to a discrete compact grid (Fig. 9), but first we must addvalues to the sets to account for edge crossings.

We compute the location of all line segment crossings in the graphi-cal layout. In an early design, we split all segments at their crossingsand added the crossing x and y positions to our sets. While this servedto spread out dense sections of the graph, thereby making it possible tofollow each edge, it also expanded the needed grid size unnecessarily.The afforded space for dense crossings also detracted from commu-nicating the overall structure of the graph. Thus, in regions with alarge number of crossings, we selectively re-route segments to sharecrossing points. We found a good heuristic was to re-route the incomingdiagonal edges of (dummy) nodes.

Adjustments for edge re-use and bundling. Package dependencygraphs often have a few nodes with either a high in-degree or a high out-degree. In our chosen layout, this results in several diagonal segmentsfanning in or out between two layers. Many of the edge crossings arecaused by these structures. The large number of segments with slightlydifferent angles is also difficult to represent with our limited set of

5The figures and study in this paper use the Tulip Python bindings. Theversion released on Github ports the layout into pure Python. The port differsin some of its sort orders and vertical spacings and thus can produce a slightlydifferent ASCII layout.

Algorithm 2: GraphTerm Edge Bundling: Shifts crossings of anode’s incoming segments if any cross another vertical segment.get bundle positions (G, crossings);

dummy nodes = get endpoints(G.links.segments);all nodes = {G.nodes, dummy nodes};bundle points = empty map();for node in all nodes do

for segment in in segments(node) doif crosses vertical(segment) then

offset factor = 0.5 × node.x / G.max x;routed y = (node.y - segment.y1) × offset factor;shift crossings(in segments(node), routed y,

crossings);break();

endend

endreturn crossings

glyphs (ASCII). In the absence of edge crossings, our choice of linedrawing heuristic, described later in this section, results in re-use ofhorizontal segments created by underscores, similar to the re-use ofvertical segments in the original graphical layout.

When we detect crossings between a diagonal segment and a verticalsegment, rather than adding the crossing position to our x and y sets,we alter the y crossing position to a set value based on the end (dummy)node of the segment. The altered y position, calculated in lines 7-8 ofAlgorithm 2, uses the end node’s x position to guarantee uniquenessin the presence of other such nodes with the same y value, choosinga position (routed y) between the node and the top of the segment(segment.y1). The procedure routes all diagonal segments to that(dummy) node through the same y value, thus avoiding increasing ourset of y values for each crossing. In the case of crossings between twodiagonal segments, if such a y value has been set by a diagonal-verticalcrossing, we use it. If two such y values exist, we omit the crossing.Otherwise, we use the computed (true) crossing value.

Fig. 8 shows our re-routing rules as applied to the dependencies ofmkfontdir. Several edges from the left cross the vertical edge intozlib. Our algorithm re-routes them, resulting in horizontal edges atdifferent heights to util-macros and pkg-config. However, thediagonal edges crossing directly below xproto are not re-routed. Wefound applying the re-routing to those edges resulted in dense andoverly boxy graphs that resembled grids. Our policy was chosen tobalance the compactness of the depiction with readability.

Gridding and Character Set. Having calculated the set of unique xand y values representing (dummy) nodes and crossings, we assigneach value to a row or column of a grid. This grid will become ourASCII representation. Conceptually, we consider the upper left corner

Algorithm 3: Translate Real Positions to Grid Pointsto grid points (G, xset, yset);

xlist = sort(xset), ylist = sort(yset);columns = empty map(), rows = empty map();col = 0, row =0;for x in xlist do columns.map(x, col), col += 2; ;for y in ylist do

rows.map(y, row), row += 2;if ∃ node in G | node.y = y then row += 2 ;

endreturn rows, columns

\

o|__ __

/

o|

___

|_ ___

| | |\/

y0'y1y2y3y3'y4

y0

Fig. 9: We assign values from our position set to the upper left cornerof each character cell. Rows with nodes are assigned an extra y value.In this fictional example, the top node spans grid lines y0 and y0′ andthe second node spans y3 and y3′ . No other rows are doubled.

of each monospaced character cell to be a grid coordinate. We thenchose ASCII characters to represent edges between them, deciding onthe set { |, , /, \, X }, preferring the underscore to the dash becauseits end point is closer to the grid corner. As the crossings are at the gridcorners, we do not use + which would be a crossing mid-cell.

A mark cannot be drawn at the corner of four character spaces, sowe place the mark for a node (o) in the cell itself. Therefore, any yvalue associated with at least one node is given two grid spaces. Fig. 9demonstrates this assignment.

To keep the ASCII layout compact, rather than maintain the relativedistances between positions, we assign the values in order in our gridcoordinates subject to some expansion function. In our implementation,we use a 2× multiplier to prevent the graph from becoming too dense,as shown in Algorithm 3. Therefore each x value is assigned to suc-cessively numbered even columns. The y values are assigned similarlywith the added row for nodes described above.

Edge Layout. Once the correspondence between graphical layoutpositions and grid points are set, we assign ASCII characters to thegrid cells. Note that segments may span many grid points and thusmany grid cells. We assign nodes (o) and purely vertical segments first.Vertical segments require only successive vertical bar (|) characters.

There are several options for drawing the non-vertical segments. Weinitially used the grid cells calculated by Bresenham’s line drawingalgorithm [7], but this results in unnecessarily crooked “lines”, clutter,and cell collisions. Instead, we break each segment into 0◦ (horizontal),45◦/135◦ (diagonal), and 90◦ (vertical) pieces summing to the effectivedisplacement. Edges that traverse more horizontally have horizontaland diagonal pieces but no vertical pieces. Edges that traverse morevertically have diagonal and vertical pieces but no horizontal pieces.We draw either the excess horizontal (with underscores in the cellabove) or vertical (with vertical bars) first, then the diagonal withslashes. Examples are shown in Fig. 10. The translation is described inAlgorithm 4. This drawing scheme leads to straighter segments, whichhave been shown to support path finding [50]. It also tends to naturallyoverlap edges, effectively coalescing edge marks along main horizontaland vertical thoroughfares.

Assigning ASCII characters to connect the segments can result incollisions. Different segments may require a different ASCII characterin the same grid cell. We appeal to the Gestalt principle of continuationto resolve these collisions. Using line breaks in edges at crossings in

Algorithm 4: Place ASCII Marks in Gridplace on grid (G, columns, rows);

order = {‘|’: 1, ‘ ’: 2, ‘\’: 3, ‘/’: 3, ‘X’: 0, ‘ ’: 4};grid = initialize to spaces();for node in G do grid.place(’o’, columns[node.x], rows[node.y]) ;for segment in G.links do

r, c, r2, c2 = transform coords(segment, columns, rows);diagonal = min((r2 - r), abs(c2 - c));vertical = max(0, (r2 - r) - abs(c2 - c));if c < c2 then slash = ‘\’; cdiag = c2 + diagonal ;else slash = ‘/’; cdiag = c2 - diagonal ;if vertical > 0 then

for i = r to r + vertical doif order[‘|’] < order[grid[c][i]] then grid.place(‘|’, c, i) ;

endr = r + vertical + 1;

elsefor j = c to cdiag do

if order[‘ ’] < order[grid[j][r - 1]] then grid.place(‘ ’, j, r - 1);

endc = cdiag + sign(c2 - c);

endfor i = r to r2 do

if order[slash] < order[grid[c][i]] then grid.place(slash, c, i) ;if order[slash] = order[grid[c][i]] and grid[c][i] 6= slash then

grid.place(‘X’, c, i);endc += sign(c2 - c);

endendreturn grid

\

_

|

__ _\

________

\\

||

Fig. 10: Graphical lines are converted to ASCII first by excess verticalor horizontal displacement then by diagonal displacement. The left lineis more vertical and thus has no underscores. The right line is morehorizontal and thus has no vertical bars.

this manner has been previously shown to have little effect on readabil-ity [39]. We observed that giving precedence to slashes over verticalbars and vertical bars over underscores works to preserve segment con-tinuity. When two opposing slashes conflict, we use an X character (asseen in Fig. 8b).

Labels. After the edges have been drawn in the grid, we place thelabels. Ideally, the labels would be placed close to their node. However,we consider the sense of graph structure and the compactness of itsrepresentation higher priorities. If there is enough empty space to theleft or the right of the node, we place the package name in that space.Preference is given to the right side to match the git-like depiction(Sect. 3.1) where all labels are on the right. Also, this allows the user tofollow a link to a node (o) and then continue reading from left to rightto see the label.

If there is not enough space for the package name, we place the labelto the right of the entire graph. The right-most unlabeled node per rowis drawn next to the graph. The rest are drawn in a bracketed list to theright of that label in the left-right order of the nodes. Brackets are usedto distinguish this list from the other labels. The rightward placementis again in deference to the git-like layout. The procedure is outlinedin Algorithm 5.

Algorithm 5: Place Labelsplace labels (G, grid);

for row in grid dofor node in row do

s = node.label;if place right of node(s) then continue() ;else if place left of node(s) then continue() ;else if is last in row(node) then place right of graph(s) ;else append to right bracket(s) ;

endendreturn grid

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

100

200

300

400

500Maximum Bracket Lengths (Labels)

Fig. 11: Histogram showing the distribution of the maximum numberof labels relegated to the bracketed list amongst the Spack packagedependency graphs. There is a peak at six due to a large number of Rlibraries. At the time of this experiment, the R package, a subgraph ofthese libraries, produced a layout with a length six list.

We considered balancing the extra labels on the left and right side,based on their position in the grid. However, this moved the structureof the graph further to the right which would require users to shift focusfrom the cursor which rests in the lower left corner.

We remark the bracketed list is not ideal, but a trade-off made toemphasize graph topology. One problem is that long lists will betruncated by the edge of the terminal window and require panning.While one could construct a package with an arbitrarily long list ofnodes on the same level, in practice we observed the length of thelongest list grew with the number of nodes in the graph and mostgraphs had a maximum list of six or fewer labels (median: 1 label,average: 2 labels). Fig. 11 is a histogram of maximum bracket lengthacross Spack packages.

4.2 graphterm Interactions

We design our interactions controls to match common command lineprograms. We do this exactly when possible or by metaphor when not.Searching and highlighting a specific node is done by typing the forwardslash character followed by the name, as done in less. We expect usersmay search for a node when they are considering specifying a versionor compiler and want to consider how that choice may affect packagesdepending on that node. Developers and maintainers may want tosearch for a node to verify its connections when debugging.

Users may traverse the nodes (via highlight) in grid order with then and p keys, similar to jumping between found matching strings inless. These interactions support examining multiple nodes or gainingand understanding of the edge coalescing for graph overview tasks.

With the exception of the arrow keys, which are not available in allterminals, we did not find consensus for directional movement. Thus,we provide both arrow keys and the set {w, a, s, d} for panning shouldthe graph not fit in the viewable area of the terminal. The latter setwas chosen for its prevalence for inputting directional movement invideo games. Searching for a node also automatically pans the graph to

(a) Direct connections highlighted.

(b) All connected nodes and paths highlighted.

Fig. 12: Two highlighting styles: (a) only direct connections (singleedge paths), and (b) showing all connected nodes (multi-edge pathsincluded). In (b), bison, m4, help2man and the edges to them arehighlighted in addition to udunits2’s direct neighbors.

ensure the node is on screen. Zooming is possible on some terminalsby changing the font size.

In addition to helping users disambiguate bundled edges and as-sociate nodes with labels, we expect the automatic highlighting ofconnected edges and neighbors to a node (as in Fig. 1, Fig. 8b, Fig. 12a)to help with connectivity and accessibility tasks like those describedin Sect. 3.2. Highlighting has been shown to aid users in these visualgraph queries [49]. Users may toggle the highlighting to highlight allnodes with a path to or from the highlighted node instead, along withthe edges in those paths (Fig. 12b).

In interactive mode, graphterm exploits the entirety of the terminalwindow via the terminal-independent ncurses text interface library.Upon quitting interactive mode, the graph is printed to the terminal inthe state it was last shown, with the exception that all rows are printedrather than only what would fit in the terminal’s display.

5 STUDY

Our goal in designing graphterm is to provide dependency graphvisualization in the context in which people need to analyze them, atthe command line, so as to benefit their workflow. In our discussions,Spack maintainers also expressed a strong preference for a visualizationthat could be used at a terminal. However, as discussed in Sect. 3.1 andSect. 4, the ASCII representations have limitations not present in thegraphical space. We conduct a study to observe the efficiency trade-offsbetween the visualizations as well as user preferences when visualizingdependency graphs during a command-line workflow.

Sensalire et al. [40] proposed a taxonomy of software visualizatintool evaluations. The taxonomy consists of ten dimensions (S1–S10):(S1) tool selection, (S2) participant, (S3) tool exposure, (S4) task se-lection, (S5) experiment duration, (S6) experiment location, (S7) ex-periment input, (S8) participant motivation, (S9) participant relation totool designers, and (S10) analysis of results. We note the relation ofour design to thise taxonomy throughout.

We design a within-subjects study comparing the graphterm, git-like, and GraphViz dependency graph visualizations. We hypothesized:

Fig. 13: Study questions, answers, and operations all take place at thecommand line. Part of the graphterm training phase is shown here.

H1. Participants who use command line interfaces will prefer ASCIIrepresentations.

H2. Participants who use command line interfaces will perform tasksmore quickly using ASCII representations when working at thecommand line.

H3. There will be no significant difference in accuracy among thevisualizations, but participants will be more confident in theiranswers when using GraphViz.

Tool Selection (S1). We chose GraphViz for comparison because it iswidely used, can generate image files from the command line, and isalready suggested by Spack. In addition to the git-like ASCII graphs,Spack can output a graph in dot file format. The Spack documentationsuggests its use with GraphViz to generate a PDF file.

Koschke [25] surveyed software engineers and found their mostcommonly used graph layout program was dot. Our survey of Githubrepositories (Sect. 2) showed the most common approach for generatinga visualization of dependencies was outputting a dot format file forrendering with GraphViz. We conclude from these findings that Spack’suse of dot and GraphViz is in-line with current practices and thereforeit is appropriate to compare this GraphViz-based workflow to the ASCIIvisualization workflows. For our study, we preserved the command linesuggested by Spack, which was similar to those suggested by many ofthe Github projects.

Spack’s suggestion of rendering dot to PDF (rather than to PNG)was also kept because PDF readers have built-in functionality for search-ing and highlighting text, such as the label of a node in the graph, whichwhile not required by the tasks, could be used to help locate nodes spec-ified by the questions.

We kept all of the graph style attributes written into the dot file bySpack. We removed the hash ID from the node labels however as thesemight confuse participants unfamiliar with Spack.

As we want to investigate the utility of the package dependencygraph visualizations for the workflow of people using the command line,it was essential that the study be conducted at a command line interface.Our study software displays instructions, tutorials, and questions on theterminal and accepts answers via terminal commands. Fig. 13 shows

the interface. Participants were allowed to use the command line asthey wished, with the exception that our software prevented them fromvisualizing the wrong graph or using a disallowed method (e.g. usinggraphterm during the GraphViz block). This limitation applied onlyto the spack graph command—participants could render dot files toPNG or look at the raw text format if they chose to do so. Similarly, aswe wanted to better emulate real-world Spack workflow, we used Spackto generate the graphs rather than relying on pre-generated graphs.

To measure the efficacy of the workflow, we chose to measure re-sponse time in terms of the entirety of the workflow—from questionto response, this allowed the user to perform other operations as theywished, as they would be able to in a real setting.Study Design. Our study is divided into four blocks, one for each visu-alization workflow (‘tool blocks’) and an additional final block whereparticipants could use the visualization of their choice. The three toolblocks contained five questions each while the user-preference blockcontained three questions. General command lines (“usage”) weregiven with each question. In the user-preference block, the questionswere prefaced with the phrase ‘Using any method’ and command linesfor all three methods were displayed.

The order of the tool blocks and the question order within them wasrandomized. A brief tutorial describing package dependency graphswas displayed at the beginning of the study. Before each tool block,an explanation of the particular visualization scheme was given, twointeractive sample questions with answers were presented, and partic-ipants were encouraged to experiment with the tools and given a listof package names with which to do so. At the end of the study, thesoftware asked the participants the open-ended questions:

• “Which graph type do you prefer and why?”

• “What features made the graphs hard or easy to understand?”

The final question solicited any additional comments. Participantsrequired between 45 and 65 minutes of active time to complete theexperiment (S5: Experiment Duration).Task Selection (S4). Participants were asked two types of questionsbased on the tasks determined in Sect. 3: direct connectivity and pathcounting. The direct connectivity questions ask which packages dependon a given package in the graph (affected by connectivity). This opera-tion occurs when a user is determining which compiler or version toset for a package and how that may affect the packages that depend onit. The path counting questions ask how many paths exist between twonodes of a given graph. This operation can be helpful for debuggingor developing new features for the package management system asmaintainers try to understand why the system resolved dependencies ina particular way. All questions were open-response to better match realworld workflow. We decided against multiple choice questions as wewere concerned they would enable process of elimination and guessing.Experiment Input (S7). We used different graphs for all four blocksto avoid memory effects. To balance the difficulty across the threetool blocks, we chose graphs with similar vertex counts, edge counts,layers, and nodes per layer. Table 3 summarizes the chosen graphsfor the tool blocks as well as the preference block6. The type ofquestion used for each is also listed. Note for the largest graph size,we asked both types of question for the same graph due to the limitedselection at that size. To further balance difficulty, we created questionstargeting similar layers in each graph with similar answers in terms ofnumbers of nodes or paths. We chose several graphs per block to testa variety of graphs without incurring tedium and fatigue, which wasreported by participants during piloting. All graphs were generatedwith the --normalize option in Spack to ensure the same structureacross different systems.Experiment Location (S6). To reach a wide audience of participants,we provided both in-person and online options for accessing the study.There were three online options: (1) loading files on a system on which

6See supplemental material for further descriptions of the graphs and ques-tions.

many Spack users have shell access, (2) running the study within aprovided virtual machine, or (3) running the study in a Docker [30]container. The in-person and virtual machine versions had defaultterminal configurations that were screen height, had black backgrounds,and had all other settings left with the system default (e.g, font sizes).We did not do any special configuration on the visualization programs orany PDF or image viewer. Participants were free to alter configurationssuch as window size and zoom level at any time, including during thetraining phase.

Table 3: Dependency Graph Characteristics for Study

Block(s) # Nodes # Edges Question Type

Tool 11 22 Paths

Tool 17-18 26-27 Dependencies

Tool 22 45 Paths

Tool 33-34 60-62 Paths, Dependencies

Preference 19 27 Paths

Preference 31 62 Dependencies

Preference 40 79 Paths

We verified the time each visualization required was approximatelyequal between graphs. We measured the time to produce a layout foreach experimental object (in random order) seven times on the machineused for the in-person sessions. We chose to compare median time aswe observed the distribution of the seven timings was usually either afew milliseconds different or an entire second different. The GraphViztime included the time to render to PDF, but not to open the PDF aswe allowed participants to use any viewer they chose. The graphtermtime was measured using printing to stdout instead of interactivemode. All experimental objects of approximately the same node counttook the same time (in rounded milliseconds) with the exception of the40 node Preference block graph and the 33-34 node Tool block graphwhere GraphViz took 1 second longer (three trials took the same timeas the other visualizations, four took one second longer).

The version of graphterm used in the study relied on the TulipPython bindings. The self-contained version currently on Github portsthe subset of the Tulip hierarchical layout used by graphterm. ThisPython-only version is significantly faster both because of the reducedalgorithm and not having to connect to the Tulip interface. Our mea-surements were done using the study version.

Participants (S2, S8, S9) and Tool Exposure (S3). We recruitedparticipants from both the Spack community (including maintainers,package developers, and users) and our organizations’ computing de-partments. Nineteen people completed the study. We asked participantsabout their command line usage during the initial questions of our study.Of them, we discarded any results where the participant could not an-swer the majority of questions in any of the three tool blocks correctly(two participants), where the participant indicated they never use thecommand line (one participant), or where the participant indicated theydid not understand what a path was after training (one participant).This left us with 15 responses (ten men and five women; eight between18-25 years of age and seven between 26-45 years of age). Six peopleparticipated remotely and nine in person. Three of the participants(S1, S2, S3) were Spack users. S1 was also a Spack contributor. S2indicated they had written small Spack packages.

None of the participants were involved in the analysis or develop-ment presented in this paper. None had seen or used graphterm before.Of the 15 participants, 7 had used GraphViz before and 14 had usedgit. Both groups included all the Spack users. We did not ask whetherthe participants were familiar with the git log graph command orwhat their motivation for participating was.

6 RESULTS AND ANALYSIS

We analyze the data collected during our study. As the goal of our studywas to assess efficiency and preference (S10: Analysis of Results) incomparison to existing approaches, we assess response time, accuracy,and confidence across the three visualizations using a linear mixedeffects model. We then examine the preferences of our participants asexpressed directly and through tasks. We identify and review commonthemes communicated by the participants about the visualizations.Finally, we discuss our conclusions based on these results as wellas limitations of the study.

Fig. 14 shows the measured response time and correctness for thetool block responses of each participant, organized by the graph sizeand question type. Fig. 15 summarizes the spread of average participantresponse time, error rate, and confidence scores and shows preferencesreported by participants.

We measured response time from when the question was shownthrough when the answer was entered. Based on our pilot, we omittedany single question time longer than five minutes (shown clamped at300 seconds in Fig. 14). This is generally an indication the participanttook a break from the study. In total, we removed four question re-sponses all from different users: two using GraphViz PDFs, one usingthe git-like visualization, and one using graphterm. In one response,the participant indicated they had left and returned, so we updated thetime recording as noted. In scoring the answers, we accepted packagenames with typos (e.g., ‘xrpoto’ instead of ‘xproto’) as all instanceswere unambiguous with respect to other names in the graph.

To account for within-subjects variance and missing trials, we builta linear mixed effects model, previously used in the visualization com-munity by Liu and Heer [28]. We model the visualization used andwhether the sample was collected in-person or remotely as fixed effects.We model the size of the graph in question (node count) as a randomintercept term. As we expect different reactions to the visualization byparticipant, we model participant as a random slope term modulatedby the visualization used. We assess significance with a likelihood-ratio test [52] using a reduced model without the visualization term.Post-hoc analysis on significant findings is performed using Tukey’sall-pairs comparisons with a Bonferroni-Holm correction. For theseanalyses, we use the R [35] lme4 [5] package for the model and themultcomp [21] package for post-hoc analysis of significant findings.

ASCII Visualizations Resulted in Decreased Response Times. Wefound a weak effect for visualization method on task completiontime, χ2(2,N = 15) = 5.812, p < 0.1, with summary coefficients of-26.89 seconds for the git-like visualization workflow and -12.59 sec-onds for the graphterm workflow, indicating tasks were completedmore quickly in the ASCII visualization workflows. Post-hoc analysisshowed significance in the difference between the git-like visualiza-tion and GraphViz PDF visualization (p < 0.05). This finding partiallysupports H2, that participants would perform tasks more quickly withASCII-based workflows when working at the command line.

Analyzing the response logs, we noted Participant S2 wrote detailedcomments with their answers in the graphterm block, increasing thetime recorded for response in that block. Also, some participants real-ized that as one graph was the subject of two questions, they need notre-create the GraphViz PDF. They simply re-opened it which resultedin a savings of Spack dependency resolution time, graph rendering time,and possible memory effects.

ASCII Visualizations Resulted in More Errors. We found a signifi-cant effect for visualization method on incorrect responses χ2(2,N =15) = 6.8848, p < 0.05 with summary coefficients of -0.206 for thegit-like visualization workflow and -0.094 for the graphterm work-flow, indicating more errors occurred for the ASCII-based workflows.Correctness was recorded as a binary value with 1 for correct and 0 forincorrect. Post-hoc analysis showed a significant difference in errors be-tween the git-like method and the GraphViz PDF method (p < 0.05).This finding is contrary to the first part of H3, that there would not be asignificant difference in accuracy among the workflows.

We analyzed the response logs for trends and explanations. Partici-pant P5 and P6’s answers in both ASCII visualizations suggest they did

11-P

17-D

22-P

33-D

33-P

Participant P1 Participant P2 Participant P3 Participant P4 Participant P5

11-P

17-D

22-P

33-D

33-P

Participant P6 Participant P7 Participant P8 Participant P9 Participant P10

0 50 100 150 200 250 300

11-P

17-D

22-P

33-D

33-P

Participant P11

0 50 100 150 200 250 300

Participant P12

0 50 100 150 200 250 300

Participant S1

0 50 100 150 200 250 300

Participant S2

0 50 100 150 200 250 300

Participant S3

Fig. 14: Response time and accuracy results for study participants during the tool blocks, organized by graph size and question type (path P ordependencies D). Time is shown in seconds on the horizontal axis. Response time includes visualization rendering time. The blue trianglesare graphterm-aided responses, the green circles are responses using the git-like depiction, and the orange squares are responses with theGraphViz-rendered PDFs. Smaller marks indicate incorrect responses. Response times greater than 300 seconds are clamped to the plot edge andwere not included in the statistical analysis.

0 50 100 150

GraphViz PDF

git-like

graphterm

Average Response Times

0.0 0.2 0.4 0.6 0.8

Error Rate

3.0 3.5 4.0 4.5 5.0

Average Confidence

0 2 4 6 8 10 12

4

1

12

Preference Counts

Fig. 15: Summary plots of participant average response time, error rate, and confidence scores along with reported preferences. The error rateboxplot has overlapping outliers (3 at 0.2 for GraphViz, 1 at 0.8 for the git-like, and 2 at 0.6 for graphterm). Preference values sum to greaterthan 15 as some participants reported multiple preferred workflows.

not understand the implied edge direction by orientation. ParticipantsP5 and P6 made six of the total ten errors in the graphterm worfklowand four of the 23 total errors in the git-like workflow. ParticipantP7 was randomly shown several path counting questions in a row dur-ing the graphterm block. The dependency questions afterward wereanswered with the (correct) number of dependencies rather than thenames and thus were marked incorrect. Participant P10 made similarmistakes in the git-like block. These mistakes represented two of theten errors made with graphterm workflow and two of the 23 errorsmade with the git-like workflow. In contrast, only three errors weremade using the GraphViz PDF workflow.

ASCII Visualizations Resulted in Lower Response Confidence. Wealso found a significant effect for visualization method on responseconfidence χ2(2,N = 15) = 7.163, p < 0.5 with summary coefficientsof -0.607 for the git-like visualization workflow and -0.283 for thegraphterm workflow, indicating participants were less confident intheir answers using ASCII visualizations. Participants were asked torate their confidence on a scale of 1 to 5 after each question. Post-hoc analysis showed a significant difference between the git-likevisualization workflow and the GraphViz PDF workflow (p < 0.01).This finding supports the second part of H3, that participants would bemore confident when using the GraphViz PDFs.

graphterm Was Preferred By Participants. During the preferenceblock, eleven of the fifteen participants used graphterm most fre-quently, with nine using only graphterm, including all participantswith Spack experience (S1, S2, S3). Two used the GraphViz PDFsexclusively, one used the git-like and GraphViz PDFs equally, andone accessed graphs in a way our system was unable to record (thisparticipant reported preferring graphterm).

When asked directly at the end of the study, eleven of the fifteenparticipants indicated that they preferred graphterm to the other twovisualizations, again including all participants with Spack experience.Participant S3 wrote “after using the --term option, I doubt I willever use any other graph format except when printing it out: for printing,--dot is better.” Of the other four participants, one indicated they likedboth graphterm and GraphViz PDFs, another liked both the git-likevisualization and GraphViz PDFs, and two preferred only the GraphVizPDFs. One of the participants who preferred the GraphViz PDFs issuedthe caveat “if I had more practice with the term graphs, I feel as if thatwould be more efficient and clear.”

The results of the preference block and the preference questionpartially support H1, that participants would prefer the ASCII rep-resentations. While participants showed preferences for graphterm,preference towards the git-like representation was minimal.

Several participants answered questions in the preference blockcorrectly, even when they had shown less accuracy with the chosendrawing method in its initial block (S2, P7, P9). While P7 can beattributed to misreading the earlier questions, the increased accuracyfor S2 and P9 may indicate learning over the course of the study.

6.1 Themes in Participant Responses

We discuss common themes found in comments made by the partici-pants, both in response to the open-ended questions at the end of thestudy and any comments typed during the study in their answers.

Use of Space. Participant S2 stated an important factor was the “useof screen space both vertically and horizontally,” noting there is morehorizontal screen space to spare. Participant P7 preferred graphtermfor its “good use of horizontal spacing.” Participant P2 thought theaspect ratio of the git-like visualization was detrimental.

The rank-based nature of all the layouts was considered useful byParticipant S1, who noted “The top-to-bottom direction also reallysimplifies things, in all the graphs.”

Scrolling. Many participants said scrolling was detrimental. This wasmost prevalent for the GraphViz PDF visualization (S1, S2, P2, P4, P5,P11), but also for the git-like visualization (S1, P4). Participant S1remarked “its tedious to find nodes and move around the pdf.”

Ambiguity. Several participants expressed some difficulty with theambiguity of edge connectivity in the graphterm layout (S1, S2, P8),but said the highlighting helped (S1, S2). Some participants notedthat understanding edge connectivity was also tricky in the GraphVizPDF (S1, S2, S3, P4) due to crossings, but Participant P11 indicatedthat it was easier as no edges branched like they do in both the git-like visualization and graphterm. While Participant P6 preferredgraphterm and GraphViz PDF to the git-like visualization, theystated none of the tools were easy to understand.

git-like visualization. Several participants indicated the edge color-ing helped them trace paths in the git-like visualization (S1, S2, P2,P4, P5), but two experienced difficulty discerning some of the colors(P8, P11). The density of the git-like visualization was considered anegative (S2, P2, P5, P11). Participant S2 wrote “my initial reaction tothe large git graphs was a viscrecal [sic] - I do not want to look at thisat all.”

GraphViz PDFs. Two of the participants who preferred the GraphVizPDFs (P3, P5) said that the arrowheads clarified what the dependencieswere. Participant P3 also said the direct labeling of nodes was helpful.Though participant P11 preferred graphterm, they also remarked theyliked the arrows in the GraphViz PDFs.

Three participants expressed distaste for using a PDF reader (S1, S2,P9). Participant S2 wrote that “having to load pdfs is annoying.” Whilesome participants during piloting reported using their PDF reader’stext search to locate a node, this was not reported during our full study.Instead, participants described having difficulty finding a node in theGraphViz layout (S1, S3, P10). Some participants suggested modifyingthe default GraphViz PDF rendering to have bigger fonts (P8) andshorter edges (S2, P11).

graphterm. Many participants cited the interactive highlighting ofgraphterm as a key feature (S1, S2, P2, P6, P7, P8, P9, P11, P12).Two participants (S1, P11) suggested enhancing the highlighting modesto color differently for direction or extend by neighborhood.

As expected, the distance between the node and the label was foundconfusing by participants (S2, P11). Participant P11 wrote “The factthat multiple node names on the same line get grouped together isannoying.”

Some participants indicated the use of the terminal as another reasonfor their preference (S1, S3, P9). Participant P9 wrote that graphterm“was just RIGHT there, straightforward to use especially when youjust want to use the command line the whole time.” Participant S3disliked that GraphViz “is not terminal based” and liked graphterm’s“keyboard based navigation.”

6.2 DiscussionBased on our quantitative measures, we observe that the workflowusing the git-like ASCII visualization leads to faster response timesfrom the command line than workflow using the GraphViz PDFs, butat a cost to accuracy and confidence. While not statistically significant,the workflow with graphterm seems to fall between the two existingSpack dependency graph visualizations on these three measures, fromwhich we infer the graphterm workflow is a viable alternative to theexisting Spack graph offerings.

Accuracy is a significant concern when making build decisions.We note none of the three options were strictly error-free. While theGraphViz workflow resulted in three errors total and the graphtermworkflow ten, two of the graphterm errors can be attributed to mis-read questions and six to insufficient training (see Sect. 6.3 below),indicating the error rate in practice may be comparable.

The GraphViz rendered visualizations can directly label the nodesunlike graphterm and are less ambiguous. Yet, the workflow with thegraphterm visualization was more preferred. Based on the commentsby participants, we believe that the major factors leading to this prefer-ence were the terminal-based nature and the interactivity. We interpretthe preference for the terminal (when already working at the terminal)to indicate participants are willing to accept a sub-optimal visualiza-tion that is convenient to their workflow. However, as the workflowwith the git-like graphs were not preferred, this trade-off betweenvisualization and workflow is not absolute. A graphical solution withmore customized interactivity may be preferable to all three presentedoptions. When we suggested such a solution to users during our taskanalysis (Sect. 3.2), they indicated having something at the commandline was of greater interest. The preference results are in line with theinitial assessment of the domain experts.

Some of the issues participants noted in the GraphViz PDFs couldbe addressed by having Spack change the graph layout style attributesit writes into the dot file, such as the font size and edge length changesproposed by the participants. Both of these parameters were alreadyexplicitly written by Spack.

6.3 LimitationsOur findings are limited by the study design as described below.

Task Design and Study Length. We designed our study questionsto test basic graph tasks derived from our task analysis (Sect. 3.2)rather than the more complicated task a real user may have. The morecomplicated tasks often rely on familiarity with software, its options(e.g., available parallel runtime implementations), and personal taste ofthe user (e.g., favored compiler). Some, like debugging Spack itself,require knowledge of the Spack codebase. We expect the more compli-cated tasks will involve several basic graph tasks per dependency graph.The cost of obtaining a graphical representation may be amortized overthis process. Alternatively, the barriers to interacting with such a rep-resentation, e.g. switching between the terminal and another program,may compound.

Furthermore, while our goal was to examine usage coming from acommand line workflow, the repeated visualization of different graphsin sequence likely does not match the workflow of Spack users, whoprobably visualize graphs more infrequently. Another study whichspaces out visualization with other command line tasks may betteremulate reality, but would increase the length of the study.

Study length was likely a factor in our ability to recruit Spack usersas participants, though it was half the maximum time of two hourssuggested by Sensalire et al. [40] when recruiting professionals. Afollow-up study could bypass the graph layout time by pre-computingthe graphs as these operations were equal across all layouts, at the costof realism in the presented workflow.

Effect of Study Setup on Response Time. Participants either per-formed the study on a local machine (11 participants) or were warnedahead of time about viewing multiple GraphViz-rendered files suchthat they might want to access the study with X11 forwarding enabled(four participants). Participants did not experience the scenario wherea separate login operation or file copy was required to view a PDF or

image file. Avoiding these scenarios by either working locally or beinginformed ahead of time may have resulted in quicker task response timewhen using the GraphViz PDFs.

We used GraphViz in our comparison as it is one of the existingoptions supported and suggested by Spack, it is widely used in the sys-tems and software space, and visualizations can be generated from thecommand line in relatively few steps and without users having to learnnew technology. We used the dot specification as provided by Spack.A customized graphical visualization or more style specification in thedot file may lead to better performance. Reminding participants thattext search is a common feature in PDFs may have helped participantswho stated they had difficulty finding nodes. Fully integrating the filecopy and graphical viewer launch may also lead to better performanceof GraphViz, but would require assumptions that limit portability ofSpack or place a per-machine configuration burden on the user. Thiswould increase the cost of the whole of the workflow, especially forusers who consult the graphs infrequently and would not be amortizingthe setup cost.

Effect of Study Setup on Error Rate. The majority of the errors inthe graphterm workflow and several in the git-like depiction work-flow came from participants P5 and P6. The responses were indicativeof not understanding that edge directionality was implied by verti-cal positioning, despite being explained in the training phase of bothASCII workflows. Participants S2 and P9 who made errors during thegraphterm block did not make the same errors using the graphtermworkflow in the preference block, which may indicate improvementover the course of the study. These observations may indicate thetraining was insufficient, leading to increased errors for the ASCIIworkflows.

Effect of Free Response Questions. We chose free response ques-tions over multiple choice to better match a realistic scenario and toavoid guess-and-check behavior. This allowed participants to answerquestions other than what was posed (e.g., answering the number ofpackages instead of their names), increasing the error rate for the ASCIIworkflows. It also lead to increased response times for participant S2during the graphterm block as this participant wrote comments withtheir answers.

7 CONCLUSION AND FUTURE WORK

We have described and validated a visual solution for analyzing Spackpackage dependency graphs for command-line workflows. The solutioncombines ASCII characters with terminal interactivity to provide DAGvisualization that supports topology-centric graph tasks identified inour task analysis and abstraction (Sect. 3.2). Our DAG visualizationsoftware, graphterm, provides a terminal-friendly compact depictionby adapting an existing graphical layout algorithm. These adaptationsinclude selectively re-routing and bundling edges and choice of ASCIImarks. The interactivity leverages idioms from widely-used commandline tools.

We designed a user study to compare the analysis workflow usinggraphterm to the workflows using the two existing package depen-dency visualizations for two of our identified tasks. The study emulatedthe setting in which Spack users work: the command line interface. Par-ticipants in our study preferred the graphterm workflow to the existingapproaches. Based on the responses to this study, we conclude thatthough ASCII-based visualization requires trade-offs not present in thegraphical space, it can be a more preferable visualization solution fromthe perspective of the entire workflow. We hope this demonstrationleads to more consideration of making these stark trade-offs to supportthe analysis of users where they are (in our case, the terminal) and notethere is a lot of opportunity for interactive tools in this space.

Having shown the viability of visualizing small graphs in ASCII, weintend to further improve the layered ASCII graph layout algorithm inthe future. We focused on bundling and re-routing diagonal edges forclarity and to make the graph less tall. We would like to further explorethe affect of the bundling on readability, possibly also consideringbundling and re-routing vertical edges to make the graph less wide.Additionally, we plan to explore shifting nodes so there are fewer per

line, considering unique marks for the nodes, or adding tooltip-likesupport to address issues with node labeling.graphterm was designed with an emphasis on graph topology tasks

that we identified as common across the users of package dependencygraphs. However, we also identified an interest in graph attribute dataamong Spack developers. While some of this information is availablethrough the indented trees, no attempt has been made to address this inthe existing graph features. We plan to address the multivariate designissues in the ASCII space. We will also experiment with adding inter-activity to other Spack analysis commands or implementing supportfor multiple coordinate views to further support these tasks.

8 ACKNOWLEDGEMENTS

The authors thank the Spack community, study participants, and review-ers for their time and aid in this research. This work was performedunder the auspices of the U.S. Department of Energy by LawrenceLivermore National Laboratory under Contract DE-AC52-07NA27344.LLNL-JRNL-746358.

REFERENCES

[1] T. Abrahamsson. Emacs Artist. http://www.lysator.liu.se/tab/artist/, Ac-cessed 2017-03, Updated 2014-03.

[2] S. Alvarez. radare2. http://github.com/radare, Accessed 2017-03.[3] D. Auber. Tulip - a huge graph visualization framework. In M. Junger and

P. Mutzel, eds., Graph Drawing Software, pp. 105–126. 2004.[4] C. Barski. vijual. http://lisperati.com/vijual/, Accessed 2017-06, Last

Updated 2010-01.[5] D. Bates, M. Machler, B. Bolker, and S. Walker. lme4: Linear mixed-

effects models using Eigen and S4. arXiv:1406.5823, 2014.[6] A. Bergel, S. Maass, S. Ducasse, and T. Girba. A domain-specific language

for visualization software dependencies as a graph. In IEEE WorkingConference on Software Visualization, pp. 45–49, 2014. doi: 10.1109/VISSOFT.2014.17

[7] J. E. Bresenham. Algorithm for computer control of a digital plotter. IBMSystems Journal, 4(1):25–30, 1965. doi: 10.1147/sj.41.0025

[8] P. Caserta and O. Zendra. Visualization of the static aspects of software:A survey. IEEE Transactions on Visualization and Computer Grahpics,17(7):913–933, 2011. doi: 10.1109/TVCG.2010.110

[9] M. Chimani, C. Gutwenger, M. Junger, G. W. Klau, K. Klein, andP. Mutzel. The Open Graph Drawing Framework (OGDF), pp. 543–569.CRC Press, 2014.

[10] B. Cornelissen, D. Holten, A. Zaidman, L. Moonen, J. J. van Wijk, andA. van Deursen. Understanding execution traces using massive sequenceand circular bundle views. In Proceedings of the IEEE InternationalConference on Program Comprehension, pp. 49–58, 2007. doi: 10.1109/ICPC.2007.39

[11] T. Dwyer, N. Henry Riche, K. Marriott, and C. Mears. Edge compressiontechniques for visualization of dense directed graphs. IEEE Transactionson Visualization and Computer Graphics, 19(12):961–968, 2013. doi: 10.1109/TVCG.2013.151

[12] J. W. E. Eaton, D. Bateman, S. Hauberg, and R. Wehbring. GNU Octaveversion 3.8.1 manual: a high-level interactive language for numericalcomputations. CreateSpace Independent Publishing Platform, 2014. ISBN1441413006.

[13] T. Gamblin. Spack. https://github.com/LLNL/spack, Accessed 2017-03.[14] T. Gamblin, M. P. LeGendre, M. R. Collette, G. L. Lee, A. Moody, B. R.

de Supinski, and W. S. Futral. The Spack package manager: Bringingorder to HPC software chaos. In SC15: International Conference for HighPerformance Computing, Networking, Storage and Analysis, pp. 1–12,Nov. 2015. doi: 10.1145/2807591.2807623

[15] E. R. Gansner, E. Koutsofios, S. C. North, and K.-P. Vo. A technique fordrawing directed graphs. IEEE Transactions on Software Engineering,19(3):214–230, 1993. doi: 10.1109/32.221135

[16] E. R. Gansner and S. C. North. An open graph visualization system and itsapplications to software engineering. Software – Practice and Experience,30(11):1203–1233, 2000.

[17] M. Ghoniem, J.-D. Fekete, and P. Castagliola. A comparison of thereadability of graphs using node-link and matrix-based representations.In Proceedings of the IEEE Symposium on Information Visualization, pp.17–24, 2004. doi: 10.1109/INFVIS.2004.1

[18] L. M. Haibt. A program to draw multilevel flow charts. In Proceedings ofthe Western Joint Computer Conference, IRE-AIEE-ACM ’59 (Western),pp. 131–137. ACM, 1959. doi: 10.1145/1457838.1457861

[19] K. N. I. Hamouda. AsciiO. http://search.cpan.org/dist/App-Asciio/lib/App/Asciio.pm, Accessed 2017-03.

[20] L. Hemens. ASCIIFlow. http://www.asciiflow.com, Accessed 2017-03.[21] T. Hothorn, F. Bretz, and P. Westfall. Simultaneous inference in general

parametric models. Biometrical Journal, 50(3):346–363, 2008. doi: 10.1002/bimj.200810425

[22] M. Howell. Homebrew, the missing package manager for OS X.http://brew.sh, Accessed November 2017.

[23] R. Keller, C. M. Eckert, and P. J. Clarkson. Matrices or node-link diagrams:which visual representation is better for visualizing connectivity models?Information Visualization, 5(1):62–76, 2006. doi: 10.1057/palgrave.ivs.9500116

[24] D. E. Knuth. Computer-drawn flowcharts. Communications of the ACM,6(9):555–563, 1963. doi: 10.1145/367593.367620

[25] R. Koschke. Software visualization in software maintenance, reverseengineering, and re-engineering: A research survey. Journal of SoftwareMaintenance and Evolution: Research and Practice, 15(2):87–109, 2003.doi: 10.1002/smr.270

[26] R. G. Kula, C. De Roover, D. German, T. Ishio, and K. Inoue. Visualizingthe evolution of systems and their library dependencies. In Proceedingsof the IEEE Working Conference on Software Visualization, pp. 127–136,Sept. 2014. doi: 10.1109/VISSOFT.2014.29

[27] B. Lee, C. Plaisant, C. S. Parr, J.-D. Fekete, and N. Henry. Task taxonomyfor graph visualization. In Proceedings of the AVI Workshop on BEyondTime and Errors: Novel Evaluation Methods for Information Visualization,pp. 1–5, 2006. doi: 10.1145/1168149.1168168

[28] Z. Liu and J. Heer. The effects of interactive latency on exploratory visualanalysis. IEEE Transactions on Visualization and Computer Graphics,20(12):2122–2131, Dec 2014. doi: 10.1109/TVCG.2014.2346452

[29] N. Markus, M. Fratarcangeli, I. S. Pandzic, and J. Ahlberg. Faster ren-dering of image mosaics and ASCII art. Computer Graphics Forum,34(6):251–261, 2015. doi: 10.1111/cgf.12597

[30] D. Merkel. Docker: Lightweight linux containers for consistent develop-ment and deployment. Linux Journal, 2014(239), Mar. 2014.

[31] A. Noack and C. Lewerentz. A space of layout styles for hierarchicalgraph models of software systems. In Proceedings of the ACM Symposiumon Software Visualization, pp. 155–164, 2005. doi: 10.1145/1056018.1056040

[32] P. D. O’Grady and S. T. Rickard. Automatic ASCII art conversion of binaryimages using non-negative constraints. In Proceedings of the Irish Signalsand Systems Conference, pp. 186–191, 2008. doi: 10.1049/cp:20080660

[33] C. Pettitt. dagre. https://github.com/cpettitt/dagre, Accessed 2017-06.[34] H. C. Purchase, M. McGill, L. Colpoys, and D. Carrington. Graph drawing

aesthetics and the comprehensions of UML class diagrams: an empiri-cal study. In Proceedings of the Australian Symposium on InformationVisualization, pp. 129–137, 2001.

[35] R Core Team. R: A Language and Environment for Statistical Computing.R Foundation for Statistical Computing, Vienna, Austria, 2014.

[36] Message Passing Interface Forum. MPI: A message-passing interfacestandard, version 3.1, June 4, 2015.

[37] D. Reniers, L. Voinea, and A. Telea. Visual exploration of programstructure, dependencies and metrics with SolidSX. In Proceedings of theIEEE International Workshop on Visualizating Software for Understandingand Analysis, pp. 1–4, 2011. doi: 10.1109/VISSOF.2011.6069461

[38] M. Russell. ascii-graphs. https://github.com/mdr/ascii-graphs, Accessed2017-06.

[39] A. Rusu, A. J. Fabian, R. Jianu, and A. Rusu. Using the Gestalt Principleof Closure to Alleviate the Edge Crossing Problem in Graph Drawings. InProceedings of the 15th International Conference on Information Visual-ization, 11. doi: 10.1109/IV.2011.63

[40] M. Sensalire, P. Ogao, and A. Telea. Evaluation of software visualizationtools: Lessons learned. In Proceedings of the IEEE International Workshopon Visualizing Software for Understanding and Analysis, pp. 19–26, Sept.2009. doi: 10.1109/VISSOF.2009.5336431

[41] G. N. Silva. APT Howto. Technical report, Debian, 2001. http://www.debian. org/doc/manuals/apt-howto.

[42] A. Simpkins and J. C. Hamano. Add history graph API. git repositorycommit c12172d2eab91b79b8181b04ab5a5332a96e34a8, accessed fromhttps://github.com/git/git June 2017, May 2008.

[43] M.-A. D. Storey and H. A. Muller. Manipulating and documenting soft-

ware structures using SHriMP Views. In Proceedings of InternationalConference on Software Maintenance, pp. 275–284, Oct. 1995. doi: 10.1109/ICSM.1995.526549

[44] K. Sugiyama, S. Tagawa, and M. Toda. Methods for visual understandingof hierarchical system structures. IEEE Transactions on Systems, Man, andCybernetics, 11(2):109–125, 1981. doi: 10.1109/TSMC.1981.4308636

[45] Y. Takeuchi, D. Takafuji, Y. Ito, and K. Nakano. ASCII art generationusing the local exhaustive search on the GPU. In Proceedings of theInternational Symposium on Computing and Networking, pp. 194–200,Dec. 2013. doi: 10.1109/CANDAR.2013.35

[46] R. Tamassia. On embedding a graph in the grid with the minimum numberof bends. SIAM Journal on Computing, 16(3):421–444, 1987. doi: 10.1137/0216030

[47] R. Tamassia, ed. Handbook of Graph Drawing and Visualization. CRCPress, Aug. 2013.

[48] Tels. Graph::Easy. http://bloodgate.com/perl/graph/manual/, Accessed2016-09. Updated: 2007-09.

[49] C. Ware and R. Bobrow. Motion to support rapid interactive queries onnode-link diagrams. ACM Transactions on Applied Perception, 1(1):3–18,2004. doi: 10.1145/1008722.1008724

[50] C. Ware, H. Purchase, L. Colpoys, and M. McGill. Cognitive measure-ments of graph aesthetics. Information Visualization, 1(2):103–110, 2002.doi: 10.1057/palgrave.ivs.9500013

[51] T. Williams, C. Kelley, R. Lang, D. Kotz, J. Campbell, G. Elber, andA. Woo. gnuplot. http://www.gnuplot.info, Accessed 2017-03.

[52] B. Winter. Linear models and linear mixed effects models in R withlinguistic applications. arXiv:1308.5499, 2013.

[53] X. Xu, L. Zhang, and T.-T. Wong. Structured-based ASCII Art. ACMTransactions on Graphics, 29(4):52:1–52:10, 2010. doi: 10.1145/1778765.1778789

[54] V. Yoghourdjian, T. Dwyer, G. Gange, S. Kieffer, K. Klein, and K. Mar-riott. High-quality ultra-compact grid layout of grouped networks. IEEETransactions on Visualization and Computer Graphics, 22(1):339–349,2016. doi: 10.1109/TVCG.2015.2467251

Katherine E. Isaacs is an assistant professor in the Department ofComputer Science at the University of Arizona. Her research interestsfocus on visual techniques for analyzing computing software and sys-tems. She received her Ph.D. in 2015 from the University of California,Davis. Prior to that, she completed a B.S. in computer science and aB.A. in mathematics at San Jose State University and a B.S. in physicsat the California Institute of Technology.

Todd Gamblin is a computer scientist in the Center for Applied Sci-entific Computing at Lawrence Livermore National Laboratory. Hisresearch focuses on scalable tools for measuring, analyzing, and vi-sualizing parallel performance data. For this work, he received anEarly Career Research Award from the U.S. Department of Energyin 2014. In addition to his research, Todd leads LLNL’s DevRAMP(Reproducibility, Analysis, Monitoring, and Performance) team, and heis the creator of Spack, a popular HPC package management tool. Toddhas been at LLNL since 2008. He received Ph.D. and M.S. degreesin Computer Science from the University of North Carolina at ChapelHill in 2009 and 2005. He received his B.A. in Computer Science andJapanese from Williams College in 2002.