Graph-theoretic approaches and tools forquantitatively assessing curricula coherence

Abstract—In this paper, we propose a method toanalyse the coherence of existing curricula at higher ed-ucation institution. We focus our attention to engineer-ing programs at universities but the proposed methodis by no means restricted to those cases. In contrastto other known methods, our approach is quantitative,decentralised, asynchronous, allows to analyse entireprograms (in contrast to single courses) and does notdepend on using specific teaching methods or tools.

We propose to perform this quantitative assessmentin two steps: first, representing the university programas an opportune graph with courses and concepts asnodes and connections between courses and concepts asedges; second, analysing the structure of the programusing methods from graph theory.

We thus perform two investigations, both leveraginga practical case – data collected from three engineer-ing programs at two Swedish universities: a) how torepresent university programs in terms of graphs (herecalled Concepts-Courses Graph (CCG)), and b) how toreinterpret the most classical graph-theoretical nodecentrality indexes and connectivity and network flowresults in order to analyse the program structure,including to discover flows and mismatches.

Index Terms—Concepts-Courses Matrix (CCM),Concepts-Courses Graph (CCG), university programdesign, graph theory, centrality, connectivity

I. Introduction

A. BackgroundDeveloping curricula is a complex task that involves

creating and assessing proposals in different organizationaland decision contexts. There is a significant body of liter-ature dealing with models of curriculum design, e.g., [20].As reported in [10], two established models dominate: theObjectives Model, that starts by specifying the objectivesor learning outcomes defined as measurable performances,see also [3], and the Process Model, that starts by definingcourse content and specifying criteria to assess students’knowledge of this content. Several variations on thesemodels exist (e.g., Tyler’s, Wheeler’s, and Kerr’s Models).

When it comes to the individual courses in a program,a new curriculum might comprise courses that alreadyexisted for example in a different but similar program, andnew courses that are designed for the specific curriculum.Deciding how to segment the material into existing ornew courses often follows a discipline-centered approach.This is likely due to the ease of practical implementation

given numerous books and support material available toaid course designers, e.g., [15, 4, 19].

It is well known that understanding how knowledgewithin a course or across courses in a program is con-ceptualised can provide a clearer basis for creating astructure and progression that better supports studentlearning. Towards reaching this goal, a well known strategyis the so-called black-box approach to the sequencing ofa curriculum [5]. This development tool has been pro-posed as part of the Conceiving, Designing, Implementing,and Operating (CDIO) standard for the management ofuniversity programs and entails representing every coursewithin a program as a set of inputs (e.g., prerequisiteskills and knowledge) and outputs (e.g., contributions tothe final learning outcomes). Coupling this informationwith all courses is expected to enable discussions, makeconnections (or lack thereof) visible, provide an overviewof the program, and eventually serve as a basis for bothplanning and improving. However, this tool is still quali-tative, and does not provide quantitative indications thatare not subject to personal interpretations.

B. Curricula ManagementCurricula management is usually required during the

entire life span of a university program for several im-portant reasons. Even in the best designed and alignedprograms, courses may be moved within the curriculumdue to pragmatical needs (e.g., they do not fit in theschedule from logistical reasons, or there is no teacheravailable) leading to changes in how and which knowledgecan be or is taught. Further, the profiles of the teacherscan, consciously or unconsciously, change the curriculaover time at least gradually.

At the same time, higher education institutions striveto more heavily integrate research-based practices at alllevels from teaching to program design and development.In fact, as pointed out in [18, p. 6], this institution-level transformation is important, and should be reflectedin opportune transformations of the university programs.However, the vagueness and lack of facility to objectivelymeasure the goals of higher education may lead facultymembers to realize and prioritize these goals based ontheir own interpretations [6, 17]. Indeed, this may leadto decisions that are heavily steered by charismatic peo-ple with strong opinions, or affected by board members’lack of time/interest, or by communication difficulties,

especially when program design and management involvescooperation between academics from different disciplinesor traditions.

Yet another problem is that a program board cannothave full control of all parts of a program, much less on howa course is examined. It is also the case that some goalsmay be implicit, unspoken or simply assumed obvious.Moreover, in many cases it is not clear what a specificcourse shall deliver in form of decided (or intended) knowl-edge accordingly, or what is its intended connection toother courses and content of the program.

Hence, in summary we identify the following short-comings in the common practise of program boards tomaintain coherence within a curriculum or part of it: (i)this operation is typically done centrally (by the programboard), which usually cannot collect or use all relevantinformation; (ii) it involves collecting, revising and review-ing information manually, which limits how much data canbe processed, and (iii) it is further done at discrete timeinstances / according to fixed cycles, and this may delayor prevent the detection of mismatches (specially if longtemporal delays are present between the implementationof local changes in courses’ contents and their theoreticalreporting in the next revision of the program).

In some cases detailed information about a course andits contents and detailed data are available, speciallyif the courses are taught through Intelligent TutoringSystems (ITSs), which record when students passed orfailed quizzes, accessed certain material and many otheraspects related to student behaviour and learning. Theinsights are then often used to model the students andderive strategies to better adapt the teaching experienceto the specific student’s needs. The tools are particularlyuseful when teaching large classes such as in MOOCs.However, they are not often used in traditional universityprograms since it requires to transform the course suchthat it can be taught using an online tool (often requiringa complete new set of materials) and having a suitableelectronic tool available, which might not be affordable ordesirable for all universities. However, since ITSs are notwidely used in standard university programs, a curriculummaintenance system should not depend on data fromsuch electronic systems. Moreover, the purpose of ITSsis to adapt teaching material and learning strategies toindividual students. In contrast, a curriculum maintenancesystem aims at improving the quality of curricula coher-ence and maintenance efforts and should also be applicableto any university context, independently of which teachingmethod is used in the different courses.

C. Introducing Quantitative Management

In this paper, we suggest a method that allows for build-ing a “curriculum monitoring system” which is expectedto improve the current method outlined above by being:

• decentralised and collaborative, by directly involvingprimarily the teachers and possibly also students inthe program (a component that also increases theprobability that they will accept and act upon theresults),

• continuous and asynchronous, in the sense enablingto assess the coherence of the program, curriculum orpart of it at any time (i.e., not necessarily being tied tofixed time instances) and as soon as some new dataare available (i.e., asynchronous as not needing thecollection of every data from every courses to alreadystart providing actionable information),

• flexible, in the sense that allows to include coursestaught with a wide range of teaching and learningmethods,

• data driven, in the sense of allowing to analyse thecurriculum, its knowledge flow and its coherencewith computer aided objective tools (compared tosummarising and revising the information manually,potentially including more data than humans canusually handle).

We expect that quantitative methods that can comple-ment classical discussion-based approaches should – atleast intuitively – lead to more sustainable and efficientmanagement of university programs, and lead to the otherancillary benefits described in the statement of contribu-tions below.

We propose a strategy that makes use of conceptsfrom graph theory, such as analysing the node centrality,connectivity and flow in a network that describes howconcepts and courses are related within a program. Ourwork relates thus to existing works that consider graphtheoretic approaches for the management of curriculastudies.

An example of such a method is the one summarizedin [14], where the author collected in her PhD thesis anextensive approach to perform a quantitative analysis ofthe structure of curricula by means of four distinct graphsthat could be used to create a study plan for students. Theobjectives of her body of work are on helping teachers andstudents create personalized study plans for individualizedstudents learning through graph theory. This means thatthere is no explicit focus on how to help teachers under-stand how to compile these graphs, collaborate to compilethem, and use the results to foster alignment among thestakeholders.

The approach presented in [1] was used to analyze con-nections between courses according to curriculum struc-ture in order to understand the coherence and structurewithin a university program. This work focuses on definingand analysing the network of prerequisites of the variouscourses, with the purpose of understanding how their rolescan differ within a curriculum. In other words, the workfocuses only tangentially on how to use the informationprovided by these networks for taking actions. Moreover,the important questions of why courses are prerequisites

for other courses and what is learned in the courses is notconsidered here.

Another paper connecting learning goals (intended asdesired results), topics, and courses within a programis [13]. Here the authors model learning goals, topics andcourses as nodes, and model prerequisite dependencies asedges, so that the relation between courses and topics arerepresented as edges. Graph analysis techniques are thenutilized to measure several aspects of a curriculum; theoverall information is then used to perform automated cur-riculum design operations (e.g., to automatically generatethe draft of course syllabi). It is important to note thatthe focus of this paper is more on the design of curricula,rather than on estimating the current situation. More-over the described approaches do not include proceduralinformation explaining who does what from a practicalperspective (in a sense, not giving ”instructions” on how topractically replicate the approaches in other universities.

We also report [12], that proposes to use graphs-orientedapproaches to answer key questions on where to focus theassessments, data collection, and corrective actions withinthe curriculum. The focus here is thus on how to analyzesystematically and quantitatively the program contents soto help faculty and administrators that are tasked withcreating quality assurance and assessment schemes.

With respect to the graph-theoretic approaches for rep-resenting university program contents discussed above, wefocus on implementing a collaborative and team-basedstrategy for obtaining graph-oriented representations thatcan help aligning expectations and language speciallyamong the teachers, and between teachers and students. Ina sense, we are more concerned with creating conditionsthat favor the implementation of constructive alignmentparadigms. We thus devise a strategy that is a bottom-up approach to obtaining graph-oriented representationsof university programs.

Structure of the manuscript: Section II describes thetools for collecting and representing quantitative infor-mation about a generic university program. Section IIIdiscusses how classical node centrality indexes and graphconnectivity can be interpreted and applied in our contextof university programs analysis. Section IV reports andexamines the results obtained from field applications ofthe proposed method. Finally, Section V presents someconcluding remarks and suggests some future research anddevelopment efforts.

*

II. The Concepts-Courses Matrix and theConcepts-Courses Graph

To build a tool that teachers can use in a bottom-upfashion to quantitatively evaluate and analyze the struc-ture of the university programs related to their courses,we exploit the intuition that courses within a programare connected through a flow of concepts that are taughtthroughout the duration of a given program. This intuition

TABLE I: Part of a Concepts-Courses Matrix taken fromthe field case of Electrical Engineering, academic year 2017/ 2018, Uppsala University, Sweden.

1TE705 1TE704 1MA008 1TE667Intro Components Algebra & El. Circ.

to El. Eng. & Circuits Vector Geom. Theory

complex num. 2 2vectors 2 1

sys. of lin. eq. 2 2Ohm’s law 2 2

Kirchoff’s laws 2 2pot. voltage 2 2

linearitymatrices 1 1 2 2

work, energy 2 2int. calculus 1

is common especially in engineering disciplines, whereconstructivist interpretations of knowledge tend to prevail.

To highlight the connections among courses, we suggestto follow a procedure based on executing two separatesteps: acquire the data, as described in Section II-A, andvisualize the data, as described in Section II-B.

A. Data acquisition through the Concepts-Courses Matrixtool

The simplest teachers collaboration strategy that wedevise is to let them define a table, in the following denotedas Concepts-Courses Matrix (CCM), where the columns/ rows correspond to the courses / concepts within theprogram (see Table I for an example). A CCM thusallocates one column per course j and one row per conceptk, so that the value of each (k, j)-th element may be usedto quantify how relevant the concept k is for course j ona predefined scale. As for which scale to use, as indicatedin [9] a simple option consists in “0” = not relevant, “1”= somewhat relevant but not central, and “2” = veryrelevant / central for the course. A more refined optionmay instead consist in associating a number to each levelof an opportune taxonomy (e.g., Bloom’s), and then leteach (k, j)-th element in the CCM be either 0, if conceptk is not related to course j, or the number correspondingto the learning level that students should ideally reachabout the k-th concept when they successfully concludecourse j. Note that, despite being more informational,using taxonomy levels would require all the collaboratingteachers that define these tables to be acquainted with theconcept of taxonomy and share the same interpretation ofthe meanings of the various levels.

Building a complete CCM for a specific program orpart of it at a specific department requires the personscollaborating on the creation of the CCM executing twosteps: a) defining which concepts shall be included, and b)interviewing (also through internet-based tools) expertsthat may give indications on the values of the elements inthe matrix.

As for step a), a natural strategy is to first build aninitial list of important knowledge by inspecting each

course description in the program (or part of it), andreceive feedback on this from the board and the teachersinvolved in the program (or part of it). A different optionmay be to consider which questions are asked in thevarious exams of the various courses.

As for step b), one may exploit several possibilities:Inputs from the teachers: One option is to collect

relevant data for each individual course from the teacherteaching that specific course, as she/he can be regardedas an expert on her/his particular subject (so that she/hecan often produce such information with ease). However,in our experience we noticed that this strategy has severaldisadvantages. First, teachers might be unwilling to spendtime on preparing such data, especially when the CCMcomprises a long list of concepts, or if they do not see im-mediate benefits from doing so. Further, such informationwill inevitably reflect what each teacher thinks or desiresto teach in her/his course. This information might be quitedifferent from the perception of both students and otherteachers or the effects actually achieved in terms of whatstudents learn. Strategies to mitigate these subjectivedistortions are proposed and analyzed in Section IV below.

Inputs from the students: To complement and vali-date the inputs from the teachers we consider also askingstudents to provide information on the various courses thatthey have been taking. This information may however bedistorted, too. For instance, students might not alwaysunderstand how concepts, facts and other knowledge areinterrelated or which knowledge is a prerequisite for agiven course. To do so, a metacognitive understandingof the course is required, but in our experience this doesnot happen for all the students and often only after sometime of reflection. We expect that some of these issuescan be resolved or attenuated by averaging over the datafrom several students. One might also consider combiningthe information from students with inputs from teachersor other sources. But great care should be taken whenintending to do so. Indeed, despite asking the same ques-tion to students and teachers, namely, which concepts wererequired or developed by a given course, their answers willdiffer due to their perspectives. Teachers will likely answeraccording to what they intend to teach, also influencedby their usually much wider metacognical understandingand reflection of the material. In contrast, students willanswer according how they understood and perceived thecourse. In fact, rather than averaging, comparing the datafrom teachers and students might be a more viable tool todetect possible mismatches between teachers’ intends andstudents’ perceptions.

Analysis of the exam questions: Another option (notexplored in our field tests) is to inspect which questions areasked in the exams of the various courses, and link thoseto the list of concepts created in step a). This informationwould also complement the teacher input about her/hisIntended Learning Outcomes (ILOs). This approach mightbe particularly useful at European universities, since ac-

cording to the Bologna process, see for example [8], examquestions should be closely connected to, and examine thelearning outcomes, i.e. course goals. Hence, these may bea valuable source to categorize the ILOs as well as thecourse goals.

Formal sources: As another option, formal sourcessuch as analysing course goals or interviewing programboards or study directors might reveal information aboutwhich requirements higher authorities intend a courseto fulfil. However, in some cases, this might be wishfulthinking or it might be difficult or impossible to extractrelevant information as authorities might not have suffi-cient insights into all courses. Further, course goals mightbe formulated unsuitably.

Note that all information sources listed above lead tosubjective data. Indeed, objective data on which conceptsare included in which course do not exist. However, inorder to minimize the influence of subjective opinions,several sources of information should be combined weigh-ing the data in an appropriate manner and consideringthe possible bias of each source. In our study we followedmainly the option “inputs from the teachers”.

B. Data visualization through the Concepts-Courses Graphtool

After obtaining the CCM described above, the programcan be represented as an undirected weighted bipartite1

graph, denoted as a Concepts-Courses Graph (CCG). Thetwo sets of nodes in this graph correspond to the coursesk and the concepts j within the program. The (k, j)-element in the CCM corresponds then to the weight ofthe edge between the concept node k and the coursenode j (e.g., see the example in Figure 1). Intuitively, theproperties of a university program (e.g., its structure, therelations and the relevance of the courses and concepts ina program, the existence of potential flaws in its design)should translate into opportune topological properties ofthe CCG. If this intuition is true, then the problem ofquantitatively analyzing a university program can be castas the problem of analyzing the opportune graph. Theproblem of understanding what can actually be inferredabout a university program through analyses of its CCGis discussed in Sections III and IV.

C. Extending the CCM and CCG tools to distinguishprerequisite information from taught information

The CCM and CCG tools defined and described abovemay be structured in a more complex way so to includemore information. For example, one major shortcomingof the CCM and CCG tools above is that they do notembed why a concept is relevant for a course. For instance,

1In the sense that its nodes can be divided into two disjointindependent sets such that all edges connect a node from one of thesesets to the other. We moreover here use “network” in the sense ofa collection of agents and connections between them which can berepresented as a graph. For definitions and a comprehensive study ofgraphs and graph theory, please refer to [7].

1TE705

1TE704

1MA008

1TE667

Intro toEl. Eng.

Components& Circuits

Algebra &Vector Geom.

El. Circ.Theory

complex num.

vectorssys. of lin. eq.

Ohm’s law

Kirchoff’s lawspot. voltage

linearity

matriceswork, energy

int. calculus

Fig. 1: Example of a CCG corresponding to the CCM inTable I.

TABLE II: Example of part of a Directed Concepts-Courses Matrix taken from the field case of Computerand Information Engineering, academic year 2017 / 2018,Uppsala University, Sweden. (T = ‘taught’ in the course,R = ‘required’ by the course)

1DT051 1DL201 1DT093information data computer

technology structures architecture

recursion 1 T 2 Tdivide & conquer 1 T 1 T

induction 1 T 1 Rdata structures 1 T 2 T

trees 1 T 2 Tlists 1 R 2 T

graphs 1 Tarrays 1 R 1 T

hashtables 1 T 2 T

a concept could be relevant because it is a necessaryprerequisite to fruitfully follow a course, or because it isan ILO developed and taught in the course. Collectingthis type of information allows more detailed analyses.To enable these, however, one has to let the CCM havetwo columns for each course: one allocated for weightsto quantify the relevance of prerequisite concepts (e.g.,the learning levels that students should ideally have tobe able to follow fruitfully the course), and the other oneallocated for weights of concepts taught or developed inthat course (e.g., the learning levels that students shouldideally reach about that concept after having passed thecourse). Alternatively, one may indicate with an opportunesymbol whether a specific concept is required or taught bya specific course. A field example is provided in Table II.

Correspondingly, one may extend the CCG defined inSection II-B by letting it become a directed, weighted,bipartite graph where edges that are directed from conceptnodes to course nodes indicate that the concepts arerequired for that course, while edges that are directed fromcourse nodes to concept nodes indicate that the conceptsare taught by that course. The resulting graph, denoted asa Directed Concepts-Courses Graph (DCCG), would thenlook like in Figure 2. Note that combining this additional

1DT051 1DL201 1DT093

recursion

divide and conquer

induction

data structures

trees

lists

graphs

arrays

hashtables

Fig. 2: Example of a DCCG corresponding to the DCCMin Table II, where for simplicity the weights of the edgeshave been omitted, and where moreover the course nodesare sorted temporally ascending from left (i.e., earliercourses) to right (i.e., latter courses). The dashed redarrows indicate concepts that are considered prerequisitesfor a given course, but taught only by following courses.

information on what are requirements and what are courseoutcomes can be used to, e.g., discover when early coursestreat a given concept as prior knowledge despite it beingonly introduced in a later course. This occurrence is forexample highlighted with red arrows in the field-exampleof Figure 2.

Our experience indicates that great care should begiven on how to graphically represent a DCCM, since itconstitutes a representation that can be very informativefor the stakeholders (i.e., boards, teachers, and students).Up to now, our choice has been to follow these guidelines:• plot course nodes on the bottom, in a temporally

ascending order;• plot concept nodes in vertical piles that lie between

course nodes, and let the positions of these piles in-dicate whether concepts are prerequisites or teachingoutcomes of the various courses.

More specifically, we propose to divide concepts in twosets: a) the ”problematic” ones, i.e., that concepts forwhich there exists some early course that treats them asprior knowledge despite they are being introduced onlyin a later course (like the ones reached by the dashedred arrows in Figure 2), and b) the non-problematic ones.Given a non-problematic concept, we place it:• just before the first course that has it as a prerequisite,

if such a course exists (e.g., ”induction” in Figure 2);• otherwise, if no courses has that concept as a pre-

requisite, then immediately after the last course thatteaches it (e.g., ”recursion” in Figure 2).

As for problematic concepts, such as ”arrays” and ”lists”in Figure 2, our proposal is to plot them in the positionthey would be if we were ignoring the edges that makethem problematic. In our experience this strategy enablesseeing the whole program as a temporal flow and helps dis-cussions on the program structure. Other representations

are possible, but at least in our limited experience theproposed one has been the most beneficial for the purposeof aiding constructive meetings.

III. Data analysis methodsIn this section we assume to have collected enough

information so that, for a given university program, boththe relative CCM described in Section II-A and the cor-responding CCG in Section II-B have been compiled. Dueto the special structure of these tools (i.e., a matrix anda graph), one can cast the problem of analyzing the prop-erties of the program into the problem of analyzing theproperties of the corresponding matrix or graph by meansof well known and established tools from matrix and graphtheory. Our purpose is then to discuss what these estab-lished tools say about potential structural problems of therepresented programs. The section is divided in three mainparts: Section III-A, discussing the pedagogical meaning ofseveral nodes centrality measures; Section III-B, discussingthe pedagogical meaning of standard network connectivitymeasures; and Section III-C, discussing the pedagogicalmeaning of cycles within a directed CCG.

A. Centrality measuresIt can be revealing to understand how “important” or

“central” certain courses and concepts are in a program,since this may give indications on which courses and con-cepts should receive special attention (e.g., in the form ofadditional students learning monitoring actions). To thisaim, we notice that several well-established node centralityindexes exist in the literature (see, e.g., [7]).

Formally, we thus let the CCG be defined as the graphG = {V, E}, where V = {1, . . . , S} is the set of nodescomposing the graph and E ⊆ V×V is the set of the edgesbetween the nodes. To every edge (i, j) ∈ E corresponds anassociated edge weight wij ≥ 0. Since we are consideringthe situation described in Section II-B, G is static (i.e., nottime-varying) and undirected (i.e., (i, j) ∈ E ⇔ (j, i) ∈ E ,wij = wji). Given this, the set of neighbors of a genericnode i is defined as Ni := {j | (i, j) ∈ E}. Finally note thatV is bipartite, i.e., V = Vcourses ∪ Vconcepts, and (i, j) ∈ Emust be so that either i ∈ Vcourses and j ∈ Vconcepts or viceversa.

1) Degree centrality: This centrality index is one of thesimplest ones, being the sum of the weights of all edgesconnected to a particular node, i.e.,

d(i) =∑

j∈Ni

wij . (1)

In our pedagogical-oriented interpretation of G, the degree-centrality indicates the weighted sum of how many coursesa particular concept is relevant (connected to) or howmany concepts are taught or are connected to a particularcourse with the weight being related to the importance onthe scale {0, 1, 2}. However, this metric is very sensitive tohow the connections are weighted. For instance, consider

that, to compile the CCM as we indicated in Section II-A,teachers have to add “1”s or “2”s to the concepts relevantto their courses. As we noticed in our field tests, differentteachers have different compilation approaches (i.e., moreconservative teachers might think that the compilationshould focus just on core concepts, and hence assign lessoverall weights than other teachers that assume that thecompilation should be “exhaustive”) . What we noticed,thus, is that the degree centrality scores are very sensitiveto the personality of the various teachers, which is nota desirable property. In order to compensate for thiseffect, one may then think that weights can be adjustedor normalised; however, normalising weights so that theaccumulated weights of each course add up to an assignednumber (e.g., the credit points associated to that course orsimply 1), then the degree will inevitably be this number,so that the metric loses its value. Hence, weights shouldbe chosen with great care and establishing a commonunderstanding among teachers on how to assign weights isessential to retrieve meaningful insights from this specificmetric.

2) Closeness centrality: This metric is intuitively de-fined as the average length of the shortest path between aspecific node and all other nodes in the graph. Formally,for a connected graph, it corresponds to

c(i) = 1∑j dist(i, j) (2)

where dist(i, j) is the (geodesic) distance between nodei and j, which is the length of a shortest path betweenthe nodes and setting the length of the edge from k to l to

1wkl

, see also [7]. Closeness roughly expresses how close theparticular node is to the remaining nodes of the network.When used in our context, a pedagogical interpretationmay be how logically “far” a certain course or concept isfrom other courses or concepts. In other words, if i is acourse node, and i has a very high closeness index, thenit means that it is focusing on concepts that are used byseveral other courses. When i is a concept node, this indexcorresponds to how diverse the related concepts are, e.g.,if they span over a large group of different subareas of theoverall concept repertoire or not. When used for concepts,we expect this metric to be high for concepts that aretaught or brought up in courses whose contents span overthe entire program. Consequently, concepts that are onlytaken up by some courses or only during some time periodsof the program are expected to be less close.

3) Eigenvector centrality (a.k.a. eigencentrality): Thisindex assigns relative scores to all nodes in the networkbased on the idea that connections to high-scoring nodescontribute more to the score of a particular node thanconnections to low-scoring nodes with the same weights.Formally, the measure is defined as

e(i) = 1λ

∑j∈N (i)

wije(j) (3)

where λ is a constant and turns out to be the largesteigenvalue of the adjacency matrix. A possible pedagogicalinterpretation for this measure is a quantification of howinfluential a course or concept is within the program.We expect this metric to give meaningful insights intothe program structure for two main reasons: First, it islikely to be high for courses that cover many concepts thatare also (very) relevant in (many) other courses. Second,this more complex measure goes well beyond what canbe achieved by simply adding weights or manual analysisof the data, and is expected to be less sensitive to theteachers’ personal interpretations of how to compile theCCM. Hence, we expect this measure to offer insights thatcan complement the ones that more intuitive and simplermetrics may give.

4) PageRank centrality: This is an adaptation of theeigenvector centrality discussed above, which assigns dif-ferent scaling factors to the edges. More precisely, it isdefined as

p(i) = α∑

j∈N(i)

wijp(j)∑

k∈Njwjk

+ 1− αN

(4)

where the attenuation factor satisfies α ∈ (0, 1). Due tothe similarities with the eigencentrality, we expect theresults from using this metric to be of similar usefulness. Adiscussion on the practical differences that we notice usingthe two metrics is given in Section IV.

5) Betweenness centrality: The graph-oriented interpre-tation of this centrality index is the one of a measure ofhow often the node acts as a “bridge” along the shortestpaths between two any other nodes. Formally, the metricis defined as

b(i) =∑

j 6=i 6=k∈V

σjk(i)σjk

(5)

where σjk denotes the total number of shortest pathsbetween nodes j and k and σjk(i) is the number of those,that pass through node i. In our pedagogical setting thisclassical index, however, seems to be of limited impor-tance. The CCG is indeed by design a bipartite graph,where concepts are exclusively connected to courses andvice versa. Hence, betweenness metrics are expected to below for almost all nodes in the graph. This implies that theindex might not discriminate among different nodes, andhence might not provide strong insights into the propertiesof a program.

B. Connectivity and network flow analysisTo complement the centrality measures presented above

it can be useful to infer how courses and concepts areconnected among themselves: this may give quantitativeindications on how well connected the program is, whichcourses / concepts complement each other, and plan fur-ther learning / teaching monitoring & assessment activ-ities. To analyse how a graph as a whole is connectedthere exist several well-established tools; among others,

we discuss the pedagogical meaning of the connectivity,(minimal) cuts and the network flow concepts.

1) Connectivity: An undirected graph is connected ifthere exists a path between each pair of nodes. Otherwiseit is disconnected. For directed graphs, three differentdefinitions exist. First, consider ignoring the directionof all edges, i.e., replacing the directed graph with anundirected graph. If the corresponding undirected graph isconnected, the original directed graph is weakly connected.Further, the directed graph is connected if for each pair ofnodes i and j, there exists at least a directed path fromnode i to j or from j to i. Finally, if both directed pathsfrom node i to j and from j to i exist for all pairs of nodesi and j, the graph is strongly connected.

For instance, if a CCG happens to be disconnected,it reveals that different parts of the university programhave no overlap or connection. This might indicate thatconcepts and courses of very different areas are includedin the program, or that the program design may have someflaws. Hence, the teachers collaborating in the definition ofthe CCG might take this as a starting point to investigatea possibly undesired compartmentalisation of the programby investigating the (weakly) connected components of thegraph.

We also note that a well designed curriculum will mostlikely contain some weakly connected DCCG. Indeed it isreasonable to expect that there will be pairs of nodes iand j for which there exists no directed path from i toj or viceversa. As an example, consider two courses thatare given in parallel, which have some shared prerequisiteconcepts and possibly some shared developed concepts.Hence, if we ignore the direction of the edges, there exists apath between the two courses. However, as the informationflows from the prerequisites through the courses to thedeveloped concepts in parallel, there exists no directedpath between the two courses. The same conclusion, i.e.,that the DCCG cannot be connected but may be weaklyconnected, holds as soon as a course has at least twoprerequisites or develops two or more concepts in the samecourse.

On the opposite end of the scale, a connected or stronglyconnected DCCG indicates temporal or logical inconsis-tencies in the program, since a strongly connected andall relevant connected2, directed graph must contain atleast one cycle. Hence, connected or strongly connectedDCCGs are not desirable and hint on the need to furtherinvestigate the curriculum structure. See Section III-Cbelow for more details.

2) Minimal cut: In relation to the concept of connectiv-ity, a natural question that arises is to understand which

2The only possible, connected DCCG without a cycle is a simpleline graph indicating a rather trivial curriculum consisting of asequence of courses connected by exactly one concept in between,which is developed by the previous course and serves as a prerequisiteof the following course. This case can easily be outruled by plottingthe DCCG.

i x jexample 1: 2 2

i x jexample 2: 1 2

i x jexample 3: 2 1

Fig. 3: Examples of basic DCCGs, where i and j refer tocourses, and x refers to a concept.

edges or nodes are essential to maintain connectivity, i.e.,which elements must not be removed or the graph becomesdisconnected. Such sets of edges or nodes are denoted edgeor vertex cuts, respectively. Further, the minimal cut ofa graph describes how sensible a graph is to loose itsconnectivity. Both the size of the minimal cut (amountof edges or vertexes to be removed) as well as the edges ornodes included in the minimal cut offer important insightsinto the structure of the graph.

Analysing the minimal cut of a CCG can give interestinginsights into the program structure and its vulnerabilities.For instance, the minimal vertex cut will list the coursesor concepts whose inclusion in the program is crucial toconnect different areas or aspects in the program. Also, theminimal edge cut will reveal which conveyance of certainconcepts in certain courses is crucial to maintain connec-tivity on the program. In other words, it will indicatewhich concepts in which courses connect different areaswithin a program.

3) Network flow: The weight of an edge in a networkcan be interpreted as its capacity to carry a physical flow,e.g., water. Interpreting every edge of a network as such acapacity, it is natural to ask how many units of flow canbe transported from one part of the network to another.Defining thus at least one source node and at least one sinknode, it is possible to compute the maximal admissibleflow that can be carried between these nodes throughwell-known algorithms, e.g., [7]. Under this maximal flowsituation it is possible that some edge carries less flowthan their maximum admittable ones; e.g., in example 2in Figure 3, the maximum flow between nodes i and j is1, and the edge between x and j is not exploited at itsmaximum capacity.

The residual capacity of an edge is then the differencebetween its natural capacity versus its usage under max-imal network flow situations. Also, note that finding themaximal flow of a network is equivalent to finding an edgecut of minimal capacity that would separate the sink fromthe source.

To give to network flows a pedagogical interpretationunder our DCCG framework, we can interpret the weights0, 1 and 2 in the CCG as capacities that describe twospecific phenomena. Referring again to Figure 3, possibleinterpretations are:

• when the edge is from a concept x to a course j, howmuch the prior understanding of the required conceptx contributes to learn the ILOs of the specific course j;

• when the edge is from a course i to a concept x,how much the specific course i contributes to teach/ facilitate the understanding of the concept x.

A natural question is then about how analysing themaximal network flow associated to a DCCG can thenbe helpful to understand the structure of a program, itsshortcomings, bottlenecks and redundancies.

First of all, to enable performing network max-flowanalyses, at least one source and one sink node mustbe defined. For this, we propose to create two additionalnodes: “0”, as a global source, and “∞”, as a global sinkof the network flow, so that:• node 0 symbolises the prior knowledge that students

are expected to have before starting a certain pro-gram. Thus, node 0 connects with infinite capacityto all concepts that are considered a required priorknowledge, e.g., from high school education. Notethat, if for a given student or student cohort it isknown that their prior knowledge of some requiredconcept is limited or lacking, the capacity of thecorresponding edge from 0 to this concept node canbe lowered to analyse the consequences of this short-coming;

• node ∞ represents the final knowledge that studentsare expected to have after finishing a certain program.All those concepts whose understanding is included orrequired for reaching the goals of the program shouldthus be connected to node ∞ with edges of infinitecapacity.

We expect that the decision about which concepts shallbe connected to nodes 0 and ∞ may require the teachersand boards of the various programs extensively discussingthe program structure.

Assume then to have added to an existing DCCG thesefictitious source and sink nodes, their corresponding edges,and to have computed the maximal and the residual flowsof the network. Some interesting cases may then arise:First, consider example 1 in Figure 3, where both edges(the one from course i to concept x, and the one fromconcept x to course j) have the same capacity. Here themaximal flow is such that everything entering in concept xcan also flow to course j. This can be interpreted as a wellaligned structure, where students are enabled in course ito build up an appropriate knowledge about x, and thenuse it in course j.

In contrast, examples 2 and 3 in Figure 3 might bothreveal some mismatch in the program. In example 2,indeed, the overall maximal flow from i to j is 1, sinceconcept x is only taught with weight 1 in course i eventhough it is required with weight 2 by the following coursej. Hence the residual flow of the edge from x to j is 1, anda plausible interpretation of this is that students may not

i j

x y

Fig. 4: Example of a cycle in a DCCGs, where i and j referto courses, while x and y refer to concepts. The weightsof the edges are omitted for simplicity.

be as well prepared for taking course j as expected by theteacher of this course. In example 3, it is instead the linkfrom i to x that has under the max-flow regime a residualcapacity 1. A plausible interpretation of this is that notall the effort that is put during course i into teaching x isrequired for the following-up course j. In a sense, reducingthis potentially unnecessary effort during course i couldfree resources that could better be allocated otherwise.

In extreme cases there may be edges with maximalresidual flows, i.e., edges that are completely unused undermaximal network flow regimes. Our interpretation is thatthese events indicate grave mismatches or inconsistencieswithin the analysed program. This might include conceptsthat are neither taught in previous courses nor can beconsidered knowledge that students should have beforestarting the program, but are nevertheless required forsome course. This situation is especially critical, sincethe lacking of this required knowledge may hinder thelearning of new concepts. The event may also indicatesituations where the teaching includes concepts that areneither required in later courses nor considered desiredprogram goals (i.e., unnecessary material).

Finally, minimal capacity cut shows where the programis least robust, in the sense that it shows which conceptsand courses are key for reaching the overall program goal.

C. Existence of cycles

Another pedagogically interesting analysis of a DCCGis related to detecting cycles in the graph. For example,consider the situation in Figure 4: here course i hasconcept x as a required knowledge, and prepares studentsto course j by teaching concept y. The situation is thoughsymmetric, since j has concept y as a required knowledge,and prepares students to course i by teaching concept x.Thus as soon as i and j are not taught in the same learningperiod, students will not be prepared to take the first ofthe courses being taught (something that theoretically willalso affect their understanding, eventually inficiating alsothe attendance to the second one). And even if i and jare instead taught in the same learning period, then thislogical fallacy can be resolved only through a great careby the teachers of i and j in designing their own coursesso that the understanding and usage of concepts x and yhappens in parallel and simultaneously.

Generalizing, moreover, plotting a DCCG so that thecourse and concept nodes follow a temporal order (as did,e.g., the field-example from Uppsala University (UU) inSection IV) makes every link that points ”backwards”highlight some program inconsistency (i.e., a situationwhere students will not be prepared to take a certaincourse because the required concepts are being taught ina consequent learning period). Note however that if nocycles are present, though, this specific problem may beresolved by opportunely changing the temporal order ofwhen the various courses are given.

IV. Results

We gathered data for Electrical Engineering and Com-puter and Information Engineering at UU, Sweden, andEngineering Physics at Lulea University of Technology(LTU), Sweden, by asking teachers to allocate weightsof the scale {0, 1, 2} for the concepts in their coursein the program according to the methods described inSections II-A and II-C. The data were then analysed bycomputing the different indices described in Section III.

The results for the centrality indices for the coursesfor the two Electrical Engineering and the EngineeringPhysics programs are visualised in Figure 5 (for LTU)and Figure 6 (for UU). As discussed above, we noticedlarge variations between how many accumulated weightsteachers assigned to a course, which suggests differentinterpretations on the scale {0, 1, 2}. Hence, the degreecentrality for the courses for the Engineering physicsprogram at LTU reflect the accumulated weights assignedfor each course. This highly correlates with the teachers’interpretation of the scale and eagerness to contributeto the project. Further, the pagerank and eigencentral-ity mostly follow this trend and do not offer additionalinsights. Lastly, the closeness index is generally high forall courses (with some drops for courses with very lowdegree) whereas the betweenness is low for most courseswith some exceptions for courses with high degrees. Infact, interviewing the head of the corresponding programboard, showed that these data do not reflect the perceivedimportance of the courses in the program but rather theteachers’ understanding of the scale.

In order to compensate for this effect, we normalized theweights in the CCM for the Electrical engineering programat UU such that all weights for a course accumulate to one.Hence, the degree centrality, shown in Figure 6 is equal toone for all courses and no information can be extracted forthis index. Further, as expected, the betweenness indexis low for almost all courses with the exception of somecourses, that include a large number of concepts and henceconnect many nodes in the graph. For the collected data,the eigenvalue index and closeness index offer insightsinto the program structure. Indeed, the courses with higheigenvalue and closeness indices were clearly indicated ascentral courses in the programs by the program board.

Figures 7 and 8 show the centrality indices for allconcepts in the CCMs for LTU and UU, respectively.The concepts are ordered by their degree index and arecalculated using the original data (i.e., not normalized)for LTU and the normalized data for UU. Note thatnormalising the degree for each course node to one, doesnot imply a normalisation of the concept nodes. Thedegree centrality and the pagerank centrality, which isvery similar to the degree, do appear to provide an overallreflection of how often a concept is taken up during theprogram in both the normalized and the unnormalizedcase. As expected, the betweenness index is low for almostall concepts. However, for the normalized case (UU) theconcepts with high betweenness seem to indicate conceptsthat are taken up by many different courses. Further, forthe normalized CCM, in cases where the degree centralityis low for the same concept, it refers to a concept that istaken up often but also often weighted with a 1 (comparedwith important concepts receiving weight 2). The oppositeseems to be true for closeness in the normalized CCM,which is high for almost all concepts in the program.However, some concepts have a low closeness despite acomparatively high degree. These are concepts that playa relatively strong role in the program but are only takenup in some courses in a rather intensive fashion. Note thatsuch insights cannot be extracted for the unnormalizedCCM for the program at LTU.

The data collected for Computer and Information Engi-neering at UU, instead, comprise information on whetherthe various given concepts shall be considered a prereq-uisite or a teaching outcome for the courses. Part of thecorresponding DCCG is shown in Figure 9. Analysing theDCCG reveals several interesting insights. First of all,analysing the max flow of the graph allows understandingmismatches in effort in the sense of spending much moretime or effort in teaching a concept, that is only requiredin few or no courses with low weights, or vice versa. Forinstance, consider “hashtables”, which is taught in threecourses, 1DL201 (Program Design & Data Structures),1DT093 (Computer Architectures) and 1DL221 (ObjectsOriented Programming) with accumulated weight 4, butonly required in one single course, 1DT096 (Operating Sys-tems), with weight 1. In contrast, the concept “induction”is only taught in one early course, 1DT051 (Introduction toInformation Technology), with weight 1 but required forthree following courses with various weights and furtherconsidered an intended learning output of the program.Both cases can be found by analysing the redundancies inthe DCCG under max flow. (Links with maximal flow inFigure 9 are shown in green and links with redundanciesare shown in blue.)

Further, analysing cycles in the graph allowed extractingmismatches in time, i.e., between courses, where concepts(e.g., “lists” and “arrays”) are assumed prior knowledge, orat least a common understanding, and are hence requiredin an early course (e.g., 1DT093) but are taught in a

later course (e.g., 1DT051). These cases are marked asred arrows in Figure 9. Either, teaching these conceptsin the later courses is redundant since the teacher of thefirst course correctly assumed that students would knowthese concepts from earlier education as for instance highschool, or students are not sufficiently prepared for theearly course due to a lack of knowledge. It may also bepossible that the courses require or teach the concept ondifferent levels in Bloom’s taxonomy. In this case, no mis-match might be present. However, in order to understandthose issues and to avoid mismatches, teachers need tocollaborate and/or exchange more detailed information. Inany case, the analysis of the corresponding DCCG allowsidentification of which aspects need to be discussed ormaybe even changed in the course and program organi-sation.

In summary, these results indicate that relevant infor-mation can be extracted from the CCG or DCCG byconsidering centrality indices of concepts and courses givenadequate normalisation of the weights or analysing theconnectivity and flow in the graph.

V. ConclusionsIn this paper, we proposed a method to analyse quan-

titative data about which concepts are relevant for whichcourses in a university program (provided by the corre-sponding teachers) and the connections between them.We showed how this information can be described by acourse-concept matrix (CCM) and a corresponding course-concept graph (CCG). Further, by analysing the centralityindices of the involved courses and concepts for two pro-grams at UU and LTU in Sweden, relevant informationcould be extracted. It appears that the results are betteraligned with the program boards’ perceptions and insightsif the weights in the CCM are weighted. Further, categoris-ing concepts as required knowledge for a course or taughtin a course allows building a related DCCG. Its analysisreveals mismatches and redundancies in the program.

Additionally, input from students might be of use in theprocess of establishing concepts for courses in two majorways. Firstly, to determine whether the concepts identifiedby the teachers match the experiences of the students.Secondly, the concepts identified by the students functionas feedback for teachers and program boards on howcourses are experienced by students. This is valuable inputfor course and program development. Large discrepanciesindicate that the course misses the target, which could bean issue in ensuring development and progression of anintended learning curve.

In this first attempt to systemize concepts of an engi-neering program and a tool to understand the intercon-nections between them, we can conclude that conceptsare probably not enough to describe the learned contentfrom courses and the whole program. Motivated by [16]one should rather include facts and procedural knowledgealong cite conceptual knowledge. Further, instead of using

the simple scale from 0 to 2, using the levels ranging from“remember” to “create” from Bloom’s revised taxonomy,[2], is expected to further improve the analysis of theuniversity program. However, the proposed graph theorytools have to be carefully revised in order to obtainmeaningful results when using the taxonomy.

Also, suitable methods should be developed to visualiseand display the graph structure and highlight the obtainedresults such as flows, cycles and discrepancies in a suitableway. Even though our preliminary experience in commu-nicating our method to students, boards, administratorsand fellow teachers revealed great interest and readinessto adopt the new method, its success will depend onwhether the information can be presented in a useful andunderstandable way for all stakeholders. We believe thatsuch a suitable visualisation would also help to realise someof the goals outlined in [11] such as• informing learners of what they can expect to achieve

from a program, so they can organise their time andefforts;

• communicating curriculum/program goals in a mean-ingful way to a broader community;

• helping to determine the extent to which learning hasbeen accomplished;

• guiding curriculum committees (within resource con-straints) to determine program(s) of study and courseofferings;

• guiding instructors when they are designing courseobjectives, content, delivery and assessment strate-gies.

However, how to reach those goals with the help of the hereproposed tool and its extensions and further developmentsneeds to be investigated in the future. Another importantdirection for future development is to include actors out-side the academia, such as the industry, both by using the(graphical) tool to allow them to get a much clearer viewon content, (program) goals and the intended progress foreach program as well as considering and combining theirdemands with the program learning outcomes.

References

[1] Preston R. Aldrich. The curriculum prerequisite net-work: Modeling the curriculum as a complex system.Biochemistry and Molecular Biology Education, 43(3),2015.

[2] Lorin W Anderson, David R Krathwohl, Peter WAirasian, Kathleen A Cruikshank, Richard E Mayer,Paul R Pintrich, James Raths, and Merlin C Wit-trock. A taxonomy for learning, teaching, and assess-ing: A revision of bloom’s taxonomy of educationalobjectives, abridged edition. White Plains, NY: Long-man, 2001.

[3] M Battersby. Outcomes-based education: A collegefaculty perspective. Learn Quarterly, 1:6–11, 1997.

[4] F Michael Connelly, Ming Fang He, and JoAnnPhillion. The Sage handbook of curriculum and in-struction. Sage, 2008.

[5] Edward F Crawley, Johan Malmqvist, Soren Ostlund,Doris R Brodeur, and Kristina Edstrom. The CDIOapproach. In Rethinking engineering education, pages11–45. Springer, 2014.

[6] Jay R Dee and William A Heineman. Understandingthe organizational context of academic program de-velopment. New Directions for Institutional Research,2015(168):9–35, 2016.

[7] Reinhard Diestel. Graph Theory. Springer-VerlagHeidelberg, New York, 2005.

[8] European Commission. The Bologna Process andthe European Higher Education Area. https://ec.europa.eu/education/policies/higher-education/bologna-process-and-european-higher-education-areaen.

[9] Eva Fjallstrom, Steffi Knorn, Kjell Staffas, and Dami-ano Varagnolo. Developing concept inventory tests forelectrical engineering: extractable information, earlyresults, and learned lessons. In Proceedings of the UKAutomatic Control Conference, 2018.

[10] Bernard SM Gatawa. The politics of the schoolcurriculum: An introduction. College press, 1990.

[11] Harry Hubball and Helen Burt. An integratedapproach to developing and implementing learning-centred curricula. International Journal for AcademicDevelopment, 9(1):51–65, 2004.

[12] Jay M Lightfoot. A graph-theoretic approach toimproved curriculum structure and assessment place-ment. Communications of the IIMA, 10(2):5, 2010.

[13] Jaime A. Pavlich-Mariscal, Mariela Curiel, and Ger-man Chavarro. CDIO curriculum design for comput-ing: a graph-based approach. In Proceedings of the15th International CDIO Conference, 2019.

[14] Raita Rollande. The Research and Implementationof Personalized Study Planning as a Component ofPedagogical Module. PhD thesis, Riga TechnicalUniversity, 2015.

[15] Patrick Slattery. Curriculum development in thepostmodern era: Teaching and learning in an age ofaccountability. Routledge, 2012.

[16] Kjell Staffas. Developing requisite motivation in en-gineering studies: A study on a master and bachelorprogram in electronic engineering at Uppsala Univer-sity. PhD thesis, Aalborg Universitet, 2017.

[17] Paul Temple. The integrative university: Why univer-sity management is different. Perspectives, 12(4):99–102, 2008.

[18] Gabriela C Weaver, Wilella D Burgess, Amy L Chil-dress, and Linda Slakey. Transforming institutions:undergraduate STEM education for the 21st century.Purdue University Press, 2015.

[19] Jon Wiles. Leading curriculum development. CorwinPress, 2008.

[20] Jon Wiles, Joseph Bondi, and Hua Guo. Curriculumdevelopment: A guide to practice. Merrill PublishingCompany, 1989.

0 0.2 0.4 0.6 0.8 1

F0051TM0029MF0004T

M0030MF0005T

M0031MF0006T

M0032MD0017EK0016KE0003E

M0018MF0007T

M0014MF0008TF0030TS0001ES0008MF0047TF0048TE0007ER0005NR0004ES7013E

centrality(←

time)

cour

se

degree closeness eigenvectorbetweenness pagerank

Fig. 5: Measured courses centrality indexes for Engineeringphysics at LTU, Lulea, Sweden.

0 0.2 0.4 0.6 0.8 1

1TE7051TE7041MA0081TE6671MA0131TE6681TE6691FA5181TE7701TE7711MA0341TE7741TD3931TE6551MS0081TE6701TD4331TE7441TE745

1TE672a1TE672b1TE672c1TE672d

centrality

(←tim

e)co

urse

degree closeness eigenvectorbetweenness pagerank

Fig. 6: Measured courses centrality indexes for ElectricalEngineering at UU, Uppsala, Sweden.

0 0.2 0.4 0.6 0.8 1

continuitylinearity

convergencelin. sys. of equations

dimension analysiscomplex numbers

vector geometrydifferential calculus

structures (programming)eigenvalues and vectors

integral calculustransformation

arraysfunctions (programming)

limitsdeterminant

matricesempirical models

nonlinear equationswork and energy

entropyNewton’s lawswave equation

stabilitycoordinate transf.

parallelismequations of motion

time-invariancecurve fitting

vectorsimpulse and momentum

lossesfiltering

frequency domaincomplexityconvolution

series expansionaverage and variance

Poles and zerostime domain

free-body diagramskinematics and kinetics

probabilitiesprobability distributions

samplingOhm’s law

multivariable calculuspoint estimation

confidence intervalsrandom variables

causalityquantizationtime shifting

Kirchoff’s lawspotential, voltage

ODE solvingabsorption/emission

gravityintensity

interference,diffractionmechanical inertia

reflection, refractionactive and reactive power

op-ampoptimisation

PDEsfriction

polarization

centrality

conc

ept

degree closeness eigenvectorbetweenness pagerank

Fig. 7: Measured concepts centrality indexes for Engineer-ing physics at LTU, Lulea, Sweden.

0 0.2 0.4 0.6 0.8 1

complex numbersvectors

lin. sys. of equationsOhm’s law

Kirchoffs lawsfrequency domainpotential, voltage

linearitymatrices

work and energyintegral calculus

differential calculusstabilityfiltering

structures (programming)average and variance

series expansionnumerical methods

time-invariancesampling

vector geometryODE solvingconvergence

nonlinear equationsactive and reactive power

limitsmagnetic induction

eigenvalues and -vectorstime domaindeterminant

Newton’s lawsequations of motion

probabilitiesprobability distributions

arrayslosses

order of accuracyiterative methods

convolutionparallelism

methods (programming)curve fitting

floating-point numbersdata types

fields, E B Hcausality

combinatoricsMaxwells laws

objects (programming)random variables

PDE sep. of variablescontinuity

time shiftingtime scaling

confidence intervalsmechanical inertia

adaptive methodimpulse and momentum

classes (programming)constructors (programming)

complexitystrings

vibrationsroundoff errors

frictionpoint estimationtruncation error

central-force motionassignments (programming)

centrality

conc

ept

degree closeness eigenvectorbetweenness pagerank

Fig. 8: Measured concepts centrality indexes for ElectricalEngineering at UU, Uppsala, Sweden.

pre 1DT051 1DL201 1DT093 1DL221 1DT096 1DL242 1DL311 PLOs

recursion

div.& conq.

induction

data struct.

trees

lists

graphs

arrays

hashtablesstacks

Fig. 9: Part of the DCCG for Computer and Information Engineering at UU showing temporal mismatches as redarrows, unused/redundant connections as blue arrows and remaining (well matched) connections are green arrows.(Note that the max-flow analysis involved all concepts but only 10 are shown here.)