Performance Modeling and Analysis of aHyperledger-based System Using GSPN

Pu Yuan∗, Kan Zheng∗, Xiong Xiong∗, Kuan Zhang† and Lei Lei‡∗ Intelligent Computing and Communications Lab, Key Lab of Universal Wireless Communications

Beijing University of Posts and TelecommunicationsBeijing 100876, China

† Department of Electrical and Computer EngineeringUniversity of NebraskaLincoln

Omaha, NE 68182, USA.‡ College of Science and Engineering

James Cook UniversityDouglas, QLD 4814, Australia

Email: yp2013@bupt.edu.cn, ajodfaj@bupt.edu.cn, kuan.zhang@unl.edu, lei.lei@jcu.edu.auCorresponding author: zkan@bupt.edu.cn

Abstract—As a highly scalable permissioned blockchain plat-form, Hyperledger Fabric supports a wide range of industryuse cases ranging from governance to finance. In this paper, wepropose a model to analyze the performance of a Hyperledger-based system by using Generalised Stochastic Petri Nets (GSPN).This model decomposes a transaction flow into multiple phasesand provides a simulation-based approach to obtain the systemlatency and throughput with a specific arrival rate. Based onthis model, we analyze the impact of different configurationsof ordering service on system performance to find out thebottleneck. Moreover, a mathematical configuration selection ap-proach is proposed to determine the best configuration which canmaximize the system throughput. Finally, extensive experimentsare performed on a running system to validate the proposedmodel and approaches.

Index Terms—blockchain, Hyperledger Fabric, performancemodeling, Generalised Stochasitc Petri Nets

I. INTRODUCTION

Blockchain technology originated from Bitcoin has beengrowing rapidly in recent years [1]. The blockchain leveragescryptographic techniques, distributed ledgers and consensus al-gorithms to provide a trusted and decentralized service for sev-eral applications [2]–[6]. Depending on the user authorizationmechanisms, blockchain can be mainly categorized into thepermissionless and the permissioned blockchain [7]. Ethereumis a programmable permissionless blockchain platform thatachieve business logic based on specific smart contracts [8],[9]. On the other hand, Hyperledger Fabric is an open sourceenterprise-grade permissioned blockchain platform with ahighly modular and configurable architecture [10]. It integratesfine-grained access control, immutable ledger and pluggableconsensus protocols. Due to those advantages, HyperledgerFabric is used in many scenarios, such as emission trading,insurance and education [11]–[14].

The performance of Hyperledger Fabric was widely studied[15], [16]. The performance differences between HyperledgerFabric version 0.6 and version 1.0 have been evaluated in

[17], which indicated that version 1.0 can significantly improvethe system performance. A repeatable evaluating methodologywas proposed to assess the performance of Hyperledger Fabricand Ethereum in [18]. The consensus protocols of those twoplatforms were compared in [19], where Hyperledger Fabricachieves the better performance than Ethereum. Moreover,extensive experiments with varying parameters on HyperledgerFabric v1.0 have been conducted to study the impact of varioussystem configurations [20]. The experimental results indicatedthat the endorsement policy verification, sequential policyvalidation of transactions in a block and the state validationand commit are three major performance bottlenecks. Basedon those analyses, several optimizations to improve the overallperformance were introduced.

However, those experiment-based analyses for HyperledgerFabric are lack of scalability and theoretical basis. In orderto further analyze the performance characteristics of thisblockchain framework, it is imperative to model the trans-action flow of it using a mathematical approach. Due to thecomplexity of the transaction flow and system configurations,the performance of Hyperledger-based system is affected byseveral factors. Thus, there are many difficulties in modelingthe system.

In this paper, we analyze the blockchain performance basedon Hyperledger Fabric. As a formal mathematical theorydesigned for modeling concurrency, causality and conflict,GSPN provides a graphical approach to decompose the requestprocessing flow of the Hyperledger-based system into multiplephases. Moreover, the GSPN-based model can be simulated toobtain the performance metrics such as latency and throughputof each phase at a non-steady state, which is suitable forthis focused scenario.To sum up, our major contributions aresummarized as follows: i.e.,

1) An analytical model is proposed to depict the transactionflow of a Hyperledger-based system using GSPN andvalidate this model by experiments.

arX

iv:2

002.

0310

9v1

[cs

.DC

] 8

Feb

202

0

2) We analyze how different ordering strategies affect thesystem performance and identify the performance bottle-neck based on the proposed model.

3) In response to the identified bottleneck above, a mathe-matical configuration selection approach is proposed todetermine the configuration parameters of the orderingservice in order to achieve the maximum throughput ofthis system.

To validate the proposed model and configuration selectionapproach, a running system is setup on a cloud server. Fur-thermore, our work has important guiding significance for thepractical use of the Hyperledger-based systems.

The remainder of this paper is organized as follows. In Sec-tion II, we investigate the related work. Section III introducesa Hyperledger-based system and the transaction flow of it.Then an analytic model based on GSPN and a configurationselection approach to achieve best system performance are pro-posed in Section IV. Next, Section V validates our model andapproach by conducting extensive experiments on a runningsystem. Finally, Section VI outlines the main conclusions.

II. RELATED WORK

Compared with those experiment-based performance anal-yses mentioned before, the performance modeling for Hyper-ledger Fabric is more critical and scalable to analyze the char-acteristics of this blockchain framework. To depict the systemaccurately, an appropriate mathematical theory is needed. As amodeling approach, Petri Nets is a formal mathematical theorywith rigorous mathematical foundation and intuitive graphicalrepresentation. Derived from Petri Nets, the Stochastic PetriNets (SPN) associates an exponentially distributed delay withthe firing of each transition to provide a clear and intuitiveformalism for generating Markov processes. Based on SPN,the Generalised Stochastic Petri Nets (GSPN) adds immediatetransitions and inhibitor arcs to prevent the model frombecoming exceedingly large. Moreover, the Stochastic RewardNets (SRN) introduces more primitives than GSPN to enhancethe expressive ability. All those nets are widely used insystem performance modeling and analysis [21]–[24]. As formodeling for Hyperledger Fabric, a SRN model of the PBFTconsensus based on Hyperledger Fabric v0.6 was presented todiscuss how the number of peers affects the consensus latencyin [25]. Furthermore, an overall performance model of Hyper-ledger Fabric v1.0+ was proposed in [26], which discussed theimpact of different parameters on system performance suchas latency, throughput and utilization. The proposed modelwas validated by using a test framework named HyperledgerCaliper. However, these studies only analyzed the performancecharacteristics based on simulation but they did not provide amethod to obtain the appropriate configuration parameters ofsystem.

III. SYSTEM OVERVIEW

A. System Architecture

A Hyperledger-based emission trading system is proposedto solve the defects of existing centralized systems [9]. The

system uses two organizations to represent the environmentalagency and the trading center respectively and share thesame ledger through one single channel. By integrating thecharacteristics of blockchain, the polluters can achieve thecredible trading service through this system. As a typicalimplementation of Hyperledger Fabric v1.2, the system ar-chitecture of the blockchain system is given in Fig. 1.

Hyperledger Fabric Network

Organization 1 Organization 2

Channel

CA CA

Peer Peer

Order

Orderer CA

HTTP Server

Client Client

App for Organization 1 App for Organization 2

Function

One

Function

Two

Function

One

Function

Two

Fig. 1: System Architecture.

This system consists of HTTP server, Web application andthe blockchain network. The HTTP server plays a role of Fab-ric client to interact with the underlying network by integratingspecific Fabric SDK. Depending on those RESTful APIs pro-vided by the HTTP server, the web application can provide avariety of services for users. The blockchain network containstwo distinct organizations, one channel which connects thoseorganizations and an orderer node to provide the orderingservice. This node adopts solo consensus protocol to guaranteethe consistency of the distributed ledger. Each organization hasa Fabric CA and a local peer. Fabric CA issues certificatesfor participants to achieve the access control policy. The peerholds an immutable ledger based on LevelDB and installscustomized chaincode which implements the business logic.Each peer in this system plays roles of not only an endorser(endorse for transactions) but also a committer (hold ledgers).In addition, all of those nodes are running in special dockercontainers.

B. Transaction Flow

Requests in Hyperledger Fabric are divided into kinds, i.e.,query and invoke, depending on whether the ledger is modi-fied. Fig. 2 depicts the transaction flow of a typical invoke-typerequest. A completed transaction consists of several phases asfollows, i.e.,

HTTP Phase. Client sends a HTTP request to the server,aiming to interact with the blockchain network. The HTTP

ClientHTTP

Server

Peer 1

Endorsing &

Committing

Peer 2

Committing

Ord

erin

g S

ervice

Simulate

&

Endorse

Validate

&

Commit

Validate

&

Commit

6 notification

Fig. 2: Transaction Flow.

server extracts essential parameters from the request body andthen constructs a transaction proposal by using the providedSDK. The generated proposal is signed with the clients cre-dentials and contains details of the specific chaincode. Then,the proposal is sent to an endorsing peer to endorse for thistransaction.

Endorsement Phase. All the peers that have installed thechaincode can play the role of an endorsing peer. Whenreceiving a transaction proposal, each endorsing peer shouldexecute tasks as follows: Firstly, the peer verifies the identityof this submitter to check whether its authorized to invokethe chaincode. Secondly, the peer executes the chaincodeto generate the response value and read-write set withoutmodifying the world state. Thirdly, endorser signs the proposalresponse with its identity and then the peer sends the responseback to the client. Finally, the client collects responses frommultiple endorsers and verifies if they satisfy the endorsementpolicy. The system adopts or policy so that one endorser isenough.

Ordering Phase. Once the transaction is fully endorsed, theclient integrated in HTTP server broadcasts it to the orderingservice. According to the specific configuration, the orderingservice orders all the transactions and packages them intoblocks with the special strategy. Then it signs the blocks anddelivers them to the leading peers using gossip protocol. Someof the critical parameters of ordering service are listed asfollows, i.e.,• BatchTimeout: The amount of time to wait before cre-

ating a batch.• MaxMessageCount: The maximum number of messages

batched into a block.• PreferredMaxBytes: The maximum number of bytes

allowed for the serialized messages in a block.When one of the above conditions is satisfied (e.g., the

number of transactions queuing in ordering service reaches theMaxMessageCount), a sequence of transactions can be batchedinto a new block. Considering that the size of transaction is

similar, in general only BatchTimeout and MaxMessageCountneed to be adjusted.

Validation & Committing Phase. After receiving blocksfrom the ordering service, the leading peers disseminate thoseblocks to all the peers, which belong to the same channeland organization. The peers first verify the signature of theblocks and then check all the transactions within them. If allthe transactions pass the endorsement validation and read-writeset validation, those blocks are appended to the ledger and theworld states are updated.

Response Phase. By registering an event listener in Chan-nelEventHub, the HTTP server can receive a notification whenthe target transaction has been committed into a block andappended to the ledger. A registered callback function cancollect details of this event and form the response data inJSON format.

IV. THE GSPN-BASED ANALYTIC MODEL OF SYSTEM

In this section, an analytic model for the Hyperledger-basedsystem mentioned in Section III is proposed. We first introducethe basic elements of GSPN and the modeling assumptions.Then the proposed model is decomposed to multiple phasesand each phase is described in details. Finally, we presenta configuration selection approach to determine the networkparameters.

A. Preliminaries

A typical GSPN model consists of basic elements as fol-lows, i.e.,

Places. Circular nodes are used to describe places, whichrepresent conditions or local system states.

Transitions. Rectangular boxes are used to describe transi-tions, which represent events occur in the system. The fire of atransition can change system status from one place to another.

Tokens. Black dots or numbers are used to describe thetokens resided in place, which represent the state quantity thata place holds.

Arcs. Arcs specify the relationship between places andtransitions. The weight of input arc represents the number oftokens consumed by the transition firing, and the weight ofoutput arc represents the number of tokens produced to theoutput place. The default weight of arc is 1.

Immediate Transitions. Thick bars are used to describeimmediate transitions, which represent events that are assumedto take no time.

Inhabit Arcs. An inhibitor arc from a place to a transitionmeans the transition cannot fire if there is a token in the place.

Based on those basic elements, a performance model for thesystem mentioned in Section III can be proposed. The systemmodeling satisfies the following assumptions, i.e.,

• The arrival of requests is a Poisson process.• The size of each transaction is constant.• Ignore the transmission latency between processing

phases.

HTTP Service

Ordering ServiceEndorsement

Verify &Commit

Tarr Pwait_h Pserve_h Th

Pidle_h

Pwait_e Pserve_e Te

Pidle_e

Pwait_o

Pserve_co

N

Pidle_o

Pwait_c0 Tc0

Pidle_c0

Pend

Pwait_c1 Pserve_c1 Tc1

Pidle_c1

Pend_c0

TnToPserve_o

Pend_c1

NetworkTransmission

Fig. 3: GSPN Model of System.

Tarr Pwait_h Pserve_h Th

Pidle_h

PnextTin

Fig. 4: Decomposed model of HTTP server.

B. Model Description

The proposed analytic model based on GSPN is shownin Fig. 3. The overall model depicts the transaction flow indetails, consisting of five phases as follows, i.e.,

HTTP Phase. This phase represents the process by whichthe HTTP server receives the request and sends the transactionproposal to the endorser. It can be regarded as a fundamentalprocessing unit. Fig. 4 decomposes this unit from the overallmodel.

The meaning of all the places and transitions are describedas follows, i.e.,

Tarr: a transition that represents the arrival of a newrequest.

Pwait h: a place that represents the request is queuing, thenumber of token #(Pwait h) denotes the queuinglength.

Pserve h: a place that represents the request is being pro-cessed.

Pidle h: a place that represents the server is idle now, thenumber of token #(Pidle h) denotes the number ofidle servers.

Tin: an immediate transition whose enable predicate is#(Pwait h) > 0 & #(Pidle h) > 0, which meansthere are idle servers and queuing requests.

Pnext: a place represents the next processing phase.The performance metrics of each place in this model can

be obtained by simulating. Assume that during the total

simulation time T , the transition Tarr has been fired X times,i.e., place Pwait h generates X tokens. The firing time intervalobeys exponential distribution with parameter λ. Let ϕp denotethe number of departed tokens of place P , τpi denote the staytime of i− th token in place P . The performance metrics ofplace P can be derived as follows, i.e.,• Throughput:

θp =ϕpT. (1)

• Average Latency:

LP =

∑i τpiϕp

. (2)

• Queuing Length:

QP = θP ∗ LP =

∑i τpiT

. (3)

Obviously, this processing unit describes a typical M/M/1queuing model, whose total latency equals to the queuing timeplus the service time, i.e.,

δH = LPwait h+ LPserve h

. (4)

Endorsement Phase. This phase describes the endorsementprocess in the processing flow. Since the system adopts the orendorsement policy, one endorser for a request is enough. Sothere is only one fundamental processing unit in this phase.According to the above analysis, the total latency of this phaseδE can be derived.

Ordering Phase. This phase describes the ordering serviceof the system. Because the block packaging, signing anddelivering are executed sequentially, only one place Pserve o isused to represent the processing of the orderer node. Differentwith the above phases, the arc between place Pwait o andimmediate transition Tin has a weight N , which indicatesthat the enable predicate of Tin is #(Pwait o) ≥ N &#(Pidle o) > 0. When Tin fires , N tokens in Pwait o areabsorbed and one token is created in Pserve o. Correspondingto the actual system, this weight represents the operation ofpacking N transactions into a block. Similarly, the total latencyof this phase δO can be derived.

Committing Phase. When the peer node receives a newblock, it performs a series of validations (e.g., MVCC) and

then commits the block into the local ledger. Those processesare abstracted into one place Pserve c to simplify the model.Since two peers in the system perform commit operationssynchronously, the HTTP server listens to the events of bothpeers at the same time. Thus, there are two parallel processionunits in this phase. The total latency of this phase depends onthe one with the larger latency.

δC = max(LPwait c0+LPserve c0

, LPwait c1+LPserve c1

). (5)

Response Phase. In order to approximate the real system,an extra transition is used to represent the network latency be-tween applications and HTTP server, which can be expressedas δT = LPend

.The total latency of this system can be derived by

∆ =

T∑i=H

δi. (6)

Moreover, it is obvious that a request has been completelyhandled after it arrives at the place Pend. Therefore, thethroughput of this system is equal to the throughput of placePend, i.e.,

Θ = θPend. (7)

Based on the proposed analytic model, the system per-formance metrics can be obtained conveniently through thesimulations after determining the rates of all transitions.

C. Network Parameter Determination

As we know, the most time-consuming operation in theordering phase is signing and delivering the blocks [18]. Itis feasible to reduce the block generation rate by adjustingthe configuration of the ordering service, which can guar-antee the ordering phase is not the performance bottleneck.Moreover, the experimental results in [24] indicate that ata high request arrival rate, the performance bottleneck ofthe Hyperledger Fabric framework exists in the endorsementphase or the committing phase (We will prove this in thenext section). Moreover, the number of transactions containedin the block (or block size) has a great influence on thesystem performance. Under this premise, we can discuss theimpact of the different parameters of ordering service, inorder to find the best configuration parameters to optimizethe system performance (i.e., make the system achieve themax throughput). The overall throughput is selected as theperformance indicator rather than the overall latency and thereasons are listed as follows, i.e.,• Many related studies indicate that the throughput is more

important than the latency in a Hyperledger-based system[18], [24].

• With the increase of the requests arrival rate, the systemlatency can grow infinitely while the throughput can reacha saturation point.

• The overall latency of a Hyperledger-based system isgreatly affected by the configuration parameters of thenetwork and the request arrival rate. It is obvious that theoverall latency obtained from experiments cant represent

the real processing time because it contains uncertainwaiting time due to those different parameters. However,considering the throughput can ignore the impact of therequest arrival rate because the throughput can reacha saturation point when the request arrival rate is bigenough.

The proposed analytic model based on GSPN indicates thatthe blockchain system is composed of multiple successiveM/M/1 networks. Thus, the system throughput is equal to thelowest throughput of those phases. Obviously, different N inmodel can greatly affect the arrival rate of the committingphase, which further affects the system performance. Consid-ering the ordering service configuration mentioned earlier, thenumber of transactions within a single block is determinedby specific packaging strategies. Assume that λ denotes thearrival rate of requests, t denotes the BatchTimeout, n de-notes the MaxMessageCount, µe denotes the service rate ofendorsement, µc denotes the service rate of committing andµc is determined by N with function f(N). Our work is tofind appropriate n and t to maximize the system throughput,i.e.,

Θmax = max min(N ∗ f(N), µe), (8)

where N is determined by n and t with function g(n, t), i.e.,

g(n, t) =

{n, λ ≥ n

t ,

bt ∗ λc+ 1, 0 < λ < nt .

(9)

It is clear that we can adjust N to make sure the through-put reaches the maximum value µe, so there is λ ≥ µe.Considering that λ is an uncertain value in the practical use,it is inappropriate to determine the configuration parametersdepending on it, so it is better to assign it to a constantvalue. It is obvious that the obtained n and t under thepremise λ = µe can still make sure the system throughputreach µe when λ > µe, thus (9) can be simplified by takingλ = µe. Moreover, n and t can be considered respectively,which ignores the impact of λ and guarantees µe is always thesmallest one in the formula, so that t can not be the bottleneck(e.g. when t and λ are small, the number of transactions arrivedin t may be less than n, thus N is limited). Therefore,

Θmax = max min(µe, n ∗ f(n),

(bt ∗ µec+ 1) ∗ f(bt ∗ µec+ 1)).(10)

Then, after determining µe and f(N), the appropriate config-uration parameters can be obtained to achieve the maximumthroughput of the system.

V. PERFORMANCE EVALUATION & OPTIMIZATION

A. System Configuration

To validate the proposed model, the system mentioned inSection III is set up on a aliyun cloud server with 4 Intel(R)Xeon(R) CPU E5-2680 v3 @ 2.50GHz processors and 8GBRAM, running Ubuntu 14.04 (64 bit). Various experiments areconducted on this running system by sending HTTP requests.To simulate the GSPN-based model needs to determine the

10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

1600

1800La

tenc

y (ms)

Arrival Request Rate (RPS)

Experiment Simulation

(a) N = 1

10 20 30 40 50 60 70 80 900

500

1000

1500

2000

2500

3000

3500

Late

ncy

(ms)

Arrival Request Rate (RPS)

Experiment Simulation

(b) N = 2

10 20 30 40 50 60 70 80 90 100 110 120 1300

400

800

1200

1600

2000

2400

2800

Late

ncy

(ms)

Arrival Request Rate (RPS)

Experiment Simulation

(c) N = 3

Fig. 5: System Latency with Different N .

10 15 20 25 30 35 405

10

15

20

25

30

35

40

Thro

ughp

ut (

RPS)

Arrival Request Rate (RPS)

Experiment Simulation

(a) N = 1

10 20 30 40 50 60 70 80 900

10

20

30

40

50

60

70

80

Thro

ughp

ut (

RPS)

Arrival Request Rate (RPS)

Experiment Simulation

(b) N = 2

10 20 30 40 50 60 70 80 90 100 110 120 1300

10

20

30

40

50

60

70

80

90

100

110

120

Thro

ughp

ut (

RPS)

Arrival Request Rate (RPS)

Experiment Simulation

(c) N = 3

Fig. 6: System Throughput with Different N .

firing time of all the transitions at first. For the HTTP phaseand endorsement phase, the latency can be calculated byprinting timestamp at the specific code point in HTTP server.For ordering phase and committing phase, the time informationcan be extracted from log files after adjusting the log level ofrelevant docker containers from INFO to DEBUG. Finally, forresponse phase, an extra empty interface is integrated in theHTTP server to obtain the end-to-end latency. The averagelatency of all phases are summarized in Table I.

TABLE I: Model Parameters.

Transision Th Te To Tc Tn

Latency (ms) 2 7 12 27 10Rate 500 143 83 37 100

B. Model ValidationBased on the running system, Locust1 is used to evaluate the

system overall latency and throughput of HTTP interfaces overincreasing request arrival rates and different N . The through-put is evaluated by Requests Per Second (RPS). Moreover, asoftware tool embedded in Matlab named pntool2 is used to

1https://www.locust.io/2http://www.pntool.ac.tuiasi.ro/

simulate the GSPN-based model under the same conditions.This tool can obtain the performance metrics (e.g., latency,throughput, queuing length) for all the transitions and placesof the model. By analyzing the simulation results, the overalllatency and throughput can be determined. Fig. 5 and Fig. 6compare the experimental results with the simulation resultsof the system. It can be observed that the experimental resultsare comparable to the simulation results and the proposedmodel can well describe the actual system. Furthermore, theoverall latency increases rapidly when throughput reachesthe saturation point and the maximum throughput is greatlyimproved due to the increase of N , which indicates that thedifferent configuration parameters (N ) has a great influenceon the system performance.

C. System Bottleneck Analysis

System bottleneck analysis is significant for performanceimprovement. For a Hyperledger-based system, once the bot-tleneck (i.e., endorsement phase, ordering phase and commit-ting phase) is identified, the optimization goal is determined.It is very hard to obtain the performance metrics of eachprocessing phase through experiments as the official SDKdoes not provide corresponding APIs for developers. Thus, weanalyze the system bottleneck based on the proposed GSPN

10 15 20 25 30 35 40

0

5

10

15

20

25

30

35

40

45A

vera

ge Q

ueui

ng L

engt

h

Arrival Request Rate (RPS)

Endorse Orderer Commit

(a) N = 1

10 20 30 40 50 60 70 80 90 100 110

0

2

4

6

8

10

12

14

16

18

20

Ave

rage

Que

uing

Len

gth

Arrival Request Rate (RPS)

Endorse Orderer Commit

(b) N = 3

10 20 30 40 50 60 70 80 90 100 110 120 130

0

2

4

6

8

10

12

14

Ave

rage

Que

uing

Len

gth

Arrival Request Rate (RPS)

Endorse Orderer Commit

(c) N = 5

Fig. 7: Queuing Length with Different N .

10 15 20 25 30 35 400

200

400

600

800

1000

1200

1400

Ave

rage

Lat

ency

(ms)

Arrival Request Rate (RPS)

Endorse Orderer Commit

(a) N = 1

10 20 30 40 50 60 70 80 90 100 1100

100

200

300

400

500

600

700

800

Ave

rage

Lat

ency

(ms)

Arrival Request Rate (RPS)

Endorse Orderer Commit

(b) N = 3

15 30 45 60 75 90 105 120 135 1500

200

400

600

800

1000

1200

Ave

rage

Lat

ency

(ms)

Arrival Request Rate (RPS)

Endorse Orderer Commit

(c) N = 5

Fig. 8: Average Latency of Each Phase with Different N .

model which has been validated. Because each place in thismodel represent a specific processing phase, the performancemetrics for each processing phase such as the average latencycan be obtained by simulating the model. Consider the averagelatency and queuing length of each phase (i.e., places likePwait h) as the metrics of system busyness, the longer queuinglength and latency lead to the worse performance. Fig. 7 showsthe average queuing length of different phases with differentvalues of N , i.e., 1, 3 and 5. When N is small, the queuinglength of committing phase increases significantly while theother two curves remain stable. However, when N = 5, thequeuing length of endorsement phase increases rapidly. Thisis because the increase of N is equivalent to decreasing thearrival rate of the committing phase. When N is large enough,the service rate of the committing phase is more than thearrival rate of requests while the endorsement phase is lessthan it. Fig. 8 shows the average latency of the three majorprocessing phases with different N , and the results matchthe previous analysis. Therefore, it can be found that thecommitting phase is the system bottleneck in case of smallN while the endorsement phase is instead in case of largeN . Thus, it is an effective approach to improve the systemperformance by determining the value of N in the practicaluse of a Hyperledger-based system.

D. Performance Optimization

The performance metrics in Fig. 6 point out that the valueof N has a great influence on the maximum throughput ofthe Hyperledger-based system. In Section IV, a mathematicalmethod is proposed to determine the configuration parametersof ordering service, in order to achieve the maximum through-put. Considering the analysis results of the system bottleneckin the previous subsection, the proposed method is actually anapproach to improve the performance of the committing phase.According to (10), when the function f(N) is determined, theappropriate n and t can be obtained. At this point, the systembottleneck is the endorsement phase.

We leverage the approach of curve fitting to approximatef(N). The experimental data and fitting result are shown inFig. 9. It is obvious that the latency of committing phase islinear with N within a certain range and the function is

h(N) = 1000/f(N) = 25.06 + 1.57 ∗N. (11)

Therefore, the maximum throughput of the system is

Θmax = max min(µe,1000

25.06n + 1.57

,1000

25.06bt∗µec+1 + 1.57

).

(12)

To solve (12), we have

Θmax = µe. (13)

n ≥ d 25.06 ∗ µe1000− 1.57 ∗ µe

e, t ≥ 26.63 ∗ µe − 1000

µe ∗ (1000− 1.57 ∗ µe). (14)

According to the results in table I, let µe = 143. Thus, thetheoretical results show that the system can reach a maximumthroughput of 143 RPS when n ≥ 5 and t ≥ 0.026 (s). Notethat t is determined under the premise of λ = µe, a largerλ can ignore the impact of t. However, restricting the rangeof t can prevent t from being the bottleneck. Generally, itis not suggested to assign n and t with a large value in thepractical environment because the large value can lead to poorperformance with a low arrival rate λ.

To validate the theoretical results above, a series of ex-periments have been conducted on the running system. Foreach N , we gradually increased the arrival rate in Locustuntil the throughput reaches the saturation point. Fig. 10depicts the relationship between N and the maximum systemthroughput and compares the simulation and experimentalresults. Our goal is to determine the maximum throughputand the corresponding N . The experimental results show thatwhen N is larger than 5, the maximum throughput becomesfloor at around 145 RPS. Thus, continuing to increase N whenN ≥ 5 can not significantly improve the throughput, which isin line with the theoretical results (i.e., 143 RPS when n ≥ 5and t ≥ 0.026 (s)). Therefore, our proposed approach candetermine the appropriate configuration of ordering service. Asfor other Hyperledger-based systems with distinct underlyingnetworks, the proposed analytic method can be also appliedto achieve the higher system throughput.

The steps to determine the configuration parameters aredescribed as follows, i.e.,:• Step 1: Obtain the service rate of endorsement phase (µe)

and committing phase (µc) by conducting experiments onthe system.

• Step 2: Determine which processing phase of the trans-action flow is the system bottleneck (The endorsementphase in our case).

• Step 3: Calculate t (BatchTimeout) and n (MaxMessage-Count) by the proposed formula.

• Step 4: Confirm the conclusion through experiments.Actually, this analysis method can be applied to a more

complex system with more organizations and channels. Theonly difference is how to get the µe and µc because the numberof channels and peers can affect the service rate. Thus, eventhough the system in the paper only contains two organizationsand one channel, the proposed analysis method is scalable andconvincing.

VI. CONCLUSION

In this paper, we have proposed a GSPN model for ablockchain system based on Hyperledger Fabric v1.2. Thismodel depicts the transaction flow of Hyperledger Fabric indetails and is aiming to evaluate the system performances.

0 5 10 15 20 25 30 35 40 45 50 55 600

20

40

60

80

100

120 Experimental Result Fitted Curve

Ave

rage

Lat

ency

(ms)

Number of Transactions (N)

Fig. 9: Latency of Committing Phase with Different N .

1 2 3 4 5 6 7 820

40

60

80

100

120

140

160M

ax T

hrou

ghpu

t (RP

S)

Number of transaction (N)

Experiment Simulation

Fig. 10: Maximum throughput with Different N .

Extensive experiments have been conducted on a real-timesystem to validate our model. The results show that thenumber of transactions in a block significantly affect thesystem performance. Based on this model, we have analyzedthe performance bottleneck with different configurations of theordering service. With an increasing number of transactionsin a block, the system bottleneck changes from committingphase to endorsement phase. In addition, we have presenteda configuration selection approach to determine the configu-ration parameters of ordering service, in order to achieve themaximum throughput. Our simulation results have shown thatwhen the number of transactions contained in a block exceeds5 and the waiting time exceeds 0.26 seconds, the throughputcan reach 143 RPS, which is in line with the real system.Furthermore, the conclusions of this paper are instructive for

the future work. Our next plan is to further improve the overallsystem performance by optimizing the endorsement phase. Forexample, adding more endorsers in each organization to loadbalance the endorsement is an effective approach.

ACKNOWLEDGMENT

This work is supported by Chinese National Natural ScienceFoundation (61671089) and China Unicom Network Technol-ogy Research Institute.

REFERENCES

[1] Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” WhitePaper, 2008.

[2] Z. Yang, K. Yang, L. Lei, K. Zheng and V. C. M. Leung, “Blockchain-Based Decentralized Trust Management in Vehicular Networks,” in IEEEInternet of Things Journal, vol. 6, no. 2, pp. 1495-1505, April 2019.

[3] A. Azaria, A. Ekblaw, T. Vieira and A. Lippman, “MedRec: UsingBlockchain for Medical Data Access and Permission Management,”2016 2nd International Conference on Open and BigData (OBD),Vienna, 2016, pp. 25-30.

[4] A. Stanciu, “Blockchain Based Distributed Control System for EdgeComputing,” 2017 21st International Conference on Control Systemsand Computer Science (CSCS), Bucharest, 2017, pp. 667-671.

[5] X. Xiong, K. Zheng, R. Xu, W. Xiang and P. Chatzimisios, “Low powerwide area machine-to-machine networks: key techniques and prototype,”in IEEE Communications Magazine, vol. 53, no. 9, pp. 64-71, September2015.

[6] F. Liu, K. Zheng, W. Xiang and H. Zhao, “Design and PerformanceAnalysis of An Energy-Efficient Uplink Carrier Aggregation Scheme,”in IEEE Journal on Selected Areas in Communications, vol. 32, no. 2,pp. 197-207, February 2014.

[7] Sachin S. Shetty; Charles A. Kamhoua; Laurent L. Njilla, “Permissionedand Permissionless Blockchains,” in Blockchain for Distributed SystemsSecurity, IEEE, 2019, pp.193-204.

[8] S. Wang, Y. Yuan, X. Wang, J. Li, R. Qin and F. Wang, “An Overviewof Smart Contract: Architecture, Applications, and Future Trends,” 2018IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018, pp. 108-113.

[9] M. Wohrer and U. Zdun, “Smart contracts: security patterns in theethereum ecosystem and solidity,” 2018 International Workshop onBlockchain Oriented Software Engineering (IWBOSE), Campobasso,2018, pp. 2-8.

[10] Elli Androulaki, Artem Barger, Vita Bortnikov, et al,. 2018. “Hy-perledger fabric: a distributed operating system for permissionedblockchains.” In Proceedings of the Thirteenth EuroSys Conference(EuroSys 18). ACM, New York, NY, USA, Article 30, 15 pages.

[11] P. Yuan, X. Xiong, L. Lei and K. Zheng, “Design and Implementationon Hyperledger-Based Emission Trading System,” in IEEE Access, vol.7, pp. 6109-6116, 2019.

[12] M. Raikwar, S. Mazumdar, S. Ruj, S. S. Gupta, A. Chattopadhyay, K.Lam, “A blockchain framework for insurance processes”, Proc. 9th IFIPInt. Conf. New Technol. Mobility Secur. (NTMS), pp. 1-4, 2018.

[13] Q. Liu, Q. Guan, X. Yang, H. Zhu, G. Green and S. Yin,“Education-Industry Cooperative System Based on Blockchain,” 20181st IEEE International Conference on Hot Information-Centric Network-ing (HotICN), Shenzhen, 2018, pp. 207-211.

[14] Q. Zheng, K. Zheng, H. Zhang and V. C. M. Leung, “Delay-OptimalVirtualized Radio Resource Scheduling in Software-Defined VehicularNetworks via Stochastic Learning,” in IEEE Transactions on VehicularTechnology, vol. 65, no. 10, pp. 7857-7867, Oct. 2016.

[15] A. Baliga, N. Solanki, S. Verekar, A. Pednekar, P. Kamat and S.Chatterjee, “Performance Characterization of Hyperledger Fabric,” 2018Crypto Valley Conference on Blockchain Technology (CVCBT), Zug,2018, pp. 65-74.

[16] T. T. A. Dinh, J. Wang, G. Chen, R. Liu, B. C. Ooi, and K.-L. Tan,“BLOCKBENCH: A Framework for Analyzing Private Blockchains,” inACM International Conference on Management of Data, ser. SIGMOD,2017, pp. 10851100.

[17] Qassim Nasir, Ilham A. Qasse, Manar Abu Talib, and Ali Bou Nassif,“Performance Analysis of Hyperledger Fabric Platforms,” Security andCommunication Networks, vol. 2018, Article ID 3976093, 14 pages,2018. https://doi.org/10.1155/2018/3976093.

[18] S. Pongnumkul, C. Siripanpornchana and S. Thajchayapong, “Perfor-mance Analysis of Private Blockchain Platforms in Varying Workloads,”2017 26th International Conference on Computer Communication andNetworks (ICCCN), Vancouver, BC, 2017, pp. 1-6.

[19] Y. Hao, Y. Li, X. Dong, L. Fang and P. Chen, “Performance Analysisof Consensus Algorithm in Private Blockchain,” 2018 IEEE IntelligentVehicles Symposium (IV), Changshu, 2018, pp. 280-285.

[20] Thakkar, Parth, Senthil Nathan, and Balaji Vishwanathan. “PerformanceBenchmarking and Optimizing Hyperledger Fabric Blockchain Plat-form.” arXiv preprint arXiv:1805.11390 (2018).

[21] L. Lei, Y. Zhang, X. S. Shen, C. Lin and Z. Zhong, “PerformanceAnalysis of Device-to-Device Communications with Dynamic Interfer-ence Using Stochastic Petri Nets,” in IEEE Transactions on WirelessCommunications, vol. 12, no. 12, pp. 6121-6141, December 2013.

[22] K. Zheng, F. Liu, L. Lei, C. Lin and Y. Jiang, “Stochastic PerformanceAnalysis of a Wireless Finite-State Markov Channel,” in IEEE Transac-tions on Wireless Communications, vol. 12, no. 2, pp. 782-793, February2013.

[23] H. Wang, L. Lei and K. Zheng, “Flow-level performance analysisof random wireless network using stochastic Petri Nets,” 2016 23rdInternational Conference on Telecommunications (ICT), Thessaloniki,2016, pp. 1-5.

[24] K. Zheng, H. Meng, P. Chatzimisios, L. Lei and X. Shen, “An SMDP-Based Resource Allocation in Vehicular Cloud Computing Systems,” inIEEE Transactions on Industrial Electronics, vol. 62, no. 12, pp. 7920-7928, Dec. 2015.

[25] H. Sukhwani, J. M. Martnez, X. Chang, K. S. Trivedi and A. Rindos,“Performance Modeling of PBFT Consensus Process for PermissionedBlockchain Network (Hyperledger Fabric),” 2017 IEEE 36th Symposiumon Reliable Distributed Systems (SRDS), Hong Kong, 2017, pp. 253-255.

[26] H. Sukhwani, N. Wang, K. S. Trivedi and A. Rindos, “PerformanceModeling of Hyperledger Fabric (Permissioned Blockchain Network),”2018 IEEE 17th International Symposium on Network Computing andApplications (NCA), Cambridge, MA, 2018, pp. 1-8.