how to measure rtos performance
DESCRIPTION
How to Measure RTOS Performance – Colin Walls In the world of smart phones and tablet PCs memory might be cheap, but in the more constrained universe of deeply embedded devices, it is still a precious resource. This is one of the many reasons why most 16- and 32-bit embedded designs rely on the services of a scalable real-time operating system (RTOS). An RTOS allows product designers to focus on the added value of their solution while delegating efficient resource (memory, peripheral, etc.) management. In addition to footprint advantages, an RTOS operates with a degree of determinism that is an essential requirement for a variety of embedded applications. This paper takes a look at “typical” reported performance metrics for an RTOS in the embedded industry.TRANSCRIPT
![Page 1: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/1.jpg)
mentor.com/embedded
Colin [email protected]
with acknowledgement to Faheem Sheikh & Dan Driscoll
How to Measure RTOS Performance
![Page 2: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/2.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 3: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/3.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 4: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/4.jpg)
Introduction – Why?
Desktop computers– Infinite CPU power
– Infinite memory
– Costs nothing
Embedded systems– Enough CPU power – just
– Adequate memory – none to spare
– Power consumption an issue
– Cost normally critical
![Page 5: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/5.jpg)
Introduction – Choosing an RTOS
RTOS solutions– Proprietary [in-house; home brew]
– Commercial– At least 200 product available
– Open source
Asking the right questions is important
Understanding the answers is critical– Pitfalls with misinterpretation
![Page 6: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/6.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 7: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/7.jpg)
RTOS Metrics
Three common categories:– Memory footprint
– Program and data
– Latency– Interrupt and scheduling
– Services performance
No real standardization– Embedded Microprocessor Benchmark Consortium
(EEMBC) not widely adopted– Oriented towards CPU benchmarking
![Page 8: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/8.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 9: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/9.jpg)
Memory Footprint
RAM and ROM requirements of RTOS– On a specific platform
ROM size:– Kernel code
– Read-only data
– Runtime library code
– Maybe in flash; possibly copied to RAM
RAM size:– Kernel data structures
– Global variables
– May need accommodate “ROM”
![Page 10: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/10.jpg)
Memory Footprint – Dependencies
Key factors affect the footprint calculation
CPU architecture– Huge effect on number of instructions
Software configuration– Which kernel components are included
– Scalability ...
Compiler optimization– Reduces code size, but may adversely affect performance
![Page 11: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/11.jpg)
Memory Footprint – Scalability
Service #1
Service #2
Service #3
Service #4
Service #271
RTOS
…rtos_call_1();…rtos_call_3();…rtos_call_157();…
ApplicationCode
Service #1
Service #3
Service #157
RTOScore
Application Memory
![Page 12: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/12.jpg)
Memory Footprint – Measurement
Making measurement need not be difficult
Two key methods:– Memory MAP file
– Generated by standard linkers– Quality and detail varies between tools
– Specific tool– Shows footprint information for selected executable image– Example: objdump
![Page 13: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/13.jpg)
Memory Footprint – Importance
Limited memory availability– Small on-chip memory
– No external memory option
– Application code is priority
Larger systems– Kernel performance a priority
– Place in on-chip memory
– Lock into cache
Using a bootloader– Non-volatile memory and RAM space used
![Page 14: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/14.jpg)
Memory Footprint – Pitfalls
Vendor data can be readily misinterpreted
Look at minimum configuration definition– It may be a tiny, impractical subset
Runtime library functions are often not included
RAM/ROM sizes should have a min/max range– RAM is likely to be application dependent
– ROM driven by kernel configuration– Need minimum “useable” size– Also maximum measure with all services
– Scalability variable – may be different kernel versions
![Page 15: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/15.jpg)
Memory Footprint – Example
Nucleus RTOS kernel is fully scalable
Example: ARM Cortex A8 in ARM mode
Built with Mentor Sourcery CodeBench toolchain
Full optimization for size
ROM = 12-30 Kbytes
RAM = 500 bytes
![Page 16: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/16.jpg)
Memory Footprint – Example
Min has essential kernel services [dynamic memory, threads, semaphores, events, queues]
– Runtime library excluded
Max includes all kernel services
Compiling for Thumb-2 mode reduces ROM by 35%– So Nucleus kernel can use 7.8 Kbytes on a Cortex-M
based controller
![Page 17: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/17.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 18: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/18.jpg)
Interrupt Latency
Definitions of interrupt latency
![Page 19: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/19.jpg)
Interrupt Latency – Definition
Different definitions
System: Time between interrupt assertion and the instant an observable response happens
OS: Duration of when the CPU was interrupted until the start of the corresponding interrupt service routine (ISR)
– This is really OS overhead
– Many vendors refer to this as the latency
– Hence often report zero latency
![Page 20: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/20.jpg)
Interrupt Latency – Measurement
Interrupt response is the sum of two distinct times:
ƮIL = ƮH + ƮOS
where:ƮH is the hardware dependent time
– depends on the interrupt controller on the board as well as the type of the interrupt
ƮOS is the OS induced overhead– Best and worst case scenarios– Worst when kernel disables interrupts
![Page 21: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/21.jpg)
Interrupt Latency – Measurement
Best approach is to record time between interrupts source and response
– Use an oscilloscope
– For example:– One GPIO pin can generate an interrupt– Another pin toggled at the start of the ISR
![Page 22: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/22.jpg)
Interrupt Latency – Importance
Specific types of designs rely on this metric:– Time critical
– Fault tolerant
If high I/O bandwidth is needed, measure the latency of a particular interrupt
Most systems can tolerate interrupt latency of tens of microseconds
![Page 23: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/23.jpg)
Interrupt Latency – Pitfalls
Main danger is interpretation of published figures
Key factors to check:
Hardware configuration– Which platform?
– CPU speed?
– Cache configuration?
– Timer frequency?
– Interrupt controller type?
OS configuration– Where is code running
from? [Flash, SDRAM ...]
– Which interrupt used?
– Code optimized for speed?
– Is metric best or average case?
![Page 24: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/24.jpg)
Interrupt Latency – Example
Nucleus RTOS– ARM Cortex A8
– 600MHz
– Running from SRAM
Average interrupt latency of less than 0.5 microseconds
![Page 25: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/25.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 26: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/26.jpg)
Scheduling Latency – Definition
Performance of RTOS thread scheduler
Very wide variation in measurement technique and interpretation
Two distinct related quantities:– Context switch time
– Scheduling overhead
![Page 27: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/27.jpg)
Scheduling Latency – Definition
Context switch time
![Page 28: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/28.jpg)
Scheduling Latency – Definition
Scheduling overhead
![Page 29: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/29.jpg)
Scheduling Latency – Measurement
Scheduling latency is the maximum of two times:
ƮSL = MAX(ƮSO, ƮCS)
where:ƮSO is the scheduling overhead
– End of ISR to start of task schedule
ƮCS is the time taken to save and restore thread context
![Page 30: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/30.jpg)
Scheduling Latency – Importance
Most systems that have stringent interrupt latency demands also need low scheduling latency
Broadly, systems that are:– Time critical
– Fault tolerant
![Page 31: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/31.jpg)
Scheduling Latency – Pitfalls
Ignoring initial system state– If system is idle, there is no time taken saving context
Key factors to check:
Hardware configuration– Which platform?
– CPU speed?
– Cache configuration?
– Timer frequency?
– Interrupt controller type?
OS configuration– Where is code running
from? [Flash, SDRAM ...]
– Which interrupt used?
– Code optimized for speed?
– Is metric best or average case?
![Page 32: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/32.jpg)
Scheduling Latency – Example
Nucleus RTOS– ARM Cortex A8
– 600MHz
– Running from SRAM
Scheduling latency is 1.3 microseconds
![Page 33: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/33.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 34: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/34.jpg)
Timing Kernel Services
RTOS may have a great many API calls
Timing of keys ones may be of interest– Focus on frequently used API calls
Four key categories:– Threading services
– Synchronization services
– Inter-process communication services
– Memory services
![Page 35: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/35.jpg)
Timing Kernel Services
Threading Services– Control of fundamental kernel functionality:
– Create thread– Start thread– Resume thread– Stop thread
– Many multi-taking applications make heavy use of these calls
![Page 36: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/36.jpg)
Timing Kernel Services
Synchronization Services– Services to synchronize between contexts, like an ISR and
a thread
– Also protection of critical code sections from concurrent access
– e.g. Semaphore may be used by Ethernet ISR to write data into shared memory that is also used by a task
– Timing is important is the application has numerous shared resources or peripherals
![Page 37: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/37.jpg)
Timing Kernel Services
Inter-process Communication Services– Services to share data between multiple threads
– Examples:– FIFOs– Queues– Mailboxes– Event flags
– Many multi-taking applications make heavy use of this type of service
![Page 38: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/38.jpg)
Timing Kernel Services
Memory Services– In a multi-threaded context, kernel is used to manage
dynamic memory
– Allocation and de-allocation times are important– Applications with large data throughput are particularly sensitive
![Page 39: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/39.jpg)
Timing Kernel Services – Pitfalls
Hardware configuration– Which platform?
– CPU speed?
– Cache configuration?
OS configuration– Where is code running from? [Flash, SDRAM ...]
– Code optimized for speed?
– Is metric best or average case?
– Is kernel configured for reduced error checking?
![Page 40: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/40.jpg)
Timing Kernel Services – Example
Nucleus RTOS Kernel Service Time in µS
Task resumption 0.3
Task suspension 0.3
Obtaining a semaphore 0.5
Set an event 0.4
Send message to queue 0.9
Allocate memory 0.2
De-allocate memory 0.7
Allocate partition 0.4
![Page 41: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/41.jpg)
Agenda
Introduction
RTOS Metrics
Memory Footprint
Interrupt Latency
Scheduling Latency
Timing Kernel Services
Conclusions
![Page 42: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/42.jpg)
Conclusions
RTOS performance data provided by vendors can be useful
It can also be misleading if it is misinterpreted
Need a thorough understanding of:– Measurement techniques
– Terminology
– Trade-offs
to conduct fair comparison
Recommendation is to rely on holistic or application-oriented measurements
![Page 43: How to Measure RTOS Performance](https://reader033.vdocuments.us/reader033/viewer/2022051314/54bcee764a795941138b45d0/html5/thumbnails/43.jpg)
mentor.com/embedded
Colin [email protected]
http://blogs.mentor.com/colinwalls
See white paper:
"Measuring RTOS Performance: What? Why? How?"
Thank you