asqf fachgruppe automation/ automation day 1 realtime capable linux: requirements...
Post on 29-Dec-2015
215 Views
Preview:
TRANSCRIPT
ASQF Fachgruppe Automation/ Automation Day 1
Realtime capable Linux: Requirements
(Standard)LinuxRealtime LinuxRealtime kernel
One (physical) address space with n-tasks
Application program 100% source compatibilitybetter binary compatibility
API
Response time: < 10 µs Response time: ~1ms
Response time: < 10 µssame
homogeneity
One process address spacewith n-threads
All tasks haveone common API implemented by
one common library
All threads of a process haveone common API implemented by
one common library
ASQF Fachgruppe Automation/ Automation Day 2
Realtime capable Linux: The important questions
• Is the POSIX-API suitable?
– Only a subset is relevant for realtime applications
– The mechanism for the notification of asynchronous events determine widespread usage
• Can it provide for competitive performance?
– Ideally it should be as good as specialized realtime kernels
– Ultimately cycle times of 31,25 µs should be possible
• What about additional changes to the Linux kernel ?
– Ideally none
– Only minimal changes allow for long time support
ASQF Fachgruppe Automation/ Automation Day 3
Thread management– Creation,Termination, Scheduling
Synchronization – Mutex – Spinlock
Communication – POSIX Message Queue– Semaphore– Shared Memory
Timer– types (periodic, one shot, absolute, relative)– resolution– clock tick– notification
The POSIX-API: Minimal set for realtime tasks
needs improvements
ASQF Fachgruppe Automation/ Automation Day 4
The POSIX Notification-API: The Timer-API
Timer sigeventobject
SIGEV_THREAD
CLOCK_REALTIME
SIGEV_SIGNAL
SIGEV_NONE
timer_create (clockid_t clockid, timer_t *timerid, struct sigevent *evp);
Problems:SIGEV_THREAD:
The implementation within the glibc is not realtime capable
another implementation fixes it (deterministic, performance improvement by a factor of 10)
SIGEV_SIGNAL:Sending a signal to the entire process is not deterministic. Potentially the entire thread list of a process must be scanned.
minimal overhead and deterministic behavior require sending to a thread (Linux can handle it)
no response based on signal handlers (signal handlers are for synchronous exceptions only)
having explicit wait only allows for extremely fast notification.
ASQF Fachgruppe Automation/ Automation Day 6
The POSIX Notification-API: Extensions(1)
t
timer fires
sigwait()Scenario a
sigwait()Scenario b
timeroverrun
sigwait()Scenario c
Can be queried This situation must get signalled
An additional call sigevent_set_notification() makes sending the signal to a thread available at the API level. This call maps the threadid of the API to the tid of the kernel allows via an additional encoding of the sigev_notify field for
supervision of timeliness implementation with best possible deterministic behavior
The existing sigevent structure needs no changes.
timer fires timer fires
POSIX definition (not very helpful)
ASQF Fachgruppe Automation/ Automation Day 7
The POSIX Notification-API: Extensions(2) A device must be a possible source of a timer tick (clock)
Example: A Profinet Realtime Ethernet Controller ERTEC provides for a 250 µsec tick, not necessarily synchronous with CLOCK_REALTIME additional call register_clock()
A firing timer is one special type of an event arbitrary asynchronous IO events should get signalled the same way additional call event_create()
Timer
sigeventobject
SIGEV_THREAD (realtime capable implementation of the library module)
CLOCK_REALTIME SIGEV_SIGNAL (inhibited)
direct IO-Event
create_event()
Timer
DeviceDriver
clock
register_clock()
sigevent_set_notification()
Signal to thread (optional with supervision)
Extensions to the POSIX API
ASQF Fachgruppe Automation/ Automation Day 8
Realtime capable Linux: Base concept
Linuxkernel
realtime kernel
threads controlled by realtime scheduler
threads controlledby Linux-Scheduler
system call entryexactly the same for both kernels
Ipipe (ADEOS)
Common Library
Linux ProcessPOSIX API
HW-Interrupts
Soft-Interrupts
static priorities(POSIX-counting)
99
0
99
0
realtimekernel Linux
Linux + Realtime
boundary implementation defined
Linux
ASQF Fachgruppe Automation/ Automation Day 9
Typical partitioning of a realtime thread setup phase time critical response path
Much functionality needed but neither
• realtime requirements nor
• high performance requirements
For all calls not implemented, the thread temporarily runs under the control of the Linux scheduler
Wait for event
event
Realtime capable Linux: Functionality of the realtime kernel
minimal functionality needed
minimal realtime kernel- timer functions and its notification
- communication and synchronization
- scheduling
ASQF Fachgruppe Automation/ Automation Day 10
Realtime capable Linux: Limits of homogeneity
Linux Kernel
Realtime Kernel
threads managed by therealtime scheduler
threads managedby theLinux scheduler
Ipipe
Linux process
HW-Interrupts
Soft-Interrupts
Signal source eg. timer
Signal source eg. IO event
A signal can only get sent to that domain which created the signal source This restriction is
important for best possible performance (allows for separation of implementation) acceptable and manageable (violations are reported)
ASQF Fachgruppe Automation/ Automation Day 11
Realtime capable Linux: Implementation Kernel
Two additional loadable modules:• A device driver
– functional extensions of the API (API-calls ioctl() system calls, additional internal interfaces for realtime capable device drivers)
– Interface for initialization/configuration of the realtime module eg. registration of a process as realtime process
• The realtime module (20K for x86)
– Functionality of the realtime kernel
– Communication between realtime and Linux kernel
Three modified kernel files:– futex.c (hook), posix_timers.c (clock registration), mqueue.c
Ipipe modifications:– Systemcall handling (additional hook at system call exit + optimization)
– Interrupt handling (optimization, support for truly preemptive realtime kernel)
ASQF Fachgruppe Automation/ Automation Day 12
Realtime capable Linux: Implementation glibc
Modified functions: • Initialization
– Registration of the realtime process
• Spinlock function also for static priorities• Realtime capable implementation for SIGEV_THREAD
Additional functions• Device driver functions via ioctls
– Registration of a timer clocks– Registration of an (IO-)event
• Thread specific sigevent notification (the kernel is able to do it) • fast message queues
ASQF Fachgruppe Automation/ Automation Day 13
Testscenario:
• Measurement with getuid(), since getpid() is handled inside glibc
• Measurement with time stamp counter (94 clocks resp. 12/5 clocks)
• glibc-2.3.6 generated with either int 80 or sysenter system entry (…/…)
Cel. 2,8 GHz1000 clocks = 357 ns
Athlon 2 GHz1000 clocks = 500 ns
343/147 Linux 2.6.15.4 original
ipipe original
1124/609
1397/914
Linu
x D
oma
inR
ealti
me
D
oma
in
Lin
ux +
Ipip
e pa
tch
+ AuD patch
+ AuD optimized
+ AuD patch
+ AuD optimized
1260/767
987/494 289/199
383/257
Realtime capable Linux: System call overhead in clocks
317/255
466/338
987/494
1155/641 /185
ASQF Fachgruppe Automation/ Automation Day 14
high priority thread
sem_post()/mq_send()
sem_wait()/mq_receive()
up
helper thread of lowest priority
down
Low priority thread must have
a chance to proceed to sem_wait() / mq_receive()
sem_wait()/mq_receive()
sem_post()/mq_send()
low priority thread
Realtime capable Linux: Performance inter-thread communication
Testscenario:Two threads communicating either
• via semaphores or
• through POSIX message queues
Note:Times for down are always longer than for up, since an additionalsem_wait() / mq_receive()-call is included.
ASQF Fachgruppe Automation/ Automation Day 16
RealtimeDomain
LinuxDomain
up down
2268
2615 4547
3980
3801
3361
3539
3780
4715
5377
15992
13304
744 1279
1071
1519 2075
4374
updown
down
down
up
up
Realtime capable Linux: Interthread communication in clocks Celeron 2,8 GHz
1000 clocks = 357 ns
Athlon 64 2,0 GHz 1000 clocks = 500 ns
up down up down
1963 2509
2530
3241 3912
6174
up down
Semaphore Message Queue Semaphore Message Queue
6721 8390
7266 19825
4715 7004
optimized
836 1417
1297 4943
1647 2215
2050 2771
2827 6666
3335 4197 7655 10364
7130 18575
5534 8106
System call entry: sysenter
Times for Realtime Linux • first switch to interrupted Linux thread• afterwards switch to woken up Linux thread (message queue)• semaphores (futex) need an additional proxy thread for futex_wake
ASQF Fachgruppe Automation/ Automation Day 17
ISR response
thread response
min max
525
1785
2688
284
3287
16149
17441
3276
min max
1024
1010
101
9446
Athlon64 2,0 GHz 1000 clocks = 500 ns
Celeron 2,8 GHz 1000 clocks = 357 ns
Realtime capable Linux: Interrupt response time in clocks Test scenario:• An application thread issues via a driver an
ipipe soft interrupt
• The ISR resumes via send_event another application thread (waiting with sigtimedwait())
• ISR frequency varied between 100 µsec and 1 sec
• System load generated via GUI (Eclipse-Start, file transfer, etc.)
The minimal figures are of interest in several respect:
• for high interrupt frequencies representative for the average and therefore the load
• typical for path length, and therefore useful for scalability projections
• What can be achieved with less powerful HW (e.g. SOC with mediocre memory interface)?
• for a dedicated processor of a multicore system (private TLB, L1-Cache, L2-Cache) the guaranteedworst case time comes closer to the minimal figure.
224 1382
1264 9240
ASQF Fachgruppe Automation/ Automation Day 18
Realtime capable Linux: Positioning to RT_PREEMPT
• The kernel implementation is transparent to application programs – An application binary for a dual domain system runs without any changes on a Linux
only system and therefore also on a RT_PREEMPT system
• Possible coexistence scenarios with RT_PREEMPT – RT_PREEMPT as an alternative if no extremely strong realtime requirements
– An additional realtime domain
• for extremely strong requirements for response time and thread communication,
• if demonstration is required (proof) tiny kernel, short paths large complex kernel, lengthy paths, nested PI mutexes, RCU locks
• Commonalities with RT_PREEMPT– The same modifications of the glibc are needed.
– The same kernel enhancements (registering a clock, realtime capable POSIX message queues) are needed
ASQF Fachgruppe Automation/ Automation Day 19
Realtime capable Linux: Conclusion
• The POSIX-Realtime-Extensions need some fine tuning
– Minimal extensions to the API definition improve usability and allow for best possible performance
• Realtime under Linux can be provided transparently
– The performance figures are comparable to specialized realtime kernels
• Ipipe is a sustainable base
– Small optimizations improve system call handling and interrupt response
top related