freebsd on freescale qoriq data path acceleration architecture devices · 2015-12-16 · freebsd on...

Post on 28-Jun-2020

1 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

FreeBSD on Freescale QorIQ Data PathAcceleration Architecture Devices

Piotr Zięcikkosmo@semihalf.com

BSDCan 2012 9-13 May, 2012

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,

I Hardware description:

I Data Path Acceleration Architecture.

I Software description:

I Porting process.

I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.I Software description:

I Porting process.

I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.

I Software description:

I Porting process.

I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.I Software description:

I Porting process.I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.I Software description:

I Porting process.

I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.I Software description:

I Porting process.I Current state,

I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Presentation outline

I Introduction,I Hardware description:

I Data Path Acceleration Architecture.I Software description:

I Porting process.I Current state,I Future work.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:

I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps

(4k ring buffer full in 2.8 ms).

I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:

I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps

(4k ring buffer full in 2.8 ms).

I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.

I Gigabit Ethernet: 1.46 Mpps

(4k ring buffer full in 2.8 ms).

I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps

(4k ring buffer full in 2.8 ms).I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:

I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:I High interrupt rate and/or latency,

I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:I High interrupt rate and/or latency,I DMA memory management overhead,

I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,

I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Introduction

Networks are too fast!

I High packet rate:I Fast Ethernet: 182 kpps.I Gigabit Ethernet: 1.46 Mpps (4k ring buffer full in 2.8 ms).

I Problems:I High interrupt rate and/or latency,I DMA memory management overhead,I Lock congestion,I Packet parsing/inspection overhead.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,

I Successor of the PowerQUICC family,I Up to 8 e500mc Cores,I Up to 2x 10GbE + 8x 1GbE,I Hardware Virtualization Support,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,I Successor of the PowerQUICC family,

I Up to 8 e500mc Cores,I Up to 2x 10GbE + 8x 1GbE,I Hardware Virtualization Support,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,I Successor of the PowerQUICC family,I Up to 8 e500mc Cores,

I Up to 2x 10GbE + 8x 1GbE,I Hardware Virtualization Support,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,I Successor of the PowerQUICC family,I Up to 8 e500mc Cores,I Up to 2x 10GbE + 8x 1GbE,

I Hardware Virtualization Support,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,I Successor of the PowerQUICC family,I Up to 8 e500mc Cores,I Up to 2x 10GbE + 8x 1GbE,I Hardware Virtualization Support,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

I Most recent Freescale Communication SoCs,I Successor of the PowerQUICC family,I Up to 8 e500mc Cores,I Up to 2x 10GbE + 8x 1GbE,I Hardware Virtualization Support,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Freescale QoriQ DPAA Family

QorIQ P3041 Communication Processor (source: P3041 Fact Sheet)

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,I Frame Manager (NICs),I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,

I Queue Manager,I Frame Manager (NICs),I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,

I Frame Manager (NICs),I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,I Frame Manager (NICs),

I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,I Frame Manager (NICs),I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,I Frame Manager (NICs),I Others:

I Security Accelerator,

I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components

I Buffer Manager,I Queue Manager,I Frame Manager (NICs),I Others:

I Security Accelerator,I Pattern Matching Engine.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Buffer Manager

I Maintains Buffers,I Pools = Set of Buffers,I Pools are managed by software,I Software and hardware may directly allocate/free buffer.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Buffer Manager

I Maintains Buffers,

I Pools = Set of Buffers,I Pools are managed by software,I Software and hardware may directly allocate/free buffer.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Buffer Manager

I Maintains Buffers,I Pools = Set of Buffers,

I Pools are managed by software,I Software and hardware may directly allocate/free buffer.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Buffer Manager

I Maintains Buffers,I Pools = Set of Buffers,I Pools are managed by software,

I Software and hardware may directly allocate/free buffer.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Buffer Manager

I Maintains Buffers,I Pools = Set of Buffers,I Pools are managed by software,I Software and hardware may directly allocate/free buffer.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Maintains Frames,I Frame Queue = Linked list of Frames,I Work Queue = Linked list of Frame Queues,I Channel = 8 ∗ (Work Queue + Priority).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Maintains Frames,

I Frame Queue = Linked list of Frames,I Work Queue = Linked list of Frame Queues,I Channel = 8 ∗ (Work Queue + Priority).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Maintains Frames,I Frame Queue = Linked list of Frames,

I Work Queue = Linked list of Frame Queues,I Channel = 8 ∗ (Work Queue + Priority).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Maintains Frames,I Frame Queue = Linked list of Frames,I Work Queue = Linked list of Frame Queues,

I Channel = 8 ∗ (Work Queue + Priority).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Maintains Frames,I Frame Queue = Linked list of Frames,I Work Queue = Linked list of Frame Queues,I Channel = 8 ∗ (Work Queue + Priority).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,

I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,

I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,

I Poll Channel: Connected to group of devices.I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:

I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:I Congestion Management (Tail drop, RED/WRED),

I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Components: Queue Manager

I Frames are enqueued to Frame Queues,I Frames are dequeued from Channels,I Channels are attached to SoC components:

I Dedicated Channel: Connected to single device,I Poll Channel: Connected to group of devices.

I Other Queue Manager features:I Congestion Management (Tail drop, RED/WRED),I Frame order restoration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:

I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:

I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:

I 1x 10Gb Ethernet + 5x 1Gb Ethernet.I DMA:

I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:

I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:

I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).

I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),

I DMA Engine,I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:

I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:I Parser,

I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:I Parser,I Key Generator,

I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:I Parser,I Key Generator,I Policer,

I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration ArchitectureComponents: Frame Manager

I MACs:I 1x 10Gb Ethernet + 5x 1Gb Ethernet.

I DMA:I Buffer Manager Interface (QMI).I Queue Manager Interface (BMI),I DMA Engine,

I Frame Processor:I Parser,I Key Generator,I Policer,I Frame Processing Module.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Software Portals

I CPU↔ DPAA communication channels,I Portal = Cache Enabled + Cache Inhibited registers,I Transaction oriented,I Efficient SoC Bus usage.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Software PortalsI CPU↔ DPAA communication channels,

I Portal = Cache Enabled + Cache Inhibited registers,I Transaction oriented,I Efficient SoC Bus usage.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Software PortalsI CPU↔ DPAA communication channels,I Portal = Cache Enabled + Cache Inhibited registers,

I Transaction oriented,I Efficient SoC Bus usage.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Software PortalsI CPU↔ DPAA communication channels,I Portal = Cache Enabled + Cache Inhibited registers,I Transaction oriented,

I Efficient SoC Bus usage.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Data Path Acceleration Architecture

Software PortalsI CPU↔ DPAA communication channels,I Portal = Cache Enabled + Cache Inhibited registers,I Transaction oriented,I Efficient SoC Bus usage.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,

I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,

I Peripheral Access Management Units,

I Logical I/O Device Number,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,

I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Hardware: Other selected features

I Cache Stashing,I Virtualization:

I Additional privilege level in e500mc core,I Peripheral Access Management Units,

I Logical I/O Device Number,I Data Path Acceleration Architecture.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,I Early kernel initialization,I Data Path Acceleration Architecture bring-up,I SMP,

I DPAA in SMP environment,I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,

I Early kernel initialization,I Data Path Acceleration Architecture bring-up,I SMP,

I DPAA in SMP environment,I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,I Early kernel initialization,

I Data Path Acceleration Architecture bring-up,I SMP,

I DPAA in SMP environment,I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,I Early kernel initialization,I Data Path Acceleration Architecture bring-up,

I SMP,I DPAA in SMP environment,

I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,I Early kernel initialization,I Data Path Acceleration Architecture bring-up,I SMP,

I DPAA in SMP environment,

I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Software

I Toolchain,I Early kernel initialization,I Data Path Acceleration Architecture bring-up,I SMP,

I DPAA in SMP environment,I Other peripherals.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Toolchain

I Support for e500v2 (predecessor of the e500mc) hadbeen already present,

I Only minor changes were required in binutils and gcc,I All patches were available from the community andFreescale.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Toolchain

I Support for e500v2 (predecessor of the e500mc) hadbeen already present,

I Only minor changes were required in binutils and gcc,I All patches were available from the community andFreescale.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Toolchain

I Support for e500v2 (predecessor of the e500mc) hadbeen already present,

I Only minor changes were required in binutils and gcc,

I All patches were available from the community andFreescale.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Toolchain

I Support for e500v2 (predecessor of the e500mc) hadbeen already present,

I Only minor changes were required in binutils and gcc,I All patches were available from the community andFreescale.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.S

I First code executed in FreeBSD kernel,I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.S

I TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,

I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.S

I TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,I Architecture depended assembly,

I Prepares environment for C.

PowerPC locore.S

I TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.S

I TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.S

I TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.SI TLB initialization,

I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

locore.SI First code executed in FreeBSD kernel,I Architecture depended assembly,I Prepares environment for C.

PowerPC locore.SI TLB initialization,I Kernel stack initialization.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

e500mc features in locore.S

I Bigger TLB,I Hypervisor privilege level,I Hardware Implementation-Dependent Registers (HIDs).

New TLB also affects pmap(9).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

e500mc features in locore.SI Bigger TLB,

I Hypervisor privilege level,I Hardware Implementation-Dependent Registers (HIDs).

New TLB also affects pmap(9).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

e500mc features in locore.SI Bigger TLB,I Hypervisor privilege level,

I Hardware Implementation-Dependent Registers (HIDs).

New TLB also affects pmap(9).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

e500mc features in locore.SI Bigger TLB,I Hypervisor privilege level,I Hardware Implementation-Dependent Registers (HIDs).

New TLB also affects pmap(9).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Early kernel initialization

e500mc features in locore.SI Bigger TLB,I Hypervisor privilege level,I Hardware Implementation-Dependent Registers (HIDs).

New TLB also affects pmap(9).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSw

I A packet processing framework,I OS Agnostic,I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSw

I A packet processing framework,I OS Agnostic,I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSwI A packet processing framework,

I OS Agnostic,I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSwI A packet processing framework,I OS Agnostic,

I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSwI A packet processing framework,I OS Agnostic,I Greatly reduces development time,

I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSwI A packet processing framework,I OS Agnostic,I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

DPAA Bring-up

NetCommSwI A packet processing framework,I OS Agnostic,I Greatly reduces development time,I Propertiary licensed.

Thanks to Freescale!Buffer, Queue and Frame Manager drivers are now availableunder the BSD license.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

NetCommSw Driver Model

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:

I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:

I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:

I void *XX_Malloc(unsigned int size)I Physical to Virtual translation is required:

I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:

I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:

I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,

I Ambiguous,I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:

I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:I Keep list of all active mappings in vm_page structure,

I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,

I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

XX Routines

I Simple routines for basic OS functions:I void *XX_Malloc(unsigned int size)

I Physical to Virtual translation is required:I Not supported by the FreeBSD kernel,I Ambiguous,

I Solution:I Keep list of all active mappings in vm_page structure,I pmap_enter() and pmap_remove() manages the list,I XX Routines: PA→ vm_page→ the mappings list→ VA.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Wrapper Drivers

I OS↔ NetCommSw API translation layer,I Access serialization layer,I Written as simple newbus attachments,I Export own API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Wrapper Drivers

I OS↔ NetCommSw API translation layer,

I Access serialization layer,I Written as simple newbus attachments,I Export own API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Wrapper Drivers

I OS↔ NetCommSw API translation layer,I Access serialization layer,

I Written as simple newbus attachments,I Export own API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Wrapper Drivers

I OS↔ NetCommSw API translation layer,I Access serialization layer,I Written as simple newbus attachments,

I Export own API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Wrapper Drivers

I OS↔ NetCommSw API translation layer,I Access serialization layer,I Written as simple newbus attachments,I Export own API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,

I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,

I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,

I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,

I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Frame Manager Wrapper Driver

I Consists of several NetCommSw submodules,I Each submodule have own API,I A single Frame Manger Wrapper Driver:

I Performs initialization of common parts,I Manages internal FMan resources,I Exports single, simplified API.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver

I Classic NIC driver from OS perspective,I Uses BMan, QMan and FMan wrapper drivers,I Binds DPAA parts together.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver

I Classic NIC driver from OS perspective,

I Uses BMan, QMan and FMan wrapper drivers,I Binds DPAA parts together.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver

I Classic NIC driver from OS perspective,I Uses BMan, QMan and FMan wrapper drivers,

I Binds DPAA parts together.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver

I Classic NIC driver from OS perspective,I Uses BMan, QMan and FMan wrapper drivers,I Binds DPAA parts together.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:

I RX buffer pool management.

I Queue Manager:

I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:

I RX buffer pool management.I Queue Manager:

I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:

I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:

I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:

I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:I MAC abstraction,

I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: DPAA Usage

I Buffer Manager:I RX buffer pool management.

I Queue Manager:I Reads/Writes frames from/to queues (associated withMACs).

I Frame Manager:I MAC abstraction,I Data flow configuration.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: TX Path

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

dTSEC Driver: RX Path

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP Bring-Up

I Based on existing e500v2 implementation,I Issues with more than 2 cores:

I IPI Multicasting

I DPAA-related issues.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP Bring-Up

I Based on existing e500v2 implementation,

I Issues with more than 2 cores:

I IPI Multicasting

I DPAA-related issues.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP Bring-Up

I Based on existing e500v2 implementation,I Issues with more than 2 cores:

I IPI MulticastingI DPAA-related issues.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP Bring-Up

I Based on existing e500v2 implementation,I Issues with more than 2 cores:

I IPI Multicasting

I DPAA-related issues.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP Bring-Up

I Based on existing e500v2 implementation,I Issues with more than 2 cores:

I IPI MulticastingI DPAA-related issues.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,

I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,

I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,

I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,

I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,

I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Mapping

I Dedicated portal for each CPU,I Same portal address on each CPU,I Such device mapping was not supported:

I Introduction of "shared" bit,I Implemented using user-defined bits in TLB entry,I Only marked entries are propagated to other cores,I Private are set on each CPU.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),

I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,

I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,

I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,

I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!

I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Configuration

I Has to be performed on each portal (CPU),I Only boot CPU can execute configuration during boot,I Other cores must initialize portals on demand:

I This may happen any time,I Portal configuration includes IRQ request,I But intr_event_create() may sleep!I Solution: Interrupt preallocation in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Interrupts

I Portal Interrupts must be bound to particular CPU,I Solution: Interrupt thread binding layer in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Interrupts

I Portal Interrupts must be bound to particular CPU,

I Solution: Interrupt thread binding layer in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

SMP vs DPAA: Issues

Portal Interrupts

I Portal Interrupts must be bound to particular CPU,I Solution: Interrupt thread binding layer in XX Routines.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,I USB Controller (EHCI compliant),I SD/MMC Controller,I I2C Controller,I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,

I USB Controller (EHCI compliant),I SD/MMC Controller,I I2C Controller,I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,I USB Controller (EHCI compliant),

I SD/MMC Controller,I I2C Controller,I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,I USB Controller (EHCI compliant),I SD/MMC Controller,

I I2C Controller,I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,I USB Controller (EHCI compliant),I SD/MMC Controller,I I2C Controller,

I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Other Peripherals

I PCI Express Bus,I USB Controller (EHCI compliant),I SD/MMC Controller,I I2C Controller,I IDMA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:

I Speed: 897 Mbit/sI CPU Utilization: 30%,I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:

I Speed: 897 Mbit/sI CPU Utilization: 30%,I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:

I Speed: 897 Mbit/sI CPU Utilization: 30%,I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:I Speed: 897 Mbit/s

I CPU Utilization: 30%,I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:I Speed: 897 Mbit/sI CPU Utilization: 30%,

I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

I Supported chips: P2041, P3041 and P5020 (in 32-bitmode),

I Simple iperf test:I Speed: 897 Mbit/sI CPU Utilization: 30%,I Of which 90% was interrupt servicing.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Current State and Results

p3041# vmstat -iinterrupt total rateirq121: bman0 1 0irq120: qman0 2784930 3584irq122: qman0 2194130 2823irq124: qman0 2263079 2912irq126: qman0 2148167 2764(...)

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:

I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:

I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,

I Hardware Checksumming,I Jumbo Frames,

I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,

I Jumbo Frames,I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:

I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:I Pattern Matching Engine,

I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:I Pattern Matching Engine,I Security Engine,

I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Future work

I More networking features:I Pooling Mode,I Hardware Checksumming,I Jumbo Frames,

I More peripherals:I Pattern Matching Engine,I Security Engine,I SATA Controller.

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Acknowledgments

I Michał Dubiel (Semihalf),I Rafał Jaworowski (Semihalf, The FreeBSD Project),I Phil Brownfield (Freescale)I Zbigniew Bodek, Piotr Nowak, Tomasz Nowicki, JanSięka, Łukasz Wójcik (all Semihalf).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Acknowledgments

I Michał Dubiel (Semihalf),

I Rafał Jaworowski (Semihalf, The FreeBSD Project),I Phil Brownfield (Freescale)I Zbigniew Bodek, Piotr Nowak, Tomasz Nowicki, JanSięka, Łukasz Wójcik (all Semihalf).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Acknowledgments

I Michał Dubiel (Semihalf),I Rafał Jaworowski (Semihalf, The FreeBSD Project),

I Phil Brownfield (Freescale)I Zbigniew Bodek, Piotr Nowak, Tomasz Nowicki, JanSięka, Łukasz Wójcik (all Semihalf).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Acknowledgments

I Michał Dubiel (Semihalf),I Rafał Jaworowski (Semihalf, The FreeBSD Project),I Phil Brownfield (Freescale)

I Zbigniew Bodek, Piotr Nowak, Tomasz Nowicki, JanSięka, Łukasz Wójcik (all Semihalf).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

Acknowledgments

I Michał Dubiel (Semihalf),I Rafał Jaworowski (Semihalf, The FreeBSD Project),I Phil Brownfield (Freescale)I Zbigniew Bodek, Piotr Nowak, Tomasz Nowicki, JanSięka, Łukasz Wójcik (all Semihalf).

FreeBSD on Freescale QorIQ Data Path Acceleration Architecture Devices

The End

Any questions?

top related