Instructor:

Rahul Mangharam Levine 272 (Office) and Levine 279 (Lab) Email: rahulm@seas.upenn.edu Web: http://www.seas.upenn.edu/~rahulm Office hours: Tue and Thu 1:30-2:30pm in Levine 279

Teaching Assistants: Detkin Lab Thejas Kesari thejas@seas.upenn.edu Office Hours: Mon 10-11am & Sun 1-2pm Jack Harkins harkj@seas.upenn.edu Office Hours: Fri 3-4pm & Sun 2-3PM Nitesh Singh niteshs@seas.upenn.edu Office Hours: Thu 6-7pm & Sat 4-5pm Grayson Honan graysonh@seas.upenn.edu Office Hours: Thu 3:30-4:30pm & Sun 3-4pm

Class Schedule: Lecture: Tuesday and Thursday 12:00pm – 1:20pm

Active Learning Center, Room 401B, 3401 Walnut Street Lab : Friday 4:00-7:00pm Detkin Lab, Moore Building

1. Course Description: This is a hands-on course on the theory and practice of developing real-time and embedded systems. The course provides an integrated approach to developing low-power systems with hardware, software, sensors, actuators, controllers and networking. Students will learn to develop a programmable embedded platform from scratch, interface a variety of sensors and actuators for interactive systems, build a gaming system, program an emulator to play retro games, control an autonomous robot and write an RTOS kernel from scratch. 1.1 Topics covered include:

• Embedded architectures of modern 8/16/32 bit embedded processors • Basics of Microcontrollers – timers, interrupts, analog to digital conversion, bootloaders • Interaction with devices - buses, memory management, device drivers and wireless comm. • Interfacing sensors, actuators and peripherals to make complete platforms • Transforming control loops to code for robots • Real-time principles - multi-tasking, scheduling, synchronization • Building low-power high-performance systems – code profiling and optimization

University of Pennsylvania Course Syllabus

ESE 519 Real-Time and Embedded Systems Fall semester, 2017

1.2 Course schedule Date Topic Lab

Aug 29 Course & Platform Overview L0 and L1 out

Aug 31 Embedded Systems Power Management L0 due on Sep 1

Sep 5 Arduino from Scratch I Sep 7 Arduino from Scratch II Sep 12 MCU Basics: Timers, Input Capture, Output Capture Sep 14 MCU Basics: Interrupts and PWM Sep 19 MCU Basics: ADC L2 out

Sep 21 Transducers L1 due on Sep 22

Sep 26 Architecture and Assembly Sep 28 Stacks and MMIO Oct 3 Emulators L2 due, L3 out

Oct 5 No class -- Fall break Oct 10 Old School Gaming Oct 12 LCD / Touchscreen Oct 17 SPI and SCI Interfacing L3 due, L4 out

Oct 19 Control Systems Oct 24 Midterm 1 -- in class Oct 26 From Control Loops to Code Oct 31 Low-power wireless communication L5 out

Nov 2 RTS: Scheduling and Concurrency L4 due on Nov 3

Nov 7 RTS: Fixed and Dynamic Priority Scheduling Nov 9 Real-time Synchronization Nov 14 RTS: Communication Nov 16 Low-power embedded design L6 out

Nov 21 RTOS overview L5 due

Nov 23 No class -- Thanksgiving Nov 28 BalanceBot Competition Nov 30 Case Study I Dec 5 Case Study II Dec 7 Course wrap-up L6 due on Dec 11

Dec 12-13 Reading Days

Dec 14 Midterm 2

Dec 18 Late lab deadline

The updated course schedule will be posted on-line – http://tinyurl.com/ESE519F17-sched

1.3 Lecture Descriptions

Lecture 1 - Course and Platform Overview What this course is about, what you will learn, do and formation of teams Course overview Labs overview Arduino from Scratch Lecture 2 - Embedded System Power Management How do you power embedded systems efficiently? How Arduino does power management Basics of Linear Regulators Basics of Switched Mode Power Source Lab overview - Arduino from Scratch (Part 1/3) Lecture 3 - Arduino from Scratch I Why just use an Arduino, when you can build it from scratch? Laying out the board Importance of decoupling and bypass capacitors In-circuit serial programmer and USB interface Clocks, crystals and resonators Lecture 4 - Arduino from Scratch II How do you program a board for the first time and every time? Program compiling and linking Memory layouts Bootloaders and uploading your program Firmware over the air Lab overview - Arduino from Scratch (Part 2/3 and 3/3) Lecture 5 - MCU Basics: Timers, Input Capture, Output Compare You’ve learnt MCUs, here is a crash course Input capture for measuring signal period and duty cycle Output compare for generating signals and measuring frequencies Configuring timers with rescaling, timer overflows Lecture 6 - MCU Basics: Interrupts and PWM Let’s drive motors! and other actuators quickly and efficiently Using timer interrupts to make your program efficient Configuring PWM for mood lighting, clock generation, and driving dc and servo motors Lecture 7 - MCU Basics: ADC A crash course on ADCs and how you will use them in the next lab ADC concepts - sampling, quantization error, range, resolution, compression and dithering Flash and Successive approximation ADCs Configuring the ADC for sensors Lab overview - Sensor-Actuator Theremin Lecture 8 - Transducers We interface a lot of sensors in this course, let’s learn how to do it systematically Interfacing resistive, capacitive, inductive, electro-chemical, MEMS and digital sensors Sensor impedance matching, calibration, sampling and fusion Using an IMU and camera sensor

Lecture 9 - Architecture and Assembly Understand the architecture to program efficiently and safely Architecture types, memory types, endianness Atmel AVR architecture, registers, memory map Examples in AVR assembly Lecture 10 - MMIO and Stacks Understand memory maps to understand how subroutines, interrupts and OS work Interfacing peripheral devices with the processor What happens when you compile C code? Single stack vs multi-stack setups Lecture 11 - Emulators Learn to program a virtual processor and play games on it Overview of 8 bit emulator and its emulation cycle How do game emulators work? Display and graphics Lecture 12 - Old School Gaming: Graphics and Audio A fun lecture on sprites and resource optimization for very constrained platforms Walk through game code, graphics and audio Lecture 13 - LCD and Touch screens Interfacing LCDs and low-level drivers Algorithms for drawing on LCDs Capturing touch with the ADC and calibrating the touchscreen Lecture 14 - Serial Communication How many pins, milliseconds and interrupts does it take to communicate? Architecture and code overview of RS232, SPI and I2C protocols Lecture 15 - Control system basics This is to prepare you for the BalanceBot lab PID control, tuning, manual refinement and Matlab toolbox Lecture 16 - From control loops to code This is to prepare you for the BalanceBot lab PID control implementation without and with real-time support Lecture 17 - Wireless Communication How to make power-hungry protocols efficient for resource-constrained platforms? Link layer protocol problems: hidden terminal, exposed terminal, near-far Energy-efficient link layer protocols built on top of IEEE 802.15.4 Lecture 18 - RTS - Scheduling and Concurrency We keep it Real… Time :) Task scheduling and context swaps Synchronization mechanisms with semaphores, mutex, atomic instructions Concurrency problems - producer/consumer, bounded buffer, deadlocks Lecture 19 - RTS - Fixed and Dynamic Priority Scheduling Fixed priority scheduling algorithms

Lecture 20 - RTS - Synchronization problems Priority inversion, starvation, deadlocks Lecture 21 - RTS - Communication Controller Area Network and Real-Time Ethernet in embedded systems Performance metrics, implementation and case study Lecture 22 - RTS - Low-power operation How to do nothing, efficiently Dynamic Voltage and Frequency Scaling CPU power modes Scheduling sleep to do nothing efficiently Lecture 23 - RTOS overview Preparation of Make an RTOS from Scratch Lab Lecture 24 - Case Study I Lecture 25 - Case Study II Lecture 26 - Course wrap-up We had fun! Let’s have some more :)

1.4 Lab Descriptions This course is focused heavily on learning by doing and the labs are a majority of the hands-on effort. We have 6 very fun labs which will take you from microcontroller basics to building complete embedded platforms and real-time systems from scratch. We will progressively go from programming bare metal on the Atmel AVR328P microcontroller on the Arduino board, to using libraries on the ARM Cortex-M3 mbed microcontroller platform and then using an RTOS. Lab 1 – Building Arduino from Scratch Why just use a readymade embedded system platform when you can learn how to build one from scratch by yourself? We will start with the basics of power management circuits to provide 3V and 5V using linear regulators. Then you will make a power source selection circuit to power from either the wall jack or a USB source. We will also make switched regulator circuits for buck and boosting voltages. After this, you will put together the entire Arduino Uno circuit on a breadboard and learn about the different engineering decisions on by-pass capacitors, resonators, clock crystals, ferrite beads, etc. You will flash the bootloader on your platform and bring it to life. Lab 2 – Sensor Actuator Lab to make a Theremin In this lab you will cover several basic microcontroller functions by programming the use of GPIO, timers for input capture and output compare, interrupts, pulse width modulation to generate signals, and the analog to digital convertor to capture sensor signals. Putting all this together you will make a musical instrument called Theremin which produces sounds of varying pitch and amplitude. Lab 3 – Make a Game Emulator We love retro games and the best way to play them is to emulate the processors from the 1970s ☺ We will implement an old school virtual processor’s instructions, memory manager and display driver to create an emulator. The emulator will interpret code of several retro games such as Blinky, Blitz, Brix, Connect 4, Invaders, Maze, Merun, Missle, Pong, Tank, Tetris, TicTac, UFO, Vbrix and Wipeoff. Through this lab you will learn to make a totally portable embedded software system.

Lab 4 – Interactive Hand-held Pong Game System This lab introduces different hardware and software interfacing techniques to integrate peripherals such as touchscreens, LCDs and motion sensors to make a fun and interactive game system. You will use the ADC to write a touchscreen drive, write a high-level graphics driver for the LCD, design an interactive game, write a computer player, interface an IMU and learn to develop a working hand-held gaming platform. Lab 5 – Balance Bot This lab introduces control with embedded systems. You will develop a self-balancing bot using stepper motors, motor drivers, an IMU and interface it with a Zigbee wireless remote (also made by you!). Then you will implement controls for stability and position control. The lab is topped off with a competition for the fastest manual bot and the fastest camera-driven autonomous bot. Lab 6 – Building an RTOS Kernel from Scratch As you can see, we like building things from scratch ☺ How better to learn about task scheduling, resource management and synchronization issues in real-time systems than making your own real-time operating system from scratch. We will focus on developing the kernel for a RTOS and investigate different task scheduling policies to better understand scheduling theory and how to develop predictable systems for safety-critical applications.

2. Course Logistics 2.1 Course Communication Canvas: Lecture notes, Lab handout, Assignment submission and Grade reporting https://canvas.upenn.edu/courses/1373944 Piazza: Questions and Discussion with Instructor, TAs, peers and also anonymous posts https://piazza.com/upenn/fall2017/ese519/home Github: Code repositories for Labs – accessible to all. https://github.com/pennESE519/fall2017 We will use the above repository to share starter code for your labs. Besides this repo, every team will be assigned a private repository to keep your code. Computer failures (e.g. server failures, network failures, power outages, etc.) are a fact of life. Plan your project to be able to cope with this degree of uncertainty. Always save a working/stable version of the project before branching a new experimental version. Use github frequently and extensively. 2.2 Diverse Teams Labs are performed in groups of two, ideally with each group consisting of at least one Computer/Electrical Engineering major and one Systems Engineering or Computer Science major. Each member of a group must participate in the labs, be present during demonstrations, and submit a joint lab report. Teams with students from the same background (e.g. all EEs or all CS) is highly undesirable and requires the instructor’s approval. The more diverse your group, the better. 2.3 Labs Sessions Lab session will be from 4pm – 7pm on Fridays. Attendance is mandatory and all TAs will be present. Working in Detkin Lab at times other than Lab Sessions You can use Detkin lab any other time if there is no course going on or if there is a seat available. A portion of the Detkin Lab is also reserved from 1-4pm on Friday for teams to work before the lab

session. All students will be given 24-hour access to Detkin. Since the lab space is shared with one or more other courses, you can expect a significant amount of crowding, particularly near the end of the semester and when assignments/projects are due. A reasonable-person principle will be applied at all times. 10. Project Kits Each team will receive a project kit which will have almost all the components required for the labs. Teams will also be given lockers to store their kits and circuit boards. You are free to take the kits with you for the duration of the semester and must return the kit at the beginning of the next lab.

3. Grading Students will be graded as a team for lab assignments and individually for midterms. 60% Lab Assignments (Lab 1=5%, Lab 2=5%, Lab 3=10%, Lab 4=10%, Lab 5=15% Lab 6=15%) 20% Midterm 1 20% Midterm 2 While lower cutoffs may be used, the following maximum grade cutoffs are guaranteed: > 90 A > 80 B > 70 C > 60 D We don’t call Midterm 2 a Final as it is not cumulative. Labs have extra credit and the weight is specified in the lab handout. 3.1 Late Policy: There are no late days; you cannot submit any lab material past the deadline (11:59pm on the due date). However, the rule may be relaxed for only one of the assignments and you may submit any one of the assignments any time before December 18, the late lab deadline. The late lab will receive up to 85% of the original lab score.

4. Cheating Policy This section on cheating is the only reference to the topic we will have throughout the course. It is expected that students taking the course have the integrity not to cheat. However, to be fair to the vast majority of honest and hard-working students, the University and SEAS's policies on cheating will be rigorously enforced. While you may be able to find code on-line, this completely defeats the purpose of learning by doing – so don’t refer to or copy any code that is not written by your team.

Learn a lot and have fun doing it!