design and implementaion of a web-based rc3e user frontend

Fredo Erxleben

Design and Implementaion of aWeb-Based RC3E User FrontendStatus Report // Dresden, April 12, 2018

Contents of this Talk

IntroductionRC3E

Django

REST

Data Model

Selected IssuesUser Interaction ConceptSupporting Multiple APIsReservation Scheduling

Results and Future Work

Design and Implementaion of a Web-Based RC3E User FrontendChair for VLSI Design, Diagnostics and Architecture // Fredo ErxlebenDresden, April 12, 2018

Slide 2 of 30

Section 1

Introduction


Slide 3 of 30

The Core Issue

RC3E ( Reconfigurable Common Cloud Computing Environment )

• System for distribution of FPGA ressources to remote users

• Actual ressources get abstracted as virtual FPGAs

• Controlled by a human administrator, who. . .

– . . . runs selected scripts on the host– . . . manages users by hand– . . . has to schedule ressources– . . . does the programming of vFPGAs and relays the results

Project Aim

Create a web-frontend that can automatize these tasks.


Slide 4 of 30

What is Django?

• Python-based framework for web applications

• Architecture is Model Template View

– Variation of MVC pattern

• Focus on reuseability, rapid prototyping and the DRY-principle1

• Huge amount of out-of-the-box features

1DRY = Don’t repeat yourself


Slide 5 of 30

Architecture of a Django Project

Project

manage.py

settings.py

urls.py

App1

forms

models

templates

urls

views

App2

. . .


Slide 6 of 30

Operation Principle of a Django-App

Browser

URL Dispatcher

HTML TemplateRendering Engine

View ModelDatabaseConnector

MySQL

DatabaseBackend

Django Controller

Web Server


Slide 7 of 30

What is REST?

REpresentational State Transfer

• Architecture principle for distributed web applications

• Allow clients to access a textual representation of a ressource on thehost

• Set of available operations on the ressources are well-defined by asimple interface

Benefit

Easy to parse for machine-to-macine communication.Useful for interaction with the RC3E-server.


Slide 8 of 30

Example for a RESTful API Call

Request:

fredo in ~ > http GET localhost :8000/ rest_api/producers/

Response:

HTTP /1.1 200 OK

Content -Type: application/json

[More http header stuff]

[

{

"id": 1,

"name": "Xilinx"

},

{

"id": 2,

"name": "Altera"

},

{

"id": 3,

"name": "Intel"

}

]


Slide 9 of 30

Section 2

Data Model


Slide 10 of 30

Preface

The data model is the backbone of the project.

• Must be represented by a relational DB

• Fine granularity to improve changeability

Important!

FPGAs are assumed to be homogenous.

Note

Multiple development iterations introduced some technical debt.But: It can be purged by refactoring cycles.


Slide 11 of 30

Data Model Details I

fpganode : nodefpga model : fpga modelnode pci : pci addressdevice pci : pci address

pci addresssystem : character[4]bus : character[2]device : character[2]function : character[1]

nodename : character[255]ip address : generic ip addresscomment : text field

fpga modelproducer : producerdesignation : character[255]region type : region typeregion count : integer

regionin fpga : fpgamemory device path : file pathregion type : region typeindex : integer

region typelut count : integerdsp count : integerregister count : integerblockram size : integer

producername : character[255]


Slide 12 of 30

Data Model Details II

region reservationregion : regionreserved by : reservation

regionin fpga : fpgamemory device path : file pathregion type : region typeindex : integer

reservationby user : django.auth usercreation date : datetimestart date : datetimeend date : datetime

Formerly known as vfpga


Slide 13 of 30

Data Model Details III

fpganode : nodefpga model : fpga modelnode pci : pci addressdevice pci : pci address

fpga modelproducer : producerdesignation : character[255]region type : region typeregion count : integer

device variablefpga : fpganame : character[255]value : text field

scriptprogrammer : programmerfpga model : fpga modeltemplate : text field

runtime variablename : character[255]value : text field

programmername : character[255]device file path : file path


Slide 14 of 30

Section 3

Selected Issues


Slide 15 of 30

Basic Principle of User Interaction I

Boil down all interaction to the following operations:

• List objects of a type

• Show object details

• Create objects

• Delete objects

Note

Allowing to modify existing objects has severe implications!(What happens when you change the region type/ count of an FPGA?)


Slide 16 of 30

Basic Principle of User Interaction II

Issue

URLs and internal project structure have to consistently represent thisprinciple.

Issue

Look and feel of the HTML pages should be uniform.

Solution

Use Djangos HTML template inheritance.

Note

Even special pages can profit from inheriting layout templates.


Slide 17 of 30

HTML Template Structure

layout general.html

layout create.html

fpgas/create.html

. . .

layout delete.html

fpgas/delete.html

. . .

layout list.html

layout show.html

welcome.html

registration/login.html

registration/logged out.html

css/layout.css

WIP


Slide 18 of 30

Automating URL-Binding Generation

Issue

Generating all the URL-bindings for the List/Show/Create/Delete pagesby hand is tedious and error-prone.

Solution

Write generator functions for the common case and only deal with thespecial cases by hand.

Note

Generator functions also check if the required templates are present.

(PS: https://www.xkcd.com/917/)


Slide 19 of 30

https://www.xkcd.com/917/

Supporting Multiple APIs I

Issue

Need to support a REST and a web API

• Both use the same protocol (HTTP/ HTTPS)

• HTTP request types GET, POST used by web API

• REST will use these as well

How to figure out what to do with an HTTP request?


Slide 20 of 30

Supporting Multiple APIs II

Idea

Give each API a different URL namespace.

Web-API:GET /web_api/producers/create/

POST /web_api/producers/create/

REST-API:GET /rest_api/producers/

POST /rest_api/producers/

GET /rest_api/producers /1/

DELETE /rest_api/producers /1/


Slide 21 of 30

Supporting Multiple APIs III

Issue

The REST-framework alters the Django-apps structure.

Solution

Split the project into one app per API.

Next Issue

Both apps have to operate on the exact same model.

Solution

Split off the model into a separate app as well.


Slide 22 of 30

Supporting Multiple APIs IVrc3e manager

. . .

backend

models

rest api

serializers

views

web api

forms

templates

views

import

Same name,Different stuff

!


Slide 23 of 30

Scheduling of Reservations I

User Input

• Region type

• Region count (consecutive)

• Desired start and end of reservation


Slide 24 of 30

Scheduling of Reservations II

View generates and issues SQL query from user input.

Database Action

1. Filter for FPGAs with correct region type

2. Filter regions without reservation in desired timeframe

3. Sort remaining regions by index

Result type: {FPGA: [Region]}


Slide 25 of 30

Scheduling of Reservations III

View then processes the query result.

Pseudocode

foreach fpga:

| foreach region:

| | if region is not consecutive:

| | | reset markers // deal with fragmentation

| |

| | mark region

| |

| | if enough regions marked:

| | | break all // Got enough reservation candidates

|

| reset markers // Reservations across FPGAs are not allowed

reserve candidates // If there are any

// Inform User


Slide 26 of 30

Scheduling of Reservations IV

Notes

• First fit scheduling does not use ressources equally

• Fragmentation may become an issue in real-life use

• Reservations are not supposed to be deleted (accounting) which mayslow down the DB after many reservation cycles

It works for the scale of RC3E but has optimization potential.


Slide 27 of 30

Section 4

Results and Future Work


Slide 28 of 30

Summary of Current State

• Data model has been developed

• Technology research is completed

• Model is implemented

• Web-API is implemented

• Currently known issues have been solved


Slide 29 of 30

Future Work

Immediate tasks:

• Finish transition to new HTML template structure

• Finish implementation of REST-API

Whishlist:

• Subsystem for uploading designs, running the FPGA programmingscripts and operation feedback

• Subsystem for statistics and accounting

• Logging RC3E interactions to DB

Ultimative Goal

Connect the frontend with the live RC3E-system.


Slide 30 of 30

Section 5

Appendix


Slide 31 of 30

Further Reading I

RC3E

RC3E: Reconfigurable Accelerators in Data Centres and Their Provisionby Adapted Service Models, Oliver Knodel, Patrick Lehmann, Rainer G.Spallek, 2016https://ieeexplore.ieee.org/abstract/document/7820030/

Rekonfigurierbare Hardwarekomponenten im Kontext vonCloud-Architekturen, Dissertation of Oliver Knodel, 2018Publication pending


Slide 32 of 30

https://ieeexplore.ieee.org/abstract/document/7820030/

Further Reading II

REST

Architectural Styles and the Design of Network-based SoftwareArchitectures, Dissertation of Roy Fielding, 2000https://www.ics.uci.edu/~fielding/pubs/dissertation/

fielding_dissertation.pdf


Slide 33 of 30

https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf

https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf

Further Reading III

Django

Official Documentationhttps://docs.djangoproject.com/en/2.0/

Django-RESThttp://www.django-rest-framework.org/


Slide 34 of 30

https://docs.djangoproject.com/en/2.0/

http://www.django-rest-framework.org/

Last Slide

Thank you for your attention.Questions?

Office: TUD, APB 1095Phone: TUD-38456E-Mail: [email protected] Git: http://github.com/VLSI-EDA/rc3e-manager

PS: We are looking for students to continue working on this project.

Bachelor thesis, anyone? ;)


Slide 35 of 30

mailto:[email protected]

http://github.com/VLSI-EDA/rc3e-manager

design and implementaion of a web-based rc3e user frontend

Documents