crystalball architecture jesse walsh. outline original plan – industry standard for building web...

CrystalBall Architecture

Jesse Walsh

Outline

• Original Plan– Industry standard for building web applications is the

3-tier approach– Security issues– Concurrency

• The New Plan– Unit 1 – Statistical analysis of data– Unit 2 – Database– Unit 3 – Prediction server– Unit 4 – Full implementation of the original plan

The Original Plan

• 2 Goals– Web interface

• User Data Management System• Allows users to visualize their data by plates/wells etc

– Crystallization Prediction engine• #1 My screens failed, what should I try next?• #2 I have a protein, given similar proteins, what conditions should

crystallize this protein?

– Database• Stores user data and publically available crystallization data• Flexible enough to handle any situation, granular enough to have all

data we could possibly use for prediction available in a mineable format

How do you design a web application?

• 3 Tier Architecture– Advantages

• Proven industrial-strength architecture• Capable of handling most issues relevant to businesses, which

means it is more than capable for us• Flexible enough to meet most demands and still independent

enough be able to handle major changes to structure• Scalable

– Disadvantages• Requires some organization and planning at the onset• Not the easiest way to do things, only pays off when

unforeseen difficulties appear later

Classic 3 Tier Architecture.net Application Architecture

http://www.simple-talk.com/dotnet/.net-framework/.net-application-architecture-the-data-access-layer/

Web Forms (UI)


• html markup• Built with

Dreamweaver• Contains design

aspects– Color scheme– Location of

dropdowns, links, etc

– Static text

Web Forms (Code Behind)


• “Interface Logic”• Code behind• Written by a developer

(php)• Mechanism for all the

bells and whistles of the web form– Fills in dropdowns and

lists– Alters pages based on

selections/preferences– Talks to database to get

data and save data– Login verification

Business Objects


• Code written in a way that can be accessed by the web form code

• Data in a useable format

• Includes all logical checks on data– Has permission to

view this?– Which data is

needed

Data Access Layer (DAL)


• Closely tied to Business Objects

• Directly interacts with database using SQL commands

• Must have functions that are relevant to Business Object– Get plate X– Get well Y– Get all solutions with

component Z– Store this new member

• Abstracts the interaction with Data Tier and Presentation Tier

Database


• MySQL Server• Persistent

storage of data


• 3 Tier Architecture applied to CrystalBall

Web Design

Interface Logic DAL Database

Prediction

Business Tier

Presentation TierData Tier


• Web Design– Formats and tools that we would like to include– Color schemes, layouts, etc.

Web Design


Prediction

Business Tier



• Interface Logic– Bells & whistles of web form– Will include login verification

Web Design


Prediction

Business Tier



• Prediction– All prediction analysis will be conducted here– Output of prediction must be in a from that can be transferred

to web form– Input into prediction comes from database, which should have

a subclass in the DAL for this data retrieval

Web Design


Prediction

Business Tier



• DAL– Needs to know database design well– Needs to be able to design queries based on needs of

prediction code and web code– *Concurrency issues are handled here*

Web Design


Prediction

Business Tier



• Database– Mostly stable schema– Server up and running

Web Design


Prediction

Business Tier


CrystalBall Security Issues

• Security must take place in several locations– Database server must be secure (physically and virtually)– Data itself must contain information to allow ownership– Web forms must respect security measures

Web Design


Prediction

**Disclaimer**This stuff would probably be better explained byour security experts. Feel free to correct me if Imisstate something. This is my personalunderstanding of our security issues.

CrystalBall Security Issues• First, lets break down the CrystalBall architecture into server and client

side components• Server side components are local (here on campus) running on

machines we own . These are more secure, since they don’t communicate over internet (may use intranet, but most likely will be on the same machine). We have more control over the security of these machines and what stuff runs on them.

Web Design


Prediction

Server Side

CrystalBall Security Issues• Client side components run on the user’s machine. Any data sent or

processed on the client side is insecure. Even if they are allowed to see the data, it is only as secure as their machine

• Notice that interface logic has both server components and client components

• Client side components come in the form of AJAX and javascript

Web Design


Prediction

Client Side


• Web Security– How to pass information between pages securely?– How to verify user and maintain authentication?– How to prevent secure information from being incorrectly accessed?– Javascript/AJAX in use?… what data will this make available to the

client?

Web Design


Prediction

Server SideClient Side


• DAL– Must be given information about which data can be

accessed, and then must respect this by only accessing data that is allowed to be seen

– i.e. given a user id, retrieve only data owned by this user

Web Design


Prediction



• Database– Computer must be protected from viruses, etc– Server must not allow unauthorized access– Database must have information about ownership and

public/private status

Web Design


Prediction


Other Issues - Concurrency

• Users who access the same data at the same time my “crash” into each other when submitting changes– If two people try to write to the same data at the same

time, they will overwrite each other’s changes. Neither user will be informed of the mistake, and the information that should have been added could be lost.

– The last to submit “wins” and overwrites the other

Concurrency Problems?• Do nothing—If concurrent users are modifying the same record, let

the last commit win (the default behavior).• Optimistic concurrency—Assume that while there might be

concurrency conflicts every now and then, the vast majority of the time such conflicts won't arise; therefore, if a conflict does arise, just inform the user that their changes can't be saved, because another user has modified the same data.

• Pessimistic concurrency—Assume that concurrency conflicts are commonplace and that users won't tolerate being told their changes weren't saved because of another user's concurrent activity; therefore, when one user starts updating a record, lock it, thereby preventing any other users from editing or deleting that record until the user commits their modifications.

http://msdn.microsoft.com/en-us/library/bb404102.aspx

http://en.wikipedia.org/wiki/Optimistic_concurrency_control

Concurrency Problems?

• Optimistic Concurrency might be the technically correct solution to our problem– “Optimistic-concurrency control works by ensuring

that the record being updated or deleted has the same values as it did when the updating or deleting process started.”

• Implementing Concurrency can be very tricky to get right and time consuming to learn/implement

A New Direction

• For various reasons decided on during our August 5th 2009 meeting, we decided to break down the original plans into modularized, publishable components

• How will this affect our design plan?

The New Plan• Unit 1 – Data and Statistical Prediction

– Get the Eddie Snell data– Import data into R in a useable format– Try multiple analysis methods and refine predictions

• Unit 2 – The Database– Port data into our schema (Eddie Snell and BMCD)

• Unit 3 – Prediction Server– Create a web applet to accept user data and return predictions based on

Unit 1• Unit 4 – CrystalBall Interface

– Begin to store data from Prediction Server– Design and Implement user interface– Recruit users

New CrystalBall Architecture• Remember the architecture discussed previously?• The new changes will essentially remove the presentation tier and replace it

with a prediction server web applet• The DAL may still be implemented, but is no longer necessary (might depend

on what happens with BMCD data)• The 3-Tier approach becomes more of a technicality. We can still place our

units into these tiers, but doing so changes nothing

Web Design


Prediction

Business Tier


Unit 1 – Data and Statistical Prediction• The predictions currently only need the Eddie Snell Data• This data can be stored in a flat file (excel, csv, etc), thus saving us the trouble of

porting it to our database• Connecting this data to the R prediction tool will be a couple lines of code (hopefully)• The database and DAL are not needed at this point• The flat file is “temporary” in the sense that our ultimate goal will be to import it into

the database and use it from there via the DAL• Prediction Engine built in this unit should answer the question

– My screens failed to produce crystals, what conditions should I try next?

Database

Prediction

Business Tier Data Tier

Eddie Snell Data

Prediction Server

Presentation Tier

Unit 2 – Database

• This part only requires the schema, which is for the most part complete

• Importing Eddie Snell data and BMCD data will help support our bid for a publication

• Basically no additional work is required here

Database

Prediction


Eddie Snell Data

Prediction Server

Presentation Tier

Unit 3 – Prediction Server• The prediction server will amount to a web page the accepts some user data,

feeds it into the prediction engine developed in unit 1, and outputs results.• No persistent storage of user data• Involves web design, some php(?) coding, and figuring out how to get the

prediction server to talk to the prediction engine• Security issue: User data must be encrypted or protected in some such way

on it’s way to and from the prediction server

Database

Prediction


Eddie Snell Data

Prediction Server

Presentation Tier

Current Goal

• If we can achieve completion of units 1, 2, and 3, we should have a prediction server, a prediction engine, data, and a mostly empty database schema

• At this point, we can evaluate the possibility of continuing on with unit 4

Database

Prediction


Eddie Snell Data

Prediction Server

Presentation Tier

Unit 4 – CrystalBall Interface• If we continue with unit 4, we will mostly return to our original

design• Eddie Snell data should be included in the database by now• Prediction server will provide a service not fully realized by the

user interface originally planned, as it can be used more spontaneously without having to input large amounts of user data

Web Design

Interface Logic DAL

Prediction

Business TierPresentation Tier

Database

Data Tier

Eddie Snell Data

Prediction Server

Unit 4 – CrystalBall Interface

• Unit 4 is where many of our problems and headaches are introduced– Concurrency becomes an issue here– Many of our security issues only become relevant

when we store private data in unit 4– Need to find users

• Something to think about when the time comes!

Thanks!

crystalball architecture jesse walsh. outline original plan – industry standard for building web...

Documents