![Page 1: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/1.jpg)
![Page 2: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/2.jpg)
Take Data Validation Seriously
Paul Milham, WildWorks
![Page 3: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/3.jpg)
WildWorks
![Page 4: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/4.jpg)
Animal Jam
![Page 5: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/5.jpg)
Outline
• Attacks • Data Validation => Security • Data Normalization => Stability • Joi • Tean • Express Integration • Hapi Integration • Questions
![Page 6: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/6.jpg)
Safety
• My job is to keep kids safe. • How do we keep our application safe? • Safe from what?
![Page 7: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/7.jpg)
Attacks
• The web is full of jerks • https://www.owasp.org/index.php/Category:Attack • Read that for a bedtime horror story
![Page 8: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/8.jpg)
SQL Injection
console.log(name); // paulconsole.log(email); // '); DROP TABLE db.user; --mysql.query(`INSERT INTO db.user (name, email) VALUES ('${name}', '${email}')`);
![Page 9: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/9.jpg)
Shell Injection
console.log(pass); // "; rm -rf /"require("child_process").exec(` php -r "print crypt('${pass}','\\$1\\$rounds=1\\$salt\\$');"`, (err, stdout, stderr) => {});// hopefully you're using containers
![Page 10: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/10.jpg)
ReDOS
const msg = 'foo=bar' + ';'.repeat(65535) + 'domain=example.com';console.time("regex");console.log(msg.search(/;+$/));console.timeEnd("regex"); // regex: 5854.071ms :(
• This is a sample vulnerability in tough cookie • https://snyk.io/vuln/npm:tough-cookie:20160722 • Be careful of "evil" regex
![Page 11: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/11.jpg)
Security
• It’s a scary world • Security is important • There’s a lot more than just those three
![Page 12: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/12.jpg)
Validation
• Verify the shape of the data • Malicious data can’t get in • First line of defense
![Page 13: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/13.jpg)
Simple Joi
"use strict";
const Joi = require("joi");
Joi.validate("srsly a string", Joi.string(), (err, value) => { console.log(err); // null console.log(value); // "srsly a string"});
![Page 14: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/14.jpg)
Joi Failure
Joi.validate(5, Joi.string(), (err, value) => { console.log(err); // Error console.log(value); // 5});
![Page 15: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/15.jpg)
Joi Schema
const schema = Joi.object().keys({ username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(), password: Joi.string().min(6).max(25).required(),});
Joi.validate({ username: "[email protected]", password: "justinbieber",}, schema, (err, value) => { console.log(err); console.log(value);});
![Page 16: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/16.jpg)
All In
const schema = Joi.object().keys({ username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(),});
Joi.validate({ username: "[email protected]", password: "justinbieber",}, schema, (err, value) => { console.log(err); // justinbieber is not allowed});
![Page 17: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/17.jpg)
All In
• Validating one field means validating them all • Hard for devs to forget
![Page 18: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/18.jpg)
Data Normalization
• Normalization is being a good citizen • Normalization creates a contract with your consumer • Normalization goes a lot deeper than this (we'll get to that later)
![Page 19: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/19.jpg)
Joi Conversion
Joi.validate("1.916", Joi.number(), (err, value) => { console.log(value.toFixed(1)); // 1.9 (No TypeError!)});
![Page 20: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/20.jpg)
Joi Defaults
Joi.validate(undefined, Joi.number().default(0), (err, value) => { console.log(value.toFixed(1)); // 0.0 (No TypeError!)});
![Page 21: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/21.jpg)
Tean
•Declarative syntax (schemas are POJOs) •Async •Convert data into models •Strict by default •https://www.npmjs.com/package/tean •Note that custom validators were recently added to Joi
![Page 22: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/22.jpg)
Tean Validation
// simple validation tean.object({animal: "string"}, {animal: “kangaroo”},(isValid, result) => { console.log(isValid); // true console.log(result); // {animal: "kangaroo"} });
![Page 23: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/23.jpg)
Tean Failure
tean.object({animal: "string"}, {animal: null}, (isValid, result) => { console.log(isValid); // false console.log(result); // ["animal (null) is not a string"] });
![Page 24: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/24.jpg)
Tean Normalization
// optional parameters tean.object({animal: “string(kangaroo,tiger)=tiger”, sparkles: "bool=true"}, {animal: "tiger"}, (isValid, result) => { console.log(isValid); // true console.log(result); // {animal: "tiger", sparkles: true} // Note that the original object is not altered! Normalized and validated data is passed into "result" in the callback });
![Page 25: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/25.jpg)
Model Mapping
tean.object(req.body.params, { accessory: "avatarAccessory", user: ["userUid"],}, (isValid, result) => {});
![Page 26: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/26.jpg)
Data Normalization
• Provides a friendly API • Provides consistency and reliability • Eliminates lots of common bugs
![Page 27: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/27.jpg)
Express
• Everyone uses it! • No built in validation! • Too many exclamation points! • https://expressjs.com/
![Page 28: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/28.jpg)
Express + Joi
app.get('/:username', function (req, res) { const schema = Joi.object().keys({ username: Joi.string().required(), });
Joi.validate(req.params, schema, (err, value) => { console.log(err); req.params = value; res.send(`${req.params.username} is the best!`); });});
![Page 29: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/29.jpg)
Express + Tean
app.get('/:user', function (req, res) { tean.object(req.body.params, { user: "userUid", }, (isValid, result) => { console.log(isValid); req.params = result;
res.send(`${result.user.name} is the best!`); });});
![Page 30: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/30.jpg)
Problem
• We’re relying on the developer to remember to validate • This is a problem for maintenance and updates • Middleware to the rescue!
![Page 31: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/31.jpg)
Route Middleware
this.app.post(options.route, tean.expressRequest(options.paramMap), (req, res) => { // do stuff options.handler(req.safeData, req, res); }, (err, req, res) => { console.log(err.stack); res.status(500).send(); } );
![Page 33: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/33.jpg)
Hapi
• Hapi isn't minimalist like Express • Lots of options out of the box • http://hapijs.com/
![Page 34: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/34.jpg)
Hapi Validation
app.route({ method: "POST", path: "/", config: { handler: (req, reply) => { reply("hey!"); }, validate: { payload: { username: Joi.string().email().required(), password: Joi.string().max(25).required(), }, }, },});
![Page 35: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/35.jpg)
Take Away
• FORCE validation of data - an opt in system isn't good enough • Make sure shape of data is acceptable • No validation, no data • This ensures malicious data does not enter your application
![Page 36: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/36.jpg)
Take Away
• FORCE normalization of data shape • Data should always have a consistent shape • Make data access and usage reliable • Eliminates lots of “stupid” bugs
![Page 37: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/37.jpg)
On the Way Out
• Have you thought about data security on the way out? • Mind blown! • Prevent Data Leaks from "heartbleed" or SQL Injection • Provide same stability contract for your client app
![Page 38: Take Data Validation Seriously - Paul Milham, WildWorks](https://reader031.vdocuments.us/reader031/viewer/2022021922/58757e501a28ab78498b6ce9/html5/thumbnails/38.jpg)
Thanks!
• Any questions? • @domrein