phil cracknell - metrics – the art of comparing apples with mango
TRANSCRIPT
INFORMATION SECURITY METRICS
COMPARING APPLES WITH MANGO
PHIL CRACKNELLWHAT I AM
• FELLOW OF THE BRITISH COMPUTER SOCIETY• CISSP• 25 YEARS IN INFORMATION SECURITY• FORMER CISO (5 TIMES)• HEADED CONSULTANCY PRACTICES• FOUNDER OF CLUBCISO• TECHNOLOGY AGNOSTIC
• AND OF COURSE…• CYBER SECURITY PERSONALITY OF THE YEAR
2015
WHAT I’M NOT
• SELLING ANYTHING• GAINING OR PROFITING FROM THE METRICS
PROJECT• ANTI-VENDOR
AND SO TO THE DEALERSHIP…QUITE LONGHARD TO PARK IN SMALL GAPSWILL FIT UNDER CAR PARK BARRIERS
THERE’S 4 OF ‘EM
NO CD FITTEDSEE ABOVE
DEPENDS ON LUGGAGETRY TO KEEP IT OFF THE KERB
WHEEL ON EACH CORNERWILL GET YOU TO WAITROSE AND BACKONE MEDIUM LOUIS VITON CARRY-ON AND 3 GOLF CLUBS
WHY THEM AND NOT US?• OUR PROFESSION (INFORMATION SECURITY) HAS WAITED PATIENTLY FOR A
BALANCED, INDEPENDENT AND COMMON COLLECTION OF METRICS, KPIS, MEASUREMENTS OR RISK INDICATORS – CALL THEM WHAT YOU WILL, BUT SOMETHING THAT CFOS HAVE HAD THE EQUIVALENT OF FOREVER.
• EBITDA, PE VALUE, DAYS TO CLOSE, FINANCE HEADCOUNT RATIO – CFOS CAN QUOTE THEM MERRILY TO EACH OTHER AS IF THEY WERE TOP TRUMPS, THEY CAN EVEN QUOTE THEM TO COOS AND CEOS AND NOT GET A PUZZLED LOOK.
• WHY HAS THIS UNIVERSAL UNIT OF MEASUREMENT EVADED INFORMATION SECURITY PROFESSIONALS FOR SO LONG?
• WHY DO WE PERSIST IN DIGGING A HOLE, MOVING ASIDE THE EARTH, TAKING OFF OUR SHOES AND JUMPING RIGHT IN IT EVERY TIME WE QUOTE THE FIGURES THAT WE DO HAVE TO A C-LEVEL IN OUR BUSINESS?
BUT WHY?
• I’LL TELL YOU WHY, IT’S BECAUSE THOSE NASTY VENDOR TYPES GOT TOGETHER, REALISED THAT WE WERE NEVER GOING TO HAVE ANYTHING PLAUSIBLE OR TANGIBLE TO SCARE OUR BOARD AND SO THEY KINDLY FILLED THE VOID, PRODUCING VALUE AFTER VALUE
• 47.6% COMPLIANCE WITH POLICY – EVEN WHEN IT’S ONLY PARTLY WRITTEN AND NOT YET SOCIALISED…
• 32,000 ENDPOINTS INFECTED WHEN WE ONLY HAVE 2,500 STAFF AND SO ON.• SERIOUSLY, WE CAN’T BLAME THE VENDORS THOUGH, THE VALUES PRODUCED BY THEM,
VALUES THAT WE HAVE RELIED UPON AND TRIED TO EXPLAIN IN TERMS THAT A BOARD CAN UNDERSTAND, GENERALLY REFLECT THE PERFORMANCE OF THEIR PRODUCT – AND WHY NOT?
PROJECT “METRICS”• BUSINESSES ARE WAKING UP TO THE FACT THAT THEY NEED METRICS/RISK
INDICATORS THAT OUR BOARD, AUDIT COMMITTEES AND NON-EXEC DIRECTORS UNDERSTAND, FOR THEY ARE THE KEY TO BUDGET, EXTRA STAFF, A CORNER OFFICE AND A JOB FOR LIFE
• OK, MAYBE TWO OF THOSE ARE NOT TRUE, BUT THEY WILL MAKE LIFE EASIER.• IT’S NOT UNCOMMON TO GET THAT MONDAY MORNING SWOOP-BY WHEN THE CEO
HAS READ SOMETHING IN THE SUNDAY TIMES AND WANTS TO KNOW “WHERE WE ARE WITH THAT ONE?”
• “AND WHAT ARE OTHERS DOING?” • METRICS, AS CLUBCISO ORIGINALLY DECIDED TO CALL THEM, ARE THE KEY TO OUR
FUTURE. THEY ARE BEING DEFINED BY CISOS ALONE, THEY DETAIL EXACTLY HOW WE DEMONSTRATE OUR EFFECTIVENESS, PINPOINT OUR RESPONSIBILITIES AND HIGHLIGHT INVESTMENT OR LACK OF IT, AND WHAT ENSUES…THEY WILL CHANGE THE WORLD.
HOW IT ALL BEGAN…• SO WE GOT 25 CISOS TOGETHER IN A WORKING PARTY, A COMBINED 350 YEARS OF INFORMATION SECURITY
EXPERIENCE, AND WE GRABBED A SUPPLY OF POST-IT NOTES, PENS AND ASKED THE CISOS TO WRITE DOWN WHAT THEY CONSIDERED TO BE THEIR TOP FIVE METRICS. HAVING STUCK THE NOTES ON THE WALL WE THEN PROCEEDED TO GROUP THE NOTES INTO COLLECTIONS OF SIMILAR VALUES. THE RESULTS SHOWED FIVE GENERAL ‘HEADINGS’ OR FAMILIES INTO WHICH THE MAJORITY OF POST-IT NOTES FELL. THIS WAS OUR STARTING POINT.
• IT’S NOT JUST ABOUT CREATING A FRAMEWORK FOR METRICS AND THEN INDIVIDUALLY PRODUCING THEM, WE HAD TO CUNNINGLY ESTABLISH A SECOND WORKSTREAM - A COMMUNICATIONS GROUP, TO LOBBY, EDUCATE AND INFORM AUDIT COMMITTEES, DATA PRIVACY OFFICERS, NON-EXEC DIRECTORS AND INFLUENCERS ON WHAT EXACTLY THE METRICS COULD DO FOR THEM.
• THEY MAY NOT FULLY APPRECIATE A TOP LEVEL METRIC AT THE MOMENT, BUT THEY ARE MORE THAN FAMILIAR WITH BOARD RISK INDICATORS, AND OUR TOP LEVEL METRICS WILL ULTIMATELY FEED INTO THESE ALREADY UNDERSTOOD VALUES AND ADD SOME FURTHER PERSPECTIVE.
• GOING FORWARD, WE WANT TO BE ABLE TO DEMONSTRATE ‘WHAT IF’ AND INVESTMENT MODELLING SCENARIOS TO SHOW TRENDS IF WE INVEST MORE, LESS OR DIFFERENTLY.
METRICS – TOP LEVELExposure
Agility
Culture
Incidents
3rd Party Management
Access & Controls
IF CARLSBERG MADE SECURITY METRICS…
CISO BUMPS INTO CFO AT COFFEE MACHINECISO - “WE HAVE DETECTED 55,000 VIRUSES THIS MONTH!”CFO - “WOW”CISO - “AND THERE WERE 60,000 THE MONTH BEFORE!”CFO - “IS THAT BETTER OR WORSE?”- SILENCE -
CFO – “ARE WE DETECTING LESS BECAUSE WE ARE LOSING LAPTOPS OFF OUR NETWORK OR ARE WE BEING TARGETED LESS?”CISO - “ERRRR”- TUMBLEWEED MOMENT –
CISO – QUICK THINKING AND CHANGING THE SUBJECT - “AND WE’RE RIDDLED WITH MALWARE YOU KNOW…?”CFO - “HOW MUCH DOES THAT COST US?”- SIGH -
