test for success: automated testing of sas® metadata ...€¦ · easier to read/write than sas,...
TRANSCRIPT
![Page 1: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/1.jpg)
Test for Success: Automated Testing of SAS® Metadata Security ImplementationsPaul HomesMetacoda
![Page 2: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/2.jpg)
About Metacoda
• SAS Alliance Silver Member since 2007
• Provide add-ons to SAS® Software for enhanced metadata visibility and exploitation
• Metacoda Plug-ins (SAS Management Console)
• Custom Tasks (SAS Enterprise Guide & AMO)
• Goals:
• Improve your productivity through enhanced metadata visibility
![Page 3: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/3.jpg)
What is Metadata Security Testing?
… & what can we test?
![Page 4: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/4.jpg)
What is Metadata Security Testing?
Verifying SAS metadata has been secured according to business and I.T. policy requirements
Production
(Lev1)
![Page 5: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/5.jpg)
What can we test?: Users Verify expected users exist:
SAS Administrator,
SAS Trusted User, …
… with expected:
group/role memberships (direct/indirect)
capabilities (indirect)
logins (own/shared)
![Page 6: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/6.jpg)
What can we test?: Groups Verify expected groups exist:
SAS Administrators,
SAS System Services,
SAS General Services, …
… with expected:
group/user members (direct/indirect)
group/role memberships (direct/indirect)
capabilities (indirect)
logins (shared)
![Page 7: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/7.jpg)
What can we test?: Roles Verify expected roles exist:
Metadata Server: Unrestricted,
Enterprise Guide: Advanced,
Visual Analytics: Report Viewing, …
… with expected:
group/user members (direct/indirect)
capabilities (direct/indirect/contributed)
![Page 8: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/8.jpg)
What can we test?: ACTs Verify Access Controls Templates
(ACTs):
Have expected permission patterns:
» Groups / Users
» Permissions
Applied to expected objects
Protected with ACTs and explicit permissions (ACEs)
![Page 9: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/9.jpg)
What can we test?: Applied Access Controls Examine Authorization tabs:
Folders, Servers, ACTs, …
Verify access controls have been applied as expected …
Access Control Templates (ACTs)
Explicit Permissions (ACEs)
![Page 10: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/10.jpg)
What can we test?: Effective Permissions Verify Effective Permissions …
for candidate users / groups
on candidate objects
The “end result” … sensitive to:
Users identity hierarchy (groups)
Objects inheritance path
ACTs & explicit permissions applied to objects in the path
Repository ACT
![Page 11: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/11.jpg)
What can we test?: Golden Rules/Best Practices Best Practice Implementation of SAS® Metadata Security at Customer Sites
in Denmark, Cecily Hoffritz & Johannes Jørgensenhttp://support.sas.com/resources/papers/proceedings11/376-2011.pdf
Very limited use of ACEs [GR#1]
Only groups in ACTs and ACEs (not users) [GR#2]
Only implicit group permission denials (PUBLIC/SASUSERS) [GR#3]
All ACTs are protected
No Group/Role membership/contribution loops
No groups with implicit groups as members (PUBLIC/SASUSERS)
![Page 12: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/12.jpg)
Metadata Security Testing: Why?
… & why re-test regularly?
![Page 13: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/13.jpg)
Metadata Security Testing: Why?A Newly Secured and Tested SAS Platform …
Production
(Lev1)
![Page 14: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/14.jpg)
Metadata Security Testing: Why?Some time later after changes from various user roles …
Production
(Lev1)
… is it still adequately secured?
tomorrow?
next week?
next month?
![Page 15: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/15.jpg)
Metadata Security Testing: Why?How can insecure resources impact you & your organization?
Production
(Lev1)
Reputation ?
Failed regulatory requirements ?
Lost customers ?
$$$ ?
![Page 16: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/16.jpg)
Metadata Security Testing: Why?
Test for consistency across multiple environments …
Production
(Lev1)
Test
(Lev2)Development
(Lev3)
Test
![Page 17: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/17.jpg)
SAS 9.3
(Lev1)
SAS 9.2
(Lev1)
Metadata Security Testing: Why?
Test for consistency during SAS version upgrades …
SAS 9.2
(Lev1)
SAS 9.3
(Lev1)SAS 9.4
(Lev1)
Test Test
![Page 18: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/18.jpg)
Metadata Security Testing Considerations
![Page 19: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/19.jpg)
Metadata Security Testing: Method
How do you perform your testing?
Manually via point & click?
Automatically via code?
How consistent are your manual tests?
Ad-hoc?
Well defined test scripts?
![Page 20: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/20.jpg)
Metadata Security Testing: Coverage
How extensive is your testing?
Handful of sensitive / troublesome objects?
Hundreds / thousands of objects?
What types of things do you test?
Folders, Reports, Stored Procs, Info Maps?
Servers, Logins, Libraries, Tables?
Users, Groups, Roles, Capabilities?
ACT definition & usage, explicit permissions?
![Page 21: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/21.jpg)
Metadata Security Testing: Duration
How long does high coverage testing take?
Weeks?
Days?
Hours?
Seconds?
![Page 22: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/22.jpg)
Metadata Security Testing: Frequency
How often do you perform testing?
Daily?
Weekly?
Monthly?
Annually?
Hardly ever: only when troubleshooting?
![Page 23: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/23.jpg)
Problems with Manual Testing From our experience:
It’s almost exclusively an ad-hoc manual process
It takes too long, it’s inconsistent & it’s error-prone
Consequently it’s not done …
with enough coverage & reliability to detect problems
with enough frequency to detect them promptly
So we looked at how we could automate it ….
![Page 24: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/24.jpg)
Automated Metadata Security Testing
![Page 25: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/25.jpg)
A Metadata Security Testing Framework
An engine that tests metadata against XML Test Specifications
![Page 26: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/26.jpg)
Why XML Test Specifications? Easier to read/write than SAS, Java or .Net code!
Wide variety of plain text or XML editors
Help from XML Schema validation
Can be auto-generated
Checked into Version Control Systems (git, svn, etc.)
Compare differences over time
![Page 27: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/27.jpg)
Test Group/Role Memberships for Users<Users complete=“false"><User required="true" name="sasadm“><DirectGroupMemberships complete="true"><Group required="true" name="SASAdministrators"/>
</DirectGroupMemberships><DirectRoleMemberships complete="true"><Role name="META: Unrestricted Users Role"/>
</DirectRoleMemberships></User><User required="true" name="sastrust">…
</User>…
</Users>
![Page 28: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/28.jpg)
Test Permission Patterns for ACTs<ACTs complete="true"><ACT required="true" repository="Foundation" name="Default ACT"><PermissionPattern complete="true"><Group required="true" repository="Foundation" name="PUBLIC"
permissions="-RM,-WM,-WMM,-CM,-R,-W,-C,-D,-A,-X,-S,-I,-U,-RF,-CT,-DT,-AT"/><Group required="true" name="SASUSERS" permissions="+RM,+WM,+CM"/><Group required="true" name="SASAdministrators"
permissions="+RM,+WM,+CM,+A"/><Group required="true" name="SAS System Services" permissions="+RM,+WM"/>
</PermissionPattern>…</ACT>…
</ACTs>
![Page 29: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/29.jpg)
Test Applied Access Controls for Objects<Objects>
<Object required="true" publicType="ACT" name="Default ACT" ><AccessControls complete="true">
<ACT required="true" name="SAS Administrator Settings"/><Group required="true" name="PUBLIC" permissions="-WM"/>
</AccessControls></Object ><Object required="true" publicType="Folder" parentFolder="/" name="System">
<AccessControls complete="true">…
</AccessControls></Object>
…</Objects>
![Page 30: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/30.jpg)
Test Effective Permissions for Objects<Objects>
<Object required="true" publicType="ACT" name="Default ACT" ><EffectivePermissions>
<Group required="true" name="SASAdministrators" permissions="+RMt,+WMt"/><Group name="SAS System Services" permissions="+RMt,-WMi"/><Group name="SASUSERS" permissions="+RMi,-WMi"/><Group name="PUBLIC" permissions="-RMi,-WMe"/><User name="sasadm" permissions="+RMi,+WMi"/><User name="sasdemo" permissions="+RMi,-WMi"/>
</EffectivePermissions></Object ><Object required="true" publicType="Folder" parentFolder="/" name=“HR">
<EffectivePermissions> … </EffectivePermissions></Object>
…</Objects>
![Page 31: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/31.jpg)
Test Golden Rules / Good Practices
…<AllowOnlyGroupsInACTs/><AllowOnlyGroupsInACEs/><AllowOnlyImplicitGroupDenials/><AllowNoACEs/><AllowNoUnprotectedACTs/><AllowNoGroupMembershipLoops/><AllowNoRoleContributionLoops/><AllowNoGroupsWithImplicitMembers/>…
![Page 32: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/32.jpg)
Testing a Single Environment: Export Once
Today: Export current/desired state asMetadata Security Test XML files
![Page 33: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/33.jpg)
Testing a Single Environment: Test & Repeat
Tomorrow, Next Week, Next Month:Compare current state to desired state using previously exported Metadata Security Test XML files
![Page 34: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/34.jpg)
Consistency Testing Different Environments
Export Metadata Security Test XML files from source environment to test for consistency in target environment.
![Page 35: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/35.jpg)
Summary & Conclusion
![Page 36: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/36.jpg)
Metadata Security Testing at Metacoda Consistency for our software testing environments
Multiple Environments (Multiple SAS versions too!)
Before: inconsistent, infrequent, multiple days of testing
Frequency: every night
Coverage: approx 3,000 tests each
Duration: less than 5 seconds each
Common, cross-environment, cross-version test scripts
Few SAS version / environment specific test scripts
![Page 37: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/37.jpg)
Manual Slow
(hours/days)
Infrequent(every few months)
Inconsistent
Low Coverage
Poor Test Documentation & Audit Logs
v.s. Automated Fast
(seconds/minutes)
Frequent(every day)
Consistent
High Coverage
Integral Test Documentation& Audit Logs
![Page 38: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/38.jpg)
Manual Slow issue detection
(days, weeks, months)
Poor use ofSAS admin time
Every Time We Test!(or not)
v.s. Automated Fast issue detection
(minutes/hours)
Better use ofSAS admin time
Create Initial Tests
Resolve Any Issues
![Page 39: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/39.jpg)
For More Information …
Blog: Testing Conditional Grants in SAS Visual Analyticshttp://platformadmin.com/blogs/paul/2015/09/testing-conditional-grants-sas-va/
Blog: Testing Recommended Practices with SAS Metadata Securityhttp://platformadmin.com/blogs/paul/2015/06/testing-recommended-practices/
Blog: SAS Metadata Security Testinghttp://platformadmin.com/blogs/paul/2014/03/sas-metadata-security-testing/
SAS Global Forum 2014 Paperhttp://support.sas.com/resources/papers/proceedings14/1761-2014.pdf
![Page 40: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/40.jpg)
Questions?
Email: [email protected]
Blog: http://platformadmin.com/
Twitter: http://www.twitter.com/PaulAtMetacoda
LinkedIn: http://au.linkedin.com/in/paulhomes
Web: http://www.metacoda.com/
Please come & talk to us at the Metacoda stand.
![Page 41: Test for Success: Automated Testing of SAS® Metadata ...€¦ · Easier to read/write than SAS, Java or .Net code! Wide variety of plain text or XML editors Help from XML Schema](https://reader031.vdocuments.us/reader031/viewer/2022011902/5f0ed5367e708231d441287e/html5/thumbnails/41.jpg)