CRA-Industry Update

February 23, 2021

Vivek SarkarBen Zorn

CRA-I Steering Committee (formed in Nov 2020)Lorrie Cranor, CMUFatma Ozcan, GoogleChris Ramming, VMwareVivek Sarkar, Georgia Tech (Co-Chair)Divesh Srivastava, AT&TGil Vandentop, IntelYY Zhou, UCSDBen Zorn, Microsoft (Co-Chair)

2

Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

3

4

Context: CRA Strategic InitiativesCRA Strategic Initiatives Implementation Ownership Recommendation

February 18, 2021

Indicates board or committee member Indicates staff

Initiatives (Phase 1) Action Planning Participants

Suggested Implementation Ownership

A1a. In collaboration with the computing research community, define tenets and practices for socially responsible computing

Ann Schwartz Drobnis Peter Harsha Ran Libeskind-Hadas Ellen Zegura

New Board Programmatic or Ad Hoc Committee

A2a. Engage the board in defining criteria for selection of activities to pursue A2b. Develop governance and decision-making framework and cycle for selection of activities

James Allan Andy Bernat Erik Russell Ellen Zegura

Executive Committee

B1b. Broaden involvement of computing researchers from member organizations, beyond the member contacts

Nancy Amato Khari Douglas Peter Harsha Brian Mosley Erik Russell Heather Wright Helen Wright Shar Steed

Staff, also see TIG recommendation as part of Governance report

B2a. Proactively influence government policy and procedures that affect the computing research community, in addition to responding to emerging issues

Alex Aiken Peter Harsha Stephanie Forrest Ellen Zegura

Government Affairs Committee

C2a. Continually gather and communicate examples of the positive impact of computing research on society

Khari Douglas Peter Harsha Brian Mosley Erik Russell Shar Steed Heather Wright

Staff

D1a. Meaningfully engage companies that conduct computing research

Ben Zorn Vivek Sarkar Lorrie Cranor Fatma Özcan Chris Ramming Divesh Srivastava Gil Vandentop YY Zhou

CRA-I Committee

E2a. Enhance awareness of computing research opportunities to a broader student population

Erik Russell Andrea Danyluk Sandhya Dwarkada Susanne Hambrusch Lori Pollock

CRA-WP or CRA-E

E2b. Utilize best practices to create guidance for ensuring a supportive community for diverse students in computing research

Erik Russell Burcin Tamer Ellen Zegura

CRA-WP

E3a. Expand CRA programming to enhance preparation experiences for computing research students

Susanne Hambrusch Lori Pollock Erik Russell

CRA-E

F1a. Institutionalize systemic, continuous, and targeted communications

Khari Douglas Peter Harsha Brian Mosley Erik Russell Shar Steed Heather Wright Helen Wright

Staff

CRA Strategic Initiatives Implementation Ownership Recommendation February 18, 2021

Indicates board or committee member Indicates staff

Initiatives (Phase 1) Action Planning Participants

Suggested Implementation Ownership

A1a. In collaboration with the computing research community, define tenets and practices for socially responsible computing

Ann Schwartz Drobnis Peter Harsha Ran Libeskind-Hadas Ellen Zegura

New Board Programmatic or Ad Hoc Committee

A2a. Engage the board in defining criteria for selection of activities to pursue A2b. Develop governance and decision-making framework and cycle for selection of activities

James Allan Andy Bernat Erik Russell Ellen Zegura

Executive Committee

B1b. Broaden involvement of computing researchers from member organizations, beyond the member contacts

Nancy Amato Khari Douglas Peter Harsha Brian Mosley Erik Russell Heather Wright Helen Wright Shar Steed

Staff, also see TIG recommendation as part of Governance report

B2a. Proactively influence government policy and procedures that affect the computing research community, in addition to responding to emerging issues

Alex Aiken Peter Harsha Stephanie Forrest Ellen Zegura

Government Affairs Committee

C2a. Continually gather and communicate examples of the positive impact of computing research on society

Khari Douglas Peter Harsha Brian Mosley Erik Russell Shar Steed Heather Wright

Staff

D1a. Meaningfully engage companies that conduct computing research

Ben Zorn Vivek Sarkar Lorrie Cranor Fatma Özcan Chris Ramming Divesh Srivastava Gil Vandentop YY Zhou

CRA-I Committee

E2a. Enhance awareness of computing research opportunities to a broader student population

Erik Russell Andrea Danyluk Sandhya Dwarkada Susanne Hambrusch Lori Pollock

CRA-WP or CRA-E

E2b. Utilize best practices to create guidance for ensuring a supportive community for diverse students in computing research

Erik Russell Burcin Tamer Ellen Zegura

CRA-WP

E3a. Expand CRA programming to enhance preparation experiences for computing research students

Susanne Hambrusch Lori Pollock Erik Russell

CRA-E

F1a. Institutionalize systemic, continuous, and targeted communications

Khari Douglas Peter Harsha Brian Mosley Erik Russell Shar Steed Heather Wright Helen Wright

Staff

Mission and Vision Statements• CRA Mission Statement: CRA’s mission is to enhance innovation by

joining with industry, government and academia to strengthen research and advanced education in computing.

• CRA-I Mission Statement (draft): CRA-I’s mission is to create a computing research ecosystem that focuses on opportunities with industry to leverage the potential synergies among industry, government, and academia for mutual benefit and improved societal outcomes.

• CRA-I Vision: CRA-I is the one-stop portal for industry to engage with the computing research community across academia, government, and other companies

5

Background: Industry/AcademiaAd Hoc Committee Report

(Presented in July 2020 Board Meeting)

https://cra.org/cra-committee-on-industry-academia-interactions-releases-report/

Vivek Sarkar (Chair), Nancy Amato, Susan Davidson, David Ebert, Mark Hill, Charles Isbell, Shwetak Patel,

Chris Ramming, Divesh Srivastava, Eric de Sturler, Marvin Theimer, Ben Zorn

Executive Summary (details in report)• Survey sent to 221 computing department chairs (special thanks to Betsy!)

• 105 responses --- indicates strong interest by CRA member organizations• Over 60% also said that they were open to engaging further on this topic

• Significant industry engagement is under way between faculty and industry, with an increasing trend

• Mix of positive and negative aspects of industry/academia engagements, with the positives outweighing the negatives

• Wealth of information from survey can help guide future CRA activities related to industry-academia engagements

• Overall: clear opportunity for CRA to add value to member institutions and industry partners by facilitating and strengthening their interactions 7

Types of industry organizations involved in faculty engagements

8

CRA-I’s Initial Contact List(to be extended)• Initial source: CCC, Georgia Tech• Additional sources: Company names

provided in Industry/Academia survey• Wide range of companies è need for a

“catalog” of options to engage with universities

• CRA-I can also help direct companies to other CRA committees and activities, as appropriate

9

1776 DC 3M Accenture Technology Labs Adobe Adventium Lab Aerospace Corporation Aetna Airbnb Alphabet Inc. Amazon Amazon Web Services AMD American Express Analytics ANSER Apple Aptiv Asurion AT&T Research Automatic Data Processing, Inc. (ADP) Automatic Labs Bank of America Bell Labs Research Blackbaud, Inc. BlackRock Bloomberg Boeing Booz Allen Hamilton Boston Scientific BT Americas BuzzFeed News BWX Technologies Capital One CAST, Inc. Center for Senior Independence Chick-fil-a Cigital Cisco Cisco Systems, Inc. CITADEL Citi CIVILIAN Civis Analytics Clever CloudMinds Inc Coca-Cola ConocoPhillips Corporation Cooper Lighting Corning Create with Context

Critical Mass Cyber Pack Ventures, Inc. D.E. Shaw Group DAI Data Dell EMC Deloitte Consulting Develop For Digital Manufacturing and Design Innovation Institute (DMDII) Disney Animation DMI Dow Chemical Dropbox EarthLink Eccalon Education Advisory Board Educational Testing Service Efficient Cities Envieta EPIC Equifax ESRI Expanse, Inc. F5 Networks Facebook Fetch Robotics Inc. Formerly Oracle Friend Frog Gartner GE Aviation GE Digital GE Global Research GE Power Genentech Tech General Motors Georgia Pacific GlobalFoundries Good Research Google Gradiant Green Hills Software, Inc. GTRI HERE Hewlett Packard Enterprise Honeywell HP Labs HubSpot i3solutions

IBM IBM Research ICSI IDA Indeed Inflection InfoSec World Bank Integral Mind Intel Intel Labs Intercontinental Exchange -- ICE IonQ J.P. Morgan JETRO John Deere Johnson & Johnson Kaiser Kaiser Permanente Center for Health Research Keysight Kimberly-Clark Koch Industries KPMG KS TechPros L3Harris Corporation Lambdalytics LLC Leidos Corp Lewis-Burke Associates LLC Lockheed Martin Lyft Macy's Magic Leap Mailchimp MarkLogic Corporation Mastercard McAfee McGrath Analytics McKesson Measure Measure UAS Inc. Medtronic MeYouHealth Micro Craft Inc. Microsoft Corporation Microsoft Research MindSpring MorphoTrust Mozilla MPH - a Porsche Company

Myomo National Security Agency NB Motors NCR Corporation Nervana New Scientist New Venture Partners NICT Nimbis Services Nissan Research Nokia Norfolk Southern Northrop Grumman Nutanix NVIDIA OIT Concepts LLC Omada Health OneTrust Opaxis Orange Silicon Valley Orbital Insight Oscar Health OSIsoft Palantir Parkview Parkview Health PayPal Pearson Peloton PennDOT Prognosys Protiviti PwC Qualcomm, Inc. QxBranch Raytheon Redfin RetailMeNot, Inc. Revel Systems Revmetrix, AI tech startup Rieg and Associates Rigetti Computing Riot Games Roblox Rockwell Salesforce SambaNova Systems Samsung Sandia National Labs SAS

SentiMetrix, Inc ServiceNow Siemens Siemens Corporation SM&A Snapchat Social Tables Splunk SRI Stanley Black & Decker State Farm Stryker Corporation SunniBrown Inc. Sunrise Technologies, Inc. Synced Technology Co. Synopsys, Inc. Tableau Software The Home Depot The MITRE Corporation The QED Group, LLC Tipograph Law Toyota Motor North America Travelers Trilateral Research Ltd Trivium Consulting working with HPCWire Truist Twilio U.S. Xpress Uber Union Pacific UPS Urjanet Verizon Connect Verizon Media Viasat Visa Visa Research VMware VMware Research Vox Media Walmart WarnerMedia WIOMAX Workday Yahoo Yu & Robinson Zillow Group

CRA-I Need Finding Process

● Initial set of interviews with the following people, representing a range of expertise across industry, academia, and government:○ Ron Brachman, Karina Edmonds, Peter Harsha, Kim Hazelwood, Mark Hill, Jim

Kurose, Margaret Martonosi, Raghu Ramakrishnan, Vijay Saraswat, Alfred Spector, Jaime Teevan, Marvin Theimer, Manuela Veloso

○ Many of our interviewees observed that there is a unique opportunity for CRA-I to fill a long-standing need with its bridging role.

● Pilot survey created to gain broader insight on CRA-I opportunities from industry partners:○ Results to be collected, analyzed and shared in July 2021 Board meeting○ Next slide has sample output from 12 responses

10

Q10 - In which of the following do you see value in what CRA-I could provide to your company? Select all that apply. (Sample output from 12 responses)

4�����,Q�ZKLFK�RI�WKH�IROORZLQJ�GR�\RX�VHH�YDOXH�LQ�ZKDW�&5$�,�FRXOG�SURYLGH�WR�\RXU

FRPSDQ\"�6HOHFW�DOO�WKDW�DSSO\�

�ÐÉÉ��ë°É¨�ü°ë­�����È°�ã��аɨ

�ÐÈÜðë°É¨�ß�ã��ß�­ț

�ÐÉÉ��ë°É¨�ü°ë­¨Ðû�ßÉÈ�Éë

�¨�É�°�ãȡÂ��ããðÜÜÐßë°É¨�Ðß

�ÐÉ�ð�ë°É¨��ÐÈÜðë°É¨ß�ã��ß�­ț

�ÐÉÉ��ë°É¨�ü°ë­Ðë­�ß��ÐÈÜ�É°�ã�ÐÉ

Üß�ȭ�ÐÈÜ�ë°ë°û��­�ÂÂ�ɨ�ã�ß�Â�ë��

ëÐ��ÐÈÜðë°É¨ß�ã��ß�­

��ð��ë°É¨��ÐÈÜ�É°�ãÐÉ���ãë�Üß��ë°��ã�°É

�ɨ� °̈ɨ�ü°ë­ðÉ°û�ßã°ë°�ã

��ð��ë°É¨ðÉ°û�ßã°ë°�ã�ÐÉ���ãë

Üß��ë°��ã�°É�ɨ� °̈ɨ�ü°ë­

�ÐÈÜ�É°�ã

Kë­�ß

/Éďð�É�°É¨�§ðëðß��°ß��ë°ÐÉã�°É�ĂÐðß

°É�ðãëßĂ

ƺ ƺțƿ ƻ ƻțƿ Ƽ Ƽțƿ ƽ ƽțƿ ƾ ƾțƿ ƿ ƿțƿ ǀ ǀțƿ

^­Ðü°É¨�ßÐüã�ƻ�ȭ�ǂ�Ч�ǂ

ș &°�Â��­Ð°���ÐðÉë

ƻ �ÐÉÉ��ë°É¨�ü°ë­������È°�ã��аɨ��ÐÈÜðë°É¨�ß�ã��ß�­ț ƻƽțƽƽɟ ƾ

Ƽ �ÐÉÉ��ë°É¨�ü°ë­�¨Ðû�ßÉÈ�Éë��¨�É�°�ãȡÂ��ã�ãðÜÜÐßë°É¨�Ðß��ÐÉ�ð�ë°É¨��ÐÈÜðë°É¨�ß�ã��ß�­ț ƻǀțǀǁɟ ƿ

ƽ �ÐÉÉ��ë°É¨�ü°ë­�Ðë­�ß��ÐÈÜ�É°�ã�ÐÉ�Üß�ȭ�ÐÈÜ�ë°ë°û���­�ÂÂ�ɨ�ã�ß�Â�ë���ëÐ��ÐÈÜðë°É¨�ß�ã��ß�­ ƻƽțƽƽɟ ƾ

ƾ ��ð��ë°É¨��ÐÈÜ�É°�ã�ÐÉ���ãë�Üß��ë°��ã�°É��ɨ�¨°É¨�ü°ë­�ðÉ°û�ßã°ë°�ã Ƽƺțƺƺɟ ǀ

ƿ ��ð��ë°É¨�ðÉ°û�ßã°ë°�ã�ÐÉ���ãë�Üß��ë°��ã�°É��ɨ�¨°É¨�ü°ë­��ÐÈÜ�É°�ã Ƽƺțƺƺɟ ǀ

ǀ Kë­�ß ƽțƽƽɟ ƻ

ǁ /Éďð�É�°É¨�§ðëðß���°ß��ë°ÐÉã�°É�ĂÐðß�°É�ðãëßĂ ƻƽțƽƽɟ ƾ

ƽƺ

YƻƺȢǀȢe�{e�ȭ�Kë­�ß

Interview Questions

We asked how CRA-I can support the following interactions:Industry-Industry (I-I)Industry-Academia (I-A)Industry-Academia-Government (I-A-G)

Top level questions:● What role can CRA-I play in providing value?● What topic areas are of particular interest and/or most challenging?● What approaches to collaboration between the entities above should

we consider?

12

Academia

Government

Industry

CRA-I Observations and Synthesis from Interviews (1)CRA-I’s Role

● Bring together industry partners & CRA for shared understanding of issues○ Including identifying partners and creating social connections

● Help connect industry/academia/government in seeding partnerships

● Create a shared understanding of best practices working with academia and other industry researchers○ Help make creating a corporate computing research culture easy and effective (e.g.,

a term suggested was “Research in a Box”)

Emerging topics/challenges mentioned:Explainable AI, ethical AI, social responsibility, cloud

13

CRA-I Observations and Synthesis from Interviews (2)Collaboration ideas

● Create mingling opportunities and information sharing that connect companies and influencers who inform computing research

● Provide a central place for industry to develop a shared perspective on important social implications of tech, including community standards (e.g., privacy, fairness, etc.)

● How? Writing white papers, hosting workshops, etc.

● Provide guidance and support effective data sharing ● Programs that help faculty and industry researchers exchange places

14

Opportunities for Collaboration with other parts of CRA

CCC- Example: quantum readiness study

- A first-time experiment in having industry input on CCC study topics- Resulted in one of the recent quadrennial papers- Company participants valued the experience and commented on how they could not imagine

another neutral forum where the conversations would have been so forthright

CRA-WP- CRA-I can introduce partners to opportunities with CRA-WP

- CRA-I can learn from CRA-WP experience with companies

CRA Government Affairs- CRA-I partners can contribute to possible GA events

- GA can benefit from CRA-I input on new government initiatives

15

Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

16

Break Out Group Discussions (11:45am hard stop)1. Connection between CRA-I and the rest of CRA: We focus on what Board members have to say about how the CRA-I

can effectively interact with other CRA committees (like CRA-WP, CERP, CCC, CRA Government Affairs, Awards, etc.)2. Opportunities and Pain Points in Industry-Academia Interactions in CS departments - The goal is to

collect input from board members on questions that CRA-I could focus on with respect to developing best practices (“design patterns”) on issues related to faculty, student, intellectual property, funding, etc.

3. Best practices on using cloud computing as research infrastructure - increasingly research is being conducted using cloud computing infrastructure. This discussion focuses on sharing best practices to leverage cloud infrastructure to accelerate the research process.

4. Data sharing practices - Data is critical for innovation in a data-driven society. What kinds of data can be shared that is of value?How do we share data while keeping rights to it (similar to patenting algorithms)?

5. Social justice – Brainstorm about how to involve industry partners with other CRA efforts on this topic. Consider questions such as: How can we leverage technology to close the injustice gaps? What role does industry and academia have to ensure equality of opportunity?

6. Responsible social media – Consider bringing partners to the table to consider questions such as: What are the technical obstacles to ensuring social media is not harmful? What roles do industry, academia and the government have in ensuring responsible social media? How do we ensure individuals’ rights of freedom of speech is not violated while preventing fake news?

7. Seeding new I/A/G alliances: Example: changing the nature of the semiconductor roadmap - For decades, the semiconductor industry has fueled Moore’s “law” with public private partnerships driven by the SRC. To maintain national leadership, the traditional semiconductor industry needs more research engagement from design and systems companies.

17

Outline

• Update on CRA-I

• Breakout groups

• Wrap-up

18

CRA-Industry Goals for 2021

- Initiate activities that require lower partner investment initially- Target: initiate “Computing Research Industry Pulse” round tables on topics of

greatest interest to industry, possibly in partnership with other CRA committees

- Initiate 1-2 task forces focusing on particular opportunities for industry/academic/government partnerships- Target: convene in-person meetings around these areas in Spring/Summer

2022- Engage with companies as potential partners

- Expand communications, outreach efforts beyond interviews- Target: populate web site, author “value” white papers by end-2021- Target: recruit initial set of member companies by February 2022- Target: coordinate activities with partners around Snowbird

and other CRA meetings19

Proposed Membership/Participants(from draft charter in 2020 report)• The CRA-I steering committee serves to catalyze and track project-based

activities that fulfill CRA-I’s vision and mission.• A CRA-I advisory board will be established, to which each member

company will be allowed to nominate up to one board representative. The board is the voice of its constituent members to surface issues and help drive CRA opportunities at the intersection of industry, academia, and government.

• Project teams. At the discretion of the steering committee, project teams will be established to execute on CRA-I projects, and can draw on a wider community of participants. Each project team must include at least two CRA-I board members.

20

Summary

• CRA-I has been launched – thanks for your support!• Our approach is start small in 2021 and build a constituency with

concrete activities• We plan to start identifying potential champions, members and

partners by end-2021• Stars are lining up for CRA-I: industry needs, CCC activities, CRA

Strategic Planning, NSF focus on technology transfer, …• Our ask for you --- let us know how you would like to participate in

CRA-I to help achieve our vision• We will also build on input from today’s breakout groups

21