laccd final report

7/31/2019 LACCD Final Report

1/48


2/48


3/48

Review of the Processes Used to

Select an Inspector General for the

Los Angeles Community College District

Submitted by

Harvey M. Rose Associates, LLC

http://www.harveyrose.com

July 16, 2012


4/48


5/48

Table of Contents

Executive Summary ................................................................................................................... i

Background, Objectives, Scope and Methodology ................................................................... 1

1. Analysis of Process for Selection of Contract Inspector General ................................... 9

2. Assessment of Qualifications and Experience of Bidders ............................................. 24

Appendix: LACCD Professional Services Procurement Policies


6/48


7/48

Executive Summary


ii

suffered from weaknesses in the clarity and specificity of contractor requirements in theRequest for Proposal (RFP), weak methods and poor documentation of the proposalreview and evaluation process and unclear evaluation criteria for, and documentation of,the finalist interview process.

The requirements and expectations for the Inspector General contractor changed betweenwhat was included in the RFP and how finalists were evaluated by the finalist interviewpanel. The firm ultimately awarded the contract was not the most qualified based on theRFP requirements, but appears to have best met the expectations of the interview paneland the Chancellor. As a result, the District did not obtain the full benefits of competitivebidding since the actual criteria used for selection varied from that in the RFP uponwhich bidders based their proposals.

LACCDs solicitation process for contract Inspector General services generally adheredto the District special and professional service procurement policies and procedures,though with some exceptions. A review of those policies and procedures found that theyare generally adequate though weaknesses were identified in the areas of: documentationrequirements; bidder evaluation and scoring protocols; and, conflict of interest vettingrequirements. The Inspector General solicitation process was found deficient in theseareas as well.

A summary of individual findings in this report is as follows.

Current District Policies and Procedures for Procurement of Professional ServicesContain Weaknesses

District policies established for the selection of contractors for professional and specializedservices is consistent with legal requirements and is generally adequate. However, our review of

existing District policies identified gaps in (1) documentation requirements; (2) scoring methodsand protocols; (3) bidder evaluation guidance; and, (4) conflicts of interest. Many of theseweaknesses were observed in the Office of Inspector General solicitation process.

The RFP Lacked Specificity and Clarity

The process for soliciting and selecting a contractor for Inspector General services varied fromcommon District practice for professional services in that it: (1) was a new function that theDistrict had no prior experience with; (2) was requested by the Board of Trustees rather than adepartment familiar with the services to be provided that would have typically prepared thescope of work, contract duration, and provided a listing of firms for outreach, and; (3) occurred

at a time when two key executive management positions were vacant: the LACCD Chancellorand the Director of Business Services.

Development of the RFP and scope of work, as well as the process for review of proposalsreceived, was delegated to the Deputy Chancellor. While the RFP provided a general overviewof the work to be done by the new function of the Office of Inspector General, it lackedspecificity and clarity in key areas pertaining to the functions. The RFP did not clearly state a


8/48

Executive Summary


iii

District expectation that the selected contractor should provide a full-time position to serve asInspector General. Though this information was communicated to potential bidders in anaddenda to the RFP, that addenda was issued May 7, 2010, giving bidders an unrealistically shortthree business days to secure a full-time senior level consultant with the appropriate skills andqualifications. Consequently, many of the bidders did not propose a team member to serve as

full-time Inspector General.

The RFP was vague on contractor deliverables. Specific deliverable requirements were notspecified other than references to the Office being required to conduct audits and investigations,make regular reports to the Board of Trustees and operate a whistleblower hotline. Therequired number of audits or investigations to be conducted or reports to be made to the Board ofTrustees was not provided, making it challenging for the bidders to determine the level of effortrequired and to prepare price estimates. The lack of specificity in this regard also limited theDistricts ability to determine how the bidders price estimates compared to each other.

The RFP was prepared without a full listing of standards, priorities and requirements of the roleof the Inspector General. There was no mention in the RFP of required Inspector Generalprofessional standards in key areas such as independence and conflicts of interest. Given thenature of the work to be performed, such standards are critical to the successful functioning ofthe Office.

A number of bidders that submitted questions prior to submitting their proposals demonstratedthat they did not have a clear understanding of what the District currently had in place in the wayof Inspector General services and what exact services the selected contractor would be requiredto perform. Such information should have been provided in the RFP.

Proposal Review Process Poorly Planned, Executed, and Documented

The eleven proposals received were reviewed and scored by a panel of District personnel andoutside parties based on the following criteria:

Qualifications, Experience, and References (25 points) Staffing, Project Organization (20 points) Work Plan Technical Approach (20 points) Fee (25 points) Miscellaneous, Exceptions, and Deviations (10 points)

The scoring sheets used for this purpose were vague and the review panelists did not receive

detailed guidance about how they should be applied to the bids for the Inspector Generals Officecontract. For example, the panel was not instructed on what appropriate experience andqualifications should include and awarded the highest score to a firm with no auditing experienceat all, though auditing was one of the two key functions to be performed by the Office ofInspector General. That highest scoring firm was subsequently disqualified.


9/48

Executive Summary


iv

While each member of the four person proposal review panel was expected to assign scores toeach of the eleven bidders, three bidders were not fully scored, resulting in lower overall scoresfor these firms. Specifically, one panelist did not provide a score for one firm due to a personalconflict of interest, and another panelist did not rate two other proposals because the firms hadbid only on the whistleblower hotline service. Through these panelists own interpretations of

how the scoring process should work, the panelists effectively rated the three firms as zero,and these firms thus received significantly lower scores through no fault of their own. Inaddition, the total scores were not used to provide overall rankings, which is an exception toDistrict policy. Rather, bidders were advanced as finalists based on a group discussion- afterindividual scoring was complete- in which panelists expressed their opinions about theproposals.

District policy calls for firms that receive the highest cumulative score to be recommended forcontract award. But the firm with the highest score was disqualified by the panel because it wasconsidered too small to do the needed work.

First Round of Interviews, Critical to Selection of an Inspector General, wasPoorly Planned and Executed

A separate interview panel was assembled to interview the four finalist firms identified by theproposal review panel. The interview panel consisted of District staff, members of the Board ofTrustees and two parties external to the District. Again, little information was provided to thepanelists about District expectations for the Inspector Generals Office. Questions were preparedin advance, but they were not used consistently for all bidders.

According to District policy, information obtained through the interviews may then beconsidered by the proposal evaluators in assigning final scores to the proposals. LACCDmanagement departed from this policy by not assigning final scores. Though the interviews were

key to final selection of a bidder, the criteria used to evaluate the bidders was not documented,and the interview results were not scored or formally documented. Rather, consistent with theproposal evaluation process, a group discussion took place after the interviews, and agreementwas reached on bidder rankings. Reference checks were conducted by an outside contractor onall four finalist firms that were interviewed1. Though the reference interviews were transcribed ina written report, is not clear how these results were used by the interview panel or District staff.

A second round of interviews was conducted with the same bidders in August, 2010 to ask moredetailed questions in a less formal setting. Unlike common District practice, there was no set offormal questions to be asked of each bidder, as was the case for the first round of interviews. Thesecond set of interviews was also not formally documented.

Two of the firms made changes in their proposed staffing between the first and second round ofinterviews. In one case, the firm added investigator staff to their team. In the other case, thebidder clarified who would actually serve as the Inspector General. Both of these changes

1We could not verify the validity of the reference checks as names of the individuals interviewed were redacted on

the copies provided to our review team.


10/48

Executive Summary


v

reportedly improved the firms qualifications but it is not clear if all bidders were given theopportunity to do this or if these two firms made these changes on their own. The two biddersthat made the changes were recommended to the Chancellor as top finalists, one of which wasultimately selected and awarded the contract.

LACCD Did Not Actively Vet Conflicts of InterestThe bidder reference checks conducted after the first round of interviews did not includebackground checks on the firms or checks for potential conflicts of interest, which would havebeen useful given the sensitive nature of the work to be performed by the Office of InspectorGeneral. For example, the fact that the president of Policy Masters, Inc. had previously workedfor Gateway Science and Engineering, Inc., a company that had been contracting with theDistrict for several years to oversee bond-funded construction work, was not disclosed oraddressed in the process as a potential conflict of interest for an Office of Inspector General thatwould be auditing and investigating that firms work.

Bidders Experience and Qualifications were not Consistently or MethodicallyAssessed

The RFP emphasized that the Office of Inspector General would be responsible for audits andinvestigations, but the eleven proposals submitted were not systematically reviewed for theiraudit and investigation experience and qualifications. In fact, the top ranked bidder had no auditexperience whatsoever. The RFP required performance, operations, financial and complianceaudit experience, but only one of the four firms advanced as a finalist reported performance oroperations audit experience in their proposal.

Inspector General Contract Award was Based on Qualifications other than those

Emphasized in the RFPThe firm ultimately selected by the District to serve as the Inspector General was Policy Masters,Inc. Members of the interview panel reported that this firm made the best presentation at the tworounds of interviews and had a team with knowledge and experience in the construction industryand knowledge of the District and its needs. While these can all be considered valuable attributesfor the selected contractor, the firm did not have experience in directing audits and investigationsor a history of successful work elsewhere as a firm since it was newly established. Asubcontractor to the firm had financial audit experience but did not have performance oroperational audit experience, as required in the RFP. The firm did not have investigationexperience initially though it added a team member who could perform that role between the firstand second round of interviews. The originally proposed team members had related experiencein school district construction projects, development of policies and procedures and financialaudits.

The interview panel appears to have redefined the role of the Inspector Generals Office throughthe solicitation process compared to how it was defined in the RFP. While it may have ended up


11/48

Executive Summary


vi

with the contractor that best met its needs out of the bidder pool, it did not obtain the full benefitsof competitive bidding since the actual criteria used for selection varied from that in the RFP.

Report Review by District

The District was provided a draft version of this report for review. Subsequently, an exitconference was conducted with District representatives to obtain any factual clarifications andcomments on the report. The District also provided a written response with more detailedresponses to the specific findings and details of the report.

Key points in the Districts written responses were agreement with the Inspector General Officesolicitation process not being well documented and the need for clarification of several points inthe Districts Professional Servicers Procurement Policy (PP-04-08) and plans for improvementin both of these areas. While better documentation and a more clear professional servicesprocurement policy would have improved the Inspector General solicitation process, a few areaswarrant additional mention:

1. In response to the point in the report that the Inspector General interview panel did notscore or methodically rank the bidders that were interviewed, the District states a numberof times in their written response that scoring the interviews is not required by currentpolicy. The more important deficiency represented by the absence of formal scoring isthat there is no record of the criteria used to assess the interviewed bidders and no recordshowing that the interview evaluation process was systematic or methodical. A finalscoring sheet, incorporating the results of the interview process, or some otherdocumentation of a methodical assessment using standardized criteria, would provideimportant evidence of an objective and transparent process.

2. The District states in its written response that having two panels for the proposal reviewand interview processes was designed for greater objectivity. Whatever its intent, anegative impact of this approach was that the interview panel did not have information onhow the bidders had been assessed relative to the criteria in the Request for Proposals,which would have provided areas to focus on in the interviews relative to standardizedevaluation criteria. Combined with a lack of familiarity with inspector general functions,the interview panel was not well equipped to evaluate the strengths and weaknesses of thefirms relative to the functions the contractor would be expected to perform.


12/48


1

Background, Objectives, Scope and Methodology

Background

Harvey M. Rose Associates, LLC (HMR) was retained by the Los Angeles City Controller toconduct a Review of the Processes Used to Select an Inspector General for the Los AngelesCommunity College District. This independent review was conducted at the request of theChancellor and President of the Board of Trustees of the Los Angeles Community CollegeDistrict based on concerns raised by the State Controllers Office1 and others regarding theselection process for Inspector General services and qualifications of the selected bidder forthese services. The purpose of the review was to assess the processes used by the Los AngelesCommunity College District (LACCD) to select a contract Inspector General to oversee theDistricts bond-funded construction program.

LACCD, serving over 250,000 students annually at nine colleges, is the largest communitycollege district in the United States. From 2001 to 2008 the voters in the LACCD approved threebond measures totaling $5.725 billion to expand and make improvements to LACCD campuses.

Due to a number of deficiencies in the resulting capital facilities development program, asidentified in a report by a District consultant, the LACCD Board of Trustees adopted a motion inMarch 2010 authorizing implementation of one of the consultants recommendations: creation ofan Office of Inspector General to oversee the bond-funded capital program and to ensure that itperforms with the utmost integrity and efficiency.2 District management decided to engage acontractor to fill this role rather than creating a new District position because bond proceedscannot be used to pay for District administrative staff and District management was concernedabout committing General Funds to cover the new expense.

Between approximately March and September 2010, the District engaged in a professionalservices solicitation process in which a Request for Proposals for a contract Office of Inspector

General was developed and issued, eleven proposals were received and evaluated by the District,four finalist bidders were interviewed, and one firm, Policy Masters, Inc. was selected andapproved by the Board of Trustees to serve as the Districts Inspector General under contract.The agreement between the District and Policy Masters, Inc. calls for an annual payment of$701,680 to perform audits and investigations related to the Districts bond-funded constructionprogram and to operate a whistleblower hotline. The contractor has been functioning in this rolesince October 2010.

Timeline of Solicitation, Selection, and Contract Award Process Events

Exhibit 1.1 shows a timeline of major events in the process of soliciting, selecting and awarding

a contract for Inspector General services. The exhibit provides the date of the event as well as abrief description of the event and, when relevant, the individuals involved.

1 The State Controllers Office conducted an audit of the LACCD Proposition A/AA and Measure J bondexpenditures. Their report, made public in August 2011, included a finding indicating irregularities in theprocurement of Inspector General Services.2 Board of Trustees Resolution BT8, adopted March 10, 2010.


13/48



2

As shown in Exhibit 1.1, the entire process took LACCD management a total of seven monthsbeginning on March 10, 2010 when the Board of Trustees passed a resolution adopting arecommendation from a consultant report to create an Office of Inspector General. On April 26,2010, approximately six weeks after the Board resolution calling for the creation of an Office ofInspector General, the District issued a Request for Proposals (RFP). Prospective bidders wereprovided nine days to submit written questions and 17 days to submit their proposals.

With input from various District staff members, the LACCD Deputy Chancellor assembled aproposal review panel on May 14, 2010, the day after proposals were due. After reviewing theeleven proposals submitted to the District, this panel, comprised of District staff and theDistricts Bond Counsel, identified four bidders as finalists to be interviewed. The followingweek, the Deputy Chancellor had staff conduct reference checks on two firms: (1) Thompson,Cobb, Bazilio & Associates, P.C. and (2) Policy Masters, Inc. Although staff may haveconducted reference checks on the two other finalists (Ernst and Young; Deloitte), we have noevidence that occurred. That same week, on May 20, 2010 (six days after the initial proposalreview) the Deputy Chancellor assembled a panel comprised of members of the Board ofTrustees, the District Chief Financial Officer and an external attorney and forensic accountant to

conduct interviews with the four finalist firms.

On August 4, 2010, or about two and a half months after the District interviewed the fourfinalists, at the direction of the president of the District Board of Trustees and the incomingChancellor, the Deputy Chancellor reassembled a slightly modified version of the interviewpanel to conduct a second round of interviews with the same four firms. Between the first andsecond round of interviews, LACCD management arranged for an outside firm, CommunityCollege Search Services, to conduct reference checks and prepare reports on the four firmsinvited to interview. On August 12, 2010, about one week after the second round of interviews,the Deputy Chancellor sent a memo to the incoming Chancellor stating that the selectioncommittee had narrowed the finalists to two firms: (1) Ernst and Young and (2) Policy Masters,Inc.

The Chancellor ultimately chose Policy Masters, Inc. after receiving the memo from the DeputyChancellor and conducting his own interviews. On October 6, 2010 the Chancellor and DeputyChancellor made a formal recommendation to the Board of Trustees to approve an agreementwith Policy Masters, Inc. for Inspector General services. The Board adopted the recommendationand the contract with Policy Masters, Inc. was executed five days later on October 11, 2010.


14/48



3

Exhibit 1.1

LACCD Inspector General Process for Procuring Inspector General Services

Event Date CommentsCapstone Report

Recommendations to LACCDBoard of Trustees & InterimChancellor

3/10/2010 Capstone Advisory Group, LLC recommended that the District create

an Office of Inspector General.

Board Resolution on Establishmentof an Office of Inspector General

3/10/2010 LACCD Board of Trustees passes a resolution directing the Chancellorto establish an OIG.

RFP 10-12 Issued by LACCD 4/26/2010 LACCD issues RFP 10-12 for the establishment of Inspector Generalservices.

Deadline for Written Questions 5/5/2010 The deadline established for proposers to submit any questions(required to be in writing) with a 2:00 PM deadline to ContractsManager.

Proposal Submission Deadline 5/13/2010 Deadline for submission of proposals was 2:00PM and had to be aphysical delivery. LACCD receives 11 proposals total. According tointerviewees, only 9 were fully responsive (for OIG andwhistleblower

hotline services).Initial Review (& Scoring) ofFirms

5/14/2010 Firms are reviewed by District staff: (1) James Watson, (2) Tom Hall,(3) Adriana Barrera; & (4) Lisalee Wells, Bond Counsel

Memo from Deputy Chancellor toMembers of the InterviewCommittee for OIG

5/17/2010 Memo from Deputy Chancellor states that another committee reviewedand rated the proposals received in response to the RFP and theproposals which are forwarded are the four most highly rated firms.

1st Round of Interviews 5/20/2010 Interviews conducted with (1) Thompson, Cobb, Bazilio & Associates;(2) Policy Masters, Inc.; (3) Deloitte Services, LP; (4) Ernst & YoungLLP. The interview panel: (1) Jeanette Gordon, District CFO; (2) StuartRudnick, private attorney; (3) Mona Field; and (4) Sylvia Scott-Hayes,District trustees; and (5) Anna Lim, KPMG forensic accountant.Lisalee Wells and Lloyd Silberstein served as "resources" to thecommittee.

Reference Checks on Four FinalistFirms

Late May -Late July

2010

LACCD utilizes Community College Search Services to conductcomprehensive reference reports on the four firms interviewed by thepanel.

Memo from Deputy Chancellor toMembers of the InterviewCommittee for OIG

6/18/2010 Memo from Deputy Chancellor was for the purpose of providingInterview Committee members reports on the reference calls for OIG.

2nd Round of Interviews 8/4/2010 Interviews conducted with same firms as first round of interviews. Thecommittee was comprised of: (1) Stuart Rudnick, private attorney; (2)Mona Field and (3) Sylvia Scott-Hayes, District trustees, and (4) DougFarrow, KPMG forensic accountant. Lisalee Wells and LloydSilberstein served as "resources" to the committee.

Deputy Chancellor Memo toIncoming Chancellor on Finalists

8/12/2010 Memo from Deputy Chancellor to incoming Chancellor re. results of2nd Round Interviews. Chancellor was notified that the committeenarrowed the finalists to (1) Ernst & Young and (2) Policy Masters, Inc.

New Chancellor Chooses Awardee 8/2010 Chancellor chooses Policy Masters, Inc. as the finalist after interviewswith Policy Masters, Inc. and Ernst & Young.

Board Item on OIG Contract 10/6/2010 Chancellor and Deputy Chancellor formally recommended establishinga contract with Policy Masters, Inc. for OIG services to the Board ofTrustees.

Contract executed with PolicyMasters, Inc.

10/11/2010 LACCD executes a contract with Policy Masters, Inc. with a start dateof 10/7/2010 and an end date of 10/6/2015.

Source: LACCD RFP 10-12 Documentation File


15/48



4

Common District Practices for Solicitation of Professional Services

We conducted a review of a sample of other professional services solicitations in order to obtainan understanding of common District practices for the solicitation of professional services.Specifically, we reviewed six other professional services solicitations that occurred between2008 and 2011. Our findings from this review found the following:

Consistent with the Inspector General solicitation, five of the six solicitations advertisedthe RFP for two weeks in the LA Daily Journal. Additionally, four of the six solicitationsincluded documentation that the RFP was sent directly to several firms that could beconsidered prospective bidders.

The average number of days between the posting of an RFP and the submission deadlinewas 22 days, which is five days longer than what was established for the InspectorGeneral solicitation. The amount of time for other solicitations ranged from 11 days to 28days.

Four of the six other solicitation files included proposal scoring sheets of some kind. Thedetail of these scoring sheets varied from a basic form without instructions on how toscore each category, to a detailed sheet that included a description of how eachsubcategory should be scored.

Three of the six solicitations included a pre-proposal meeting for prospective bidders,while three did not. The District did not hold a pre-proposal meeting for the InspectorGeneral solicitation.

Two of the six other solicitations appeared to include one round of interviews. Unlike theInspector General solicitation, none of the six other solicitations included a second roundof interviews.

Unlike the Inspector General solicitation, the results of the two solicitations that includedinterviews were consistent with District policy that indicates that: 1) scores will beassigned to proposals; and, 2) the results of interviews, if conducted, may be consideredby proposal evaluators to determine each bidders final score. While interviews andinterview scoring is optional, the policy does indicate that a final score will beassigned. Therefore, some process of scoring or documenting the Inspector Generalbidder interviews would have been beneficial in assigning final scores supporting theselection decision.

Other District Professional Services Solicitation Procedures

BuildLACCD, the District organization responsible for managing the bond-funded constructionprogram, maintains its own set of professional service solicitation policies and procedures sincethey hire contractors for bond-funded activity separate from the District process. For professionalservices solicitations greater than $78,900 (threshold for approval by the Board of Trustees),BuildLACCDs guidelines call for formal establishment of an Evaluation Team, evaluationgrading, interview and selection criteria and a process and schedule. The policies also call fordocumentation of all steps in the process. The solicitation of Inspector General services did not


16/48



5

fall under these procedures because of the independent nature of the Office of Inspector General.Specifically, the Office of Inspector General would be responsible for oversight of theBuildLACCD program. It therefore would be inappropriate for the District to have allowedBuildLACCD staff to oversee the solicitation and selection of Inspector General services.

Objectives and Scope

The scope of this independent review focused on an evaluation of LACCDs procurementprocesses for specialized professional services. The specific review objectives, as developed bythe Office of the City Controller and agreed to by LACCD, were as follows:

1. Evaluate the process used by LACCD during 2010 for the solicitation, selection, andcontract award for Inspector General services;

2. Determine how the selection process considered the experience and qualifications ofproposing firms, to ensure that the selected firm met the expectations of the LACCDBoard of Trustees for an Inspector General;

3. Determine whether LACCD followed its own procurement rules for the selection ofInspector General services, given the overarching objective of adhering to a fair and openprocess that would ensure that the most qualified proposer was selected; and,

4. Evaluate the selection process and recommended award to determine if there is evidenceof any inappropriate actions that occurred with regards to the selection and award.

Methods

The audit team conducted this review in accordance with Government Auditing Standards,December 2011 Revision by the Comptroller General of the United States. To achieve the projectobjectives, the review team conducted the following tasks:

1. Conducted an entrance conference with the Districts Chancellor and General Counsel andControllers staff to introduce ourselves, describe the review process and protocol, and topresent a Request for Information. The District provided a number of key documents at thismeeting.

2. The HMR team reviewed key documents pertaining to the Inspector General solicitation,including: a) the Districts written professional services procurement policies; b) InspectorGeneral Request for Proposal (drafts and final document); c) 11 proposals received; d)completed proposal scoring sheets; e) interview questions for interviewed bidders; f) staffmemorandum and file documents pertaining to the subject solicitation; g) email and other

communications among and between District staff, Board members and bidders; h) bidderreference check documentation; i) the agreement with Policy Masters, Inc. for contractInspector General services; j) invoices submitted to the District by Policy Masters, Inc. for itsInspector General services; k) bid protests filed by bidders who submitted proposals for theInspector General contract; l) draft District conflict of interest policies; m) District and Officeof Inspector General organization charts; n) Statement of Economic Interests (State ofCalifornia Form 700) for all members of the District Board of Trustees and for District staff


17/48



6

who played a role in the Inspector General solicitation process; o) the State ControllersOffice Audit of Proposition A/AA and Proposition J Bond Expenditures dated August 2011,which included an analysis of the Inspector General selection process; and, p) the Districtreporting responding to the State Controllers audit.

3. Interviews were conducted with members of the Districts Board of Trustees, managers andstaff and external parties who either played an administrative role in the solicitation andselection process or were members of the proposal review and/or interview panels. Allmembers of the two panels from external organizations were also interviewed, except for therepresentatives of one firm who did not agree to be interviewed.

4. A sample of other professional services contract solicitation documents was reviewed toassess common District practices in the preparation of Requests for Proposals,communications with vendors, review and scoring of proposals and contract awards.

5. A draft version of this report was submitted to the City Controllers Office and Districtmanagement for review and comments prior to preparation and transmittal of the final report.

Criteria

Legal Requirements for the Procurement of Professional Services

State law exempts community college districts from awarding special and professional servicecontracts to the lowest responsible bidder.3 This is in contrast to State requirements governingcommunity college district contracts for commodities and repair and maintenance servicesgreater than $50,000 and public project contracts greater than $15,000, all of which must beawarded to the lowest responsible bidder.

4Consistent with these State laws, the Districts

policies and procedures do not require formal bidding or award of professional and specialservice contracts to the lowest responsible bidder, though competitive bidding is encouraged

whenever possible.

District Policies and Procedures for Solicitation of Professional Services

District policies established for the selection of contractors for professional and specializedservices is consistent with legal requirements and is generally adequate.

Existing District polices lay out a multi-step process for the solicitation, evaluation, and selectionof prospective professional services contractors. Specifically, key aspects of the policies andprocedures stipulate that the process for procurement of special and professional services shouldinclude the following:

1. The RFP should include a scope of work that describes the work to be accomplished, thetime frame for completion, any relevant professional or industry standards to be followed,and the deliverables- or work products- to be provided during and at the end of theengagement.

3 Government Code Section 53060.4 Public Contract Code Sections 20650-20660


18/48



7

2. The RFP should include evaluative criteria, which utilizes qualitatively objectivestandards for determining whether a vendor should be selected. The policies state thatmore subjective evaluation criteria also may be applied (such as previous clientssatisfaction as reflected in reference checks). The policies also state that price (hourly rateor overall project pricing) should always be solicited in the RFP and evaluated as part ofthe overall proposal. The polices further state that points or weights are to be assigned to

the various evaluation criteria and published in the RFP, so that both proposers andevaluators know in advance how proposals will be comparatively scored.

3. For professional services anticipated to be equal or exceed the statutory bid threshold, theDistrict Contracts and Purchasing Section should legally advertise for proposals in thesame manner as bids for Large Purchases are advertised. A notice requesting proposalswill be published at least once a week for two weeks in a newspaper of generalcirculation published in the district. A copy of the RFP should also be posted on theLACCD website.

4. The RFP document should generally contain the following information:a. Background on the District, the requesting college (if applicable), and other

information relevant to describing the District as a prospective customer;

b. Explanation of why the specific services are needed;c. Instructions to proposers on how and where to submit their proposals;d. Instructions on what information to include in the proposal and how to organize

the information;

e. Evaluation criteria to be applied and their relative weights;f. Scope of work;g. Price proposal form, on which the proposer enters its identifying information,

pricing, and signature;

h. Terms and conditions of the contract or the proposed contract template to beoffered to the successful proposer, if award is made;

i. Miscellaneous forms and instructions.5. Any material changes to the RFP are to be made only through written addendum

distributed to all who received the original RFP. Copies of the addendum should also beposted on the LACCD website.

6. Whenever it is deemed necessary in order to facilitate the communication of informationbetween the District and prospective proposers, a pre-proposal meeting will be convenedby the District prior to the submittal deadline.


19/48



8

7. Proposals received after the deadline will not be considered. Unlike sealed bids,proposals are typically not opened in a public setting. Instead, they are opened by Districtemployees in private so as not to expose the contents of any proposal to competitors.

8. Proposals will be evaluated in accordance with the criteria and weights published in theRFP. To further the objectivity of the process, a panel of evaluators- including

representatives from other District locations or from outside LACCD altogether- may beconvened to score the offers independently. The proposal receiving the highest score willbe deemed the offer most advantageous to the District and be recommended for contractaward.

9. After the proposals have been initially evaluated, but before the final selection of asupplier, interviews may be conducted with the proposers judged to be in the competitiverange based on the evaluation scores. Competitive range is defined as the field ofproposers which, by virtue of the written proposal, could conceivably be awarded thecontract resulting from the RFP. Information obtained through the interviews may thenbe considered by the proposal evaluators in assigning final scores to the proposals.

a. In addition, negotiations may be conducted with proposers in the competitiverange with the objective of eliciting a more advantageous offer for the District.When conducting negotiations, care must be taken to not (a) divulge the contentsof one proposers offer to any other proposer, and (b) give one proposer acompetitive advantage over another.

10.Any award of a contract to a successful proposer must be authorized or ratified by theBoard of Trustees at a regular public meeting.

11.A copy of the original RFP, including all addenda issued thereto, and originals of allproposals, correspondence and other proposer-generated documents will be retained by

the issuer of the RFP for the duration of the contract award or for a longer period ifrequired by the Districts records retention policy.

The policies also include a clause stating that, at its sole discretion and without cause, theDistrict may, without awarding a contract, reject all proposals, cancel the RFP and either re-issuethe RFP or cancel the entire solicitation. A copy of the Districts written policies and proceduresfor Procurement of Special and Professional Services is attached as an Appendix to this report.


20/48


9

1. Process Used for Procuring Inspector General Services

District policies established for the selection of contractors for professional andspecialized services are consistent with legal requirements and are generally

adequate. However, gaps in District procedures in general and specifically forthe solicitation of Inspector General services include: (1) documentation

requirements; (2) scoring; (3) panelist guidance; and, (4) conflicts of interest that

should be addressed to ensure District adherence to best practices forprofessional services solicitations.

The process used by the Los Angeles Community College District (LACCD) forthe solicitation, selection, and contract award for Inspector General services was

poorly documented and did not consistently follow all District policies and

common practices.

The Inspector General Request for Proposals (RFP), which was prepared by theContracts and Purchasing Manager and the Deputy Chancellor, adequately

reflected the general intentions of the Board of Trustees. However, the RFP didnot include a fully developed listing of professional standards, priorities and

requirements for the Inspector General role. The RFP did not state that

contractors should propose a full-time consultant to serve as Inspector General,

a point that was not clarified until close to the proposal due date, making itunlikely for many bidders to meet this expectation.

The proposal review and scoring process was poorly planned and executed anddid not consistently follow District policy and common practices. The scoring

sheets used for proposal evaluation did not adequately link back to the

specifications in the RFP. Proposal review panelists were not provided withwritten or detailed verbal guidance prior to commencing evaluations. The

proposal review process was poorly documented and individual scores of all

raters were not aggregated to determine each firms score, as District policysuggests.

Interviews with the four finalist bidders, critical to the selection of a firm, werepoorly planned and executed. Interview panelists were not provided with written

guidance about the role of, or District expectations for, the Inspector Generals

Office. District policy states that interview results may be used to determinefinal proposal scores, but final proposal scores were never assigned to the

bidders nor was any formal documentation of the interview panels assessment

prepared. Rather, a consensus was reportedly reached after a group discussion

took place with interview panelists expressing their opinions about the bidders.

LACCD management did not vet potential conflicts of interest among thepanelists or the bidders. Though the independence of the Office of Inspector

General was critical to its success, District management took no steps to vet

conflicts of interest. At the time, there was no District policy in place addressingpotential bidder or bidder evaluation panelist conflicts of interest in the

procurement of professional services.


21/48

Section 1: Process Used for Procurement of Inspector General Services


10

Finding #1: Current District Policies and Procedures for Procurement of

Professional Services Contain Weaknesses

District policies established for the selection of contractors for professional and specializedservices is consistent with legal requirements and is generally adequate. However, our review ofexisting District policies identified gaps in (1) documentation requirements; (2) scoring methods

and protocols; (3) bidder evaluation guidance; and, (4) conflicts of interest. We found thatexisting policies do not specify the extent of documentation necessary for the proposalevaluation or interview process. Our review of this particular solicitation found that LACCDsselection hinged critically on the interview process, which was not well documented.Additionally, the Districts policy does not appear to provide sufficient guidance on the use ofseparate panelists for the proposal evaluation and the interview processes as was done for thisparticular solicitation. Finally, existing policies do not address vetting potential conflicts ofinterest of panelists or proposers.

Our review found that LACCD management took no steps to ensure that the bidder evaluationpanelists did not have conflicts of interest with bidders or that the bidders themselves didnt have

any conflicts of interest that would prevent them from serving as an independent InspectorGeneral for the District.

Finding #2: The RFP Lacked Specificity and Clarity

The process for soliciting and selecting a contractor for Inspector General services varied fromcommon District practice for professional services in that it: (1) was a new function that theDistrict had no prior experience with; (2) was requested by the Board of Trustees rather than adepartment familiar with the services to be provided that would typically prepare the scope ofwork statement, determine the contract duration, and provide a listing of firms for outreach; and,(3) occurred at a time when two key executive management positions were vacant: the LACCD

Chancellor and the Director of Business Services. It was also unusual because, unlike mostDistrict professional services solicitations, the Board of Trustees was heavily involved in theprocess since they had authorized the creation of the Office of Inspector General, the new Officewould be reporting directly to the Board, and there had been significant publicity pertaining todeficiencies in the Districts bond-funded capital expansion program.

The RFP posted for the solicitation of Inspector General services, prepared by the Contracts andPurchasing Manager with oversight from the Deputy Chancellor, appears to have adequatelyreflected the general intentions of the Board of Trustees, but it did not include details on the levelof staffing expected from the contractor, specific deliverables, or a listing of standards, prioritiesor requirements for the Inspector Generals Office.

RFP not Clear on Expectation that Contractor Provide a Full-Time Designated Position to Serveas Inspector General

Prior to the development of an RFP, there was some uncertainty amongst the LACCD Board ofTrustees and management over whether the Office of Inspector General should be an in-houseposition or filled by a contractor, and whether it could be funded through general funds or bond


22/48



11

funds. LACCD management ultimately determined that the function should be filled by acontractor since the bond-funded program would not be permanent. District management alsoconcluded that it would be inappropriate for BuildLACCD, the Districts bond-funded buildingprogram, to assume responsibility for the solicitation and selection of an Inspector General sinceit would be an oversight function of their operations. Under these circumstances, the InterimChancellor put the Deputy Chancellor in charge of the solicitation and selection process for

Inspector General services. However, a job description initially prepared for an in-houseInspector General position was used as the basis for the Scope of Work section of the RFP.Typical Duties are listed in the RFP for the Office of Inspector General (emphasis added), notfor an individual. However, in interviews, members of the interview panel indicated that theywere concerned with bidders that had not proposed providing a full-time individual to serve asthe Inspector General for the District.

The language of the RFP states that the prospective bidders could be an individual, similar to acivil service approach to hiring for a specific position. For instance, the first paragraph of theintroduction of the RFP states in part that, the purpose of this RFP is to solicit offers from aqualified and interested individual or firm or joint venture (emphasis added). Further, the scope

of work section lists the minimum requirements ofindividual or firm (emphasis added). Basedon written questions submitted to the District, a number of bidders appeared to have been unclearabout whether they were supposed to provide a mix of staff to support an Inspector General, fillthe role with a number of individuals, or provide one individual to serve as Inspector General,and support staff.

While LACCD management has asserted that by virtue of the background statement anddescription of the Districts requirements, the RFP communicated the Districts need for a fulltime operation, the need for a full time position is not explicitly stated in the RFP and, as statedunder Finding #3 below, responses from the District to written questions submitted by potentialbidders on this topic confirms the lack of clarity in the RFP on this matter.

RFP Vague on Contractor Deliverables

The critical Scope of Work section of the RFP was prepared by the Deputy Chancellor withsome input from District stakeholders such as BuildLACCD representatives and the Districtsgeneral counsel, but it appears, with limited knowledge of the role and duties or standards andqualifications expected of an inspector general. Specific deliverable requirements were notspecified for the Office of Inspector General in the RFP other than references to the Office beingrequired to conduct audits and investigations and make regular reports to the Board ofTrustees. The required number of audits or investigations to be conducted or reports to be madeto the Board of Trustees was not provided, making it challenging for the bidders to prepareestimates of resources needed and related pricing.

RFP Lacked a Full Listing of Standards, Priorities and Requirements of Inspector General Role

The RFP for Inspector General services was prepared without a full listing of standards,priorities and requirements of the role of the Inspector General. While the RFP Scope of Workemphasized the conduct of investigations and audits, there was no mention of required InspectorGeneral professional standards in areas such as independence and conflicts of interest. The


23/48



12

Association of Inspectors Generals Principles and Standards for Offices of Inspector Generaland the U.S. Government Accountability Offices Government Auditing Standards both containspecific independence and other standards applicable to Inspector General offices and publicagency financial and performance audits, respectively, that would have been valuable to includein the RFP as standards to which the contractor would be expected to adhere.

While the RFP listed current certification as a Certified Internal Auditor (CIA) or Certified FraudExaminer (CFE) as a minimum requirement, there is no mention of whether it would also berequired or desirable for the selected individual or firm to have a current certification as aCertified Inspector General (CIG). Further, there is no reference to the selected individual or firmadhering to the continuing professional education standard for CIGs and whether that would befunded by the District or by the selected bidder.

Finding #3: Pre-Deadline Outreach and Communication with Bidders

Consistent with District Policy and Practices, but Bidders Questions

Indicated RFP Not Clearly Written and District Responses Provided

Inadequate Time for Bidders to Modify their Proposals

The Contracts and Purchasing Manager managed the outreach and solicitation process, consistentwith District policy and practices. The Contracts and Purchasing Manager posted the RFP in theLA Daily Journal, on the LACCD website, and in journals relating to risk management andInspector General activities. The Contracts and Purchasing Manager also directly solicited somefirms, consistent with District policy and practices, but it is unclear which firms were chosen andwhy. While the Districts policies for procurement of special and professional services states thatthe requestor is to provide a list of suggested suppliers to which the RFP should be sent,interviewees have indicated that there was no designated requestor due to the unique nature ofthis solicitation.

Written Questions from Prospective Bidders Indicate that Elements of the RFP were Not Clear

LACCD management did not hold a pre-proposal meeting, nor were they required to do so underDistrict policy. Rather, District management permitted bidders to submit written questions.While the Contracts and Purchasing Manager received written questions and posted answers, theDeputy Chancellor reviewed the questions and prepared formal responses for posting on theDistricts website. An addenda to the RFP was issued May 7, 2010 with responses to thequestions.

While the Contracts and Purchasing Manager has indicated that the number of questions thatwere received were about average, the content of the written questions received by LACCD

management indicate that fundamental aspects of the Inspector General role had not been fullyarticulated in the RFP. Some examples of these questions include:

Does the LACCD currently have a whistleblower hotline or similar process in place? Is the RFP asking for services to establish the office (policies, procedures, governance)

or for actually staffing?


24/48



13

Is the LACCD organization looking to retain an individual IG and one staff person andallow the IG to hire consultants to perform the duties of the office or is the LACCD

organization looking to retain a firm to include an IG and whatever staff is required to

perform the necessary duties of the office?

In its written responses to these and other questions in RFP Addenda 3, the District addressed the

question of whether or not the bidders would be expected to provide a full-time consultant toserve as Inspector General. However, Addenda #3 was issued Friday, May 7, 2010, leaving onlythree full business days before proposals were due (on Thursday, May 13 at 2 p.m.) for bidderswho had not understood the District staffing expectation to arrange for a full-time senior levelconsultant with sufficient experience and qualifications to serve as full-time Inspector General.Given the specialized nature of this position, it was not realistic to expect that firms would beable to secure such a person or would have one on-staff who could make that commitment withsuch short notice. In fact, a number of bidders submitted proposals that did not include a full-time Inspector General.

Bidders had a Short Time to Assemble Teams and Prepare their Proposals

The RFP was issued April 26, 2010 and proposals were due May 13, 2010, slightly less thanthree weeks later. There does not appear to be any District policy or requirement about theamount of time to allow for bidders to prepare a proposal but, given the specialized and variedscope of services to be provided, this turnaround time seems short, since many bidders not onlyhad to prepare the proposal document but had to assemble a team of subcontractors to be able toprovide all required services. As mentioned in Section 1, the average amount of time from RFPposting to the submission of a proposal for a sample of six similar solicitations was 22 days, orfive days longer than allotted for the Inspector General solicitation.

Consistent with the Districts policies and procedures, three addenda were issued to the RFP andsent to all bidders (as acknowledged in their proposals). These addenda covered: (1) Districtprocedures in the event of any material changes to the RFP; (2) an announcement of the date forthe first round of interviews; and (3) a compilation of all questions and answers submitted bypotential bidders. Additionally, consistent with the Districts policies and procedures, the RFPwas advertised for a period of 10 business days.

Finding #4: Proposal Review Process Poorly Planned, Executed, and

Documented

The proposal review and scoring process was poorly planned and executed and did notconsistently follow District policy and common practices. Specifically, the scoring sheets usedfor the proposal evaluation were devised the morning of the evaluation and did not containenough detail to adequately link back to the specifications in the RFP. Further, the proposalreview panelists were not provided with written or detailed verbal guidance about how to applythe evaluation criteria prior to commencing the evaluations. Additionally, the proposal reviewprocess was poorly documented.

The review and evaluation process of all eleven proposals received took place in its entirety onone day, May 14, 2010, the day after the proposals were received from the eleven bidders. The


25/48



14

Deputy Chancellor convened the proposal review process and chose herself and three otherindividuals: (1) the Executive Director of Facilities Planning and Development; (2) the DistrictsBond Counsel; and, (3) the Contracts and Purchasing Manager to score the proposals and choosefinalists to be evaluated by a separate panel through an interview process.

Scoring Sheets were Vague and Not Accompanied by Written Guidance

The scoring sheet used by each of the four panelists, which was devised by a District staff personat the request of the Deputy Chancellor two days before the review, did not contain enough detailto adequately link back to the requirements established in the RFP. Specifically, the scoring sheetwas simply modeled from the broad set of evaluation criteria listed in the RFP as follows:

Qualifications, experience, references (25 points); Staffing, Project Organization (20 points); Work Plan/Technical Approach (20 points); Fee (25 points); Miscellaneous, including exceptions/deviations (10 points)

Rather than expanding on these categories, the scoring sheet only included the broad categoriesas shown in Exhibit 1.1, which illustrates the layout of the sheet. Notably, the scoring sheetprovided 10 points to bidders under the category of Miscellaneous, includingexceptions/deviations. For this category it is not clear if panelists were to provide points fordeviations from the RFP or take away points for deviations from the RFP. In fact, it is unclearwhy this is a category for scoring bidders. At least one panelist that we interviewed indicated thatthey were unclear how to score this section.

In addition to the rows and columns shown in Exhibit 1.1, the actual score sheets had a place atthe bottom for the evaluator to write in the date and their name. Although there was nodesignated area for reviewers to leave notes, some wrote comments in the margin or at thebottom of the page.

The vague nature of the scoring sheet and evaluation criteria appears to have led to differinginterpretations of the criteria. This is illustrated by wide discrepancies in scoring between thereviewers for the same firm under the same category between reviewers. For instance, one firmwas scored a 13 for qualifications by one proposal reviewer while the other reviewers scored thefirm between 20 and 23. In another instance, a different firm was awarded 25 points for fees byone reviewer while the others scored the firm between zero and eight points.

The vague nature of the scoring sheets was compounded by a complete absence of writtendirections and little verbal guidance to reviewers prior to conducting the reviews. No instructionswere given as to how to deal with reviewer conflicts of interest. While one reviewer recusedhimself from scoring one firm, the decision to do so was self-initiated and voluntary.


26/48



15

Exhibit 1.1

Proposal Evaluation Scoring Sheet Illustration

Bidder*

[Pre-assignedProposal #]

Qualifications,

Experience, andReferences (25)

Staffing,

ProjectOrganization

(20)

Work Plan

TechnicalApproach

(20)

Fee

(25)

Miscellaneous,

Exceptions, andDeviations (10)

TOTAL

Points

1

2

3

4

5

6

7

8

9

10

11

Source: LACCD RFP 10-12 File

* Bidders names did not appear on the scoring sheet; instead, each bidder was assigned a number.

Proposal Scores Not Aggregated

The proposal scoring for Inspector General Services was conducted in an informal manner andwas not fully consistent with LACCD policy. Specifically, District policy states that if a panel isconvened to score proposals, the proposal receiving the highest score will be deemed the offermost advantageous to the District and be recommended for contract award. Rather, bidderswere advanced as finalists based on a group discussion- after scoring was complete- in whichpanelists expressed their opinions about the proposals. Further, individual reviewer scores werenot aggregated for the reviewers information prior to this discussion.

Top Bidder Disqualified and Three Bidders Not Fully Scored

Top Bidder Disqualified

The firm that received the highest score on its proposal, as shown in Exhibit 1.2, was notawarded the contract nor chosen as one of the finalists to advance to the interview stage of thesolicitation process. Rather, this bidder, Fu-Gen, Inc., was reportedly disqualified by the proposalreview panel on the basis of being too thinly staffed. This decision was apparently made duringthe group discussion (after scoring had taken place), but was never documented. This decision isapparently at odds with District policy that the firm with the highest score should be deemed the


27/48



16

offer most advantageous to the District. Fu-Gen was not given an opportunity to address thereviewers concerns, though two other finalist firms were allowed to enhance or change theirteams after their proposals were submitted.

Three Bidders Not Fully Scored

Three bidders were not fully scored, resulting in lower overall scores for these firms. Theabsence of full scoring for these firms was not based on District policy or guidance provided bythe Deputy Chancellor. Rather, two reviewers determined on their own to not score a firm asshown in Exhibit 1.2. Specifically, one reviewer recused himself from scoring the firm CapstoneAdvisory Group, LLC because the reviewer had a professional relationship with the firm. Adifferent reviewer did not provide scores at all for two firms that had only bid on thewhistleblower hotline component of the contract. As a result of these individual decisions madeby two of the reviewers, these three firms had lower overall scores, through no fault of their ownsince the reviewer(s) who chose not to score these proposals effectively awarded them a zero.

Exhibit 1.2

Proposal Review Scores for Eleven Bidders Sorted by Total Score

Bidder

Reviewer1

Total Notes#1 #2 #3 #4

1 Fu-Gen 68 68 83 75 294 Disqualified

2 Ernst & Young 81 79 55 70 285 Selected for interview

3 Deloitte 81 77 51 70 279 Selected for interview

4 TCBA2 65 75 61 75 276 Selected for interview

5 Policy Masters 73 71 55 70 269 Selected for interview6 Magallanes 65 72 52 65 254

7 Business Controls 52 58 52 60 222

8 Capstone Advisory 62 50 -- 65 177 One reviewer recused self fromscoring

9 RGL Forensics 52 49 28 40 169

10 Network, Inc. 31 -- 72 35 138 Whistleblower Hotline proposalonly; not scored by one reviewer,resulting in a significant change tooverall score

11 Ethics Point 23 -- 66 35 124 Whistleblower Hotline proposalonly; not scored by one reviewer,resulting in a significant change tooverall score


1 Names of reviewers have been redacted.2 Thompson, Cobb, Bazilio, and Associates


28/48



17

Proposal Review Process Poorly Documented

The proposal review process was poorly documented, which led to some confusion. Specifically,the Contracts and Purchasing Manager created a score sheet key with assigned numbers for eachfirm based on alphabetical order. This score sheet listing did not match up with the otherwiseidentical proposal log, which listed firms in the order that their proposal was received. This

discrepancy is shown in Exhibit 1.3.

The score sheet key and the proposal log looked identical, leading to confusion by the StateControllers Office in their audit pertaining to which Inspector General bidders received whichscores. The State Controllers Office3 apparently was under the impression that the numbersassigned to firms on the proposal log put together by the Contracts and Purchasing Manager onMay 13, 2010 (the deadline for proposal submission) were the same numbers assigned to thebidders on the scoring sheets.

Exhibit 1.3

Firm Listing (Chronological by Time Received) vs. Score Sheet Key (Alphabetical)

Assigned

Number

Bids Received Proposal Log Created by

Contracts & Purchasing Manager on

5/13/2010 Score Sheet

1 Ernst & Young Business Controls

2 Deloitte Services LP Capstone Advisory Group, LLC

3 Magallanes Associates International Deloitte Services LP

4 Fu-Gen Inc. Ernst & Young

5 Policy Masters, Inc. Ethics Point

6 RGL Forensics Fu-Gen

7 Capstone Advisory Group, LLC Magallanes Associates International

8 The Network, Inc. Policy Masters, Inc.

9 Business Controls, Inc. RGL Forensics

10 Thompson, Cobb, Bazilio, & Associates,P.C.

The Network, Inc.

11 Ethics Point Thompson, Cobb, Bazilio, & Associates, P.C.


Reference Checks Conducted on Two Firms, but Basis Not Documented

Following the proposal review on May 14, 2010, the Deputy Chancellor requested staff toconduct reference checks on two of the bidders. Specifically, reference calls were made

3 The State Controllers Office conducted an audit of the LACCD Proposition A/AA and Measure J bondexpenditures. Their report, made public in August 2011, included a finding on the procurement of Inspector GeneralServices.


29/48



18

regarding QIU Accountancy (a partner of Policy Masters, Inc.) and Thompson, Cobb, Bazilio, &Associates, P.C. The Districts file for the Inspector General solicitation does not document whyLACCD management chose to make reference checks only on these two firms and not the othertwo finalists (Deloitte and Ernst and Young).

Finding #5: First Round of Interviews, Critical to Selection of an Inspector

General, was Poorly Planned and Executed

The first round of interviews with the four firms picked as finalists was critical to the selection ofa firm, but was poorly planned and executed. Interview panelists were not provided writtenguidance about the role of Inspector General or District expectations for the selected contractoroutside of the RFP. Further, in contrast to District policy and common practice, standardizedscoring sheets were not provided or used to evaluate the interviewees/presentations. Rather, thepanelists appear to have come to a consensus after a group discussion took place with panelistsexpressing their opinions about the bidders.

The interview panel convened by the Deputy Chancellor was made up of a group of individuals

different from the proposal review panel. The interview panel included two members of theBoard of Trustees, the District Chief Financial Officer, and two outside representatives: (1) anattorney who had some experience representing LACCD and had previous experience as aprosecutor and, (2) a representative from KPMG, a firm which had previously been contractedby the District for audit and other services. Additionally, the Director ofLACCDBuildand theDistricts Bond Counsel sat in on the panel to serve as resources to the panelists, as did theDeputy Chancellor.

Interview Process Poorly Planned

Though interview logistics were adequately administered, the interview process was not well

planned and the interview panel was not well prepared for the interviews. Specifically, theinterview panelists were not provided with written guidance about the role of the InspectorGeneral or District expectations for the new Office, outside of the original RFP. Further, theinterview panelists were not provided with any information about the scores awarded by theproposal review panel. Rather, the Deputy Chancellor simply provided the interview panelistswith a copy of the RFP, copies of the proposals submitted by the four finalists selected by theproposal review panel, and a very brief explanation that another committee reviewed and ratedthe proposals received in response to this RFP. The proposals which are being forwarded to youare of the four most highly rated firms. These materials were provided three days before theinterviews took place. There was no discussion of the disqualification of top-ranked Fu-Gen.

Interview Questions Not Strictly Followed

The Deputy Chancellor, with some input from other District staff, prepared 15 questions, whichwere later narrowed to eight, to ask the bidders. However, the questions were not fully orformally used during the interviews. Rather, the format of the interviews was to allow thebidders to introduce their key members and make a presentation, followed by questions from thepanel that were not restricted to those previously prepared. Interview panelists have indicatedthat all bidders were asked about deficiencies in their team or proposals.


30/48



19

While there is no District policy requiring that standardized questions be asked of all interviewedbidders, doing so is a common practice in public agency professional services solicitations.Giving every bidder an equal opportunity to demonstrate their skills and abilities anddocumenting this process are indicators of an objective, transparent process.

Interview Process Did Not Follow District Policy or Common Practice

LACCD management did not follow District policy or common practices in its conduct of theinterview process. Though not explicitly required, District policy states that if interviews areconducted, the results may be used by the proposal evaluators in assigning final scores to theproposals,. The policy specifically states that,

after the proposals have been initially evaluated, but before selection of a supplier, interviewsmay be conducted with the proposers judged to be in the competitive rangeinformation obtainedthrough the interviews may then be considered by the proposal evaluators in assigning final scoresto the proposals.

LACCD management chose not to follow the policy option of scoring or otherwise methodically

assessing each firms interview results for use in determining final bidder scores. In fact, finalbidder scores, or any other formal final rankings of the bidders, were not assigned by theproposal evaluators. Rather, consistent with the proposal evaluation process used for this RFP, agroup discussion took place after the interviews. The Deputy Chancellor then summarized in awritten report to the incoming Chancellor what she felt was the consensus of this groupdiscussion. Additionally, in contrast with District practice, standardized scoring sheets were notprovided or used to evaluate the presentations made by the four finalist firms.

Further, the process was not consistent with District practices and policy whereby the panelchosen to review proposals typically also determines final bidder scores. In contrast, the DeputyChancellor chose different panelists for the proposal review process and for the interview

process. The District states that this was done at the request of a member of the Board ofTrustees and to ensure an objective process.

While the composition of the interview panel was reasonable for this solicitation, a negativeimpact of using two panels for proposal review and interviews was that the interview panel didnot have information on how the four finalists compared to all other bidders relative to theevaluation criteria in the Request for Proposals. Such information would have provided areas tofocus on in the interviews relative to standardized evaluation criteria. Further, the interviewresults were not provided to the proposal evaluators to determine final scores, as District policysuggests.

Interview Results Not Well Documented

The results of the initial interview process were not well documented. As previously mentioned,there were no scoring sheets or other written summary, provided by or sent to panelists, of theinterview process. Additionally, there is no direct record of bidder responses to the preparedquestions or panelist assessments. The only documentation available of the initial interviewprocess is an emailed written report sent to the incoming Chancellor by the Deputy Chancelloron June 28, 2010, about five weeks after the interviews were held. The email briefly summarized


31/48



20

the results of the interviews. Specifically, the email stated, in part, the selection committeeinterviewed the four firms and identified one firm as being far and away the best prepared for theinterview and with the best working knowledge for the Office of the Inspector General. Thiswritten document was not provided to the interview panelists, though all panelists interviewedagreed with the primary conclusion in the report that Policy Masters, Inc. had performed best inthe interviews.

Reference Checks Conducted on Four Finalist Firms by Outside Contractor

After the first round of interviews on May 20, 2010 the Deputy Chancellor was prepared toquickly wrap up the selection process and provide a recommendation to the Board of Trustees.Two of the four finalist firms, Policy Masters, Inc. and Thomspon, Cobb, Bazilio, and Associateshad been contacted to schedule follow-up interviews and District staff conducted referencechecks on some of the team members for these two bidders. However, after a discussion betweenone of the members of the Board of Trustees who sat on the interview panel and the incomingChancellor about the process, the process was deliberately slowed down, to allow the incomingChancellor to be involved in the selection. The member of the Board of Trustees directed the

Deputy Chancellor to conduct background reference checks on the four finalist firms and theirlead persons.

Background Reference Checks Well Documented, but Had Little Effect on Selection of a

Finalist

In late May and June of 2010 the firm Community College Search Services, which was undercontract with the District for executive search services typically used when hiring a new collegepresident or manager, was requested by the District to conduct background reference checks onthe four finalist firms and their lead persons. The background reference checks on the leadpersons consisted primarily of 14 standard questions asked for each firms lead person, with a

summary of responses received from the references listed by the bidders. The questions weredevised internally by LACCD staff and provided to the search firm. The background referencechecks also included bullet point lists of strengths and weaknesses. However, the referencechecks did not include background checks on any of the firms or lead persons to identify anycriminal records or potential conflicts of interest. The District paid $5,200 for this service.

Although the background reference checks had little impact on the outcome of the selectionprocess, as the references were all generally positive, they were probably the best documentedstep of the Inspector General procurement process. However, it should be noted that the namesof the references were redacted in copies of these reports provided to our review team, so theirauthenticity cannot be verified.

The background reference checks did not result in any bidders being disqualified or anysignificant change in the relative rankings from the initial interviews. Further, CommunityCollege Search Services was not provided specific direction by District management, beyondcalling listed references, to conduct a meaningful reference or background check on the biddingfirms as a whole. The Deputy Chancellor noted in a June 28, 2010 email to the incomingChancellor that the information that Community College Search Services was receiving on thefirms was very limited.


32/48



21

Reference Checks Did Not Include a Detailed Background Investigation

There does not appear to be a District policy on conducting such background checks, but giventhe sensitive nature of the Office of Inspector General and independence standards generallyapplied on inspector generals, the need for the office to be independent of real and perceivedimpairments, checks for conflicts of interest would have been prudent.

Results of Second Round of Interviews Consistent with First Round

On August 4, 2010 the Deputy Chancellor convened a second round of interviews with the samefour finalist firms. The planning, conduct, and results of the second round of interviews weresimilar to the first round. Several members of the panel indicated to us that the goal of the secondround of interviews was to have a more informal discussion with the lead person and essentialstaff from each bidder. The interview panelists were similar to the first round including: the sametwo members of the Board of Trustees and the outside attorney; but two changes were made tothe panel: (1) the Chief Financial Officer was unable to attend the second round of interviewsand (2) KPMG sent a different representative because the original representative was

unavailable. The prepared questions for the second round of interviews included four basic open-ended questions. As with the first round, there was no scoring or other direct documentation ofthe panels findings and, reportedly, not all questions were systematically asked of all bidders.

Two Firms Made Changes to their Proposed Teams after the First Round of Interviews

At least two firms came to the second round of interviews with significant changes to theirproposals. Though the staff changes were accepted by the panel, it is unclear if the firms firstasked for permission from LACCD officials if they could make the changes, were advised to doso by District officials, or if they took it upon themselves to make the changes without consultingLACCD. Specifically, Policy Masters, Inc. added an investigator to join their proposal and Ernstand Young changed their lead person who would serve as the Inspector General. Both firms werechosen as the two finalists to be recommended to the Chancellor. Neither Thompson, Cobb,Bazilio & Associates, PC nor Deloitte Services LP made changes to their proposals after the firstround of interviews, nor were they recommended to the Chancellor as finalists.

Final Selection of an Inspector General Made by the Chancellor

On August 12, 2010, approximately one week after the second round of interviews, the DeputyChancellor sent the Chancellor a memorandum summarizing the interviews and the panelsrecommendations. Specifically, the memo put forward Ernst & Young and Policy Masters, Inc.for further consideration by the Chancellor with a strong stated preference for Policy Masters. Inthe closing paragraph the Deputy Chancellor states:

Members [of the interview panel] expressed the view that their recommendation following the first roundof interviews remained the same. If nothing else, committee members felt their previous recommendationhad been affirmed and expressed the opinion that while they were forwarding the names of two firms,Policy Masters was far and ahead the best firm and that Ernst & Young was being forwarded as a distantsecond choice.


33/48



22

The Chancellor chose to recommend Policy Masters Inc. after interviewing both firms. OnOctober 6, 2010 the Chancellor and Deputy Chancellor formally requested that the Board ofTrustees approve the establishment of a contract with Policy Masters, Inc. for Inspector GeneralServices.

Finding #6: LACCD Did Not Actively Vet Conflicts of Interest

Given the importance of inspector general offices being independent, LACCD did a poor job ofvetting conflicts of interest from the bidder evaluation panelists and the bidders. Districtmanagement took no steps to ensure that panelists had any compromising relationships with anyof the bidders. Similarly, no action was taken to check if any of the bidders had compromisingrelationships with members of the Board of Trustees, District personnel or contractors who theywould have to oversee if they were to be awarded the contract for Inspector General services. Forexample, the president of Policy Masters, Inc. had previously worked for Gateway Engineering,a company under contract with the District for bond-funded construction management services.This information was disclosed in her resume in the Policy Masters proposal, but the District didnot require any statements from this or other bidders disclosing their current or past relationships

with such firms. While there is currently no policy in place for vetting conflicts of interest in theprocurement of specialized and professional services, District management should considerimplementing such policies as a prudent control measure.

Conclusions

The process used by the Los Angeles Community College District (LACCD) for the solicitation,selection, and contract award for Inspector General services was poorly documented and did notconsistently follow all District policies and common practices.

The Request for Proposals (RFP) posted for the solicitation of Inspector General services, which

was prepared by the Districts Contracts and Purchasing Manager and the Deputy Chancellor,appears to adequately reflect the general intentions of the Board of Trustees. However, the RFPdid not contain a listing of priorities or requirements for the Inspector General role, specificdeliverables to be provided by the contractor, the District expectation that the contractor shouldprovide a full-time position to service as Inspector General, or required professional standardswith which the contractor would be expected to comply. The proposal review and scoringprocess was poorly planned, executed, and documented and did not consistently follow Districtpolicy and common practices.

The first round of interviews with the four firms picked as finalists was critical to the selection ofa firm, but was poorly planned, executed, and documented. Interview panel members were not

provided written guidance about the role of Inspector General or District expectations for theselected contractor outside of the RFP. Further, in contrast to District policy and commonpractice, standardized scoring sheets were not provided or used to evaluate theinterviewees/presentations. LACCD management did a poor job of vetting potential conflicts ofinterest among the panelists as well as the bidders. District management took no steps to vetconflicts of interest and there is no District policy in place that addresses conflicts of interest inthe procurement of professional services.


34/48



23

Recommendations

The LACCD Board of Trustees should:

1.1. Direct staff to revise the Districts Business Operations Policy and Procedures, PP-04-08,to include more specific guidance on:

a. The evaluation of proposals, including specific guidance on how to develop a scoresheet linked to standards and requirements specified in the RFP. The revised guidanceshould include a template to further assist District staff.

b. The conduct of interviews, including specific guidance on how to develop aninterview score sheet. The revised guidance should include a template to further assistDistrict staff.

c. How to properly document the solicitation and selection of special and professionalservices.

d. What steps should be taken to ensure that no evaluators or bidders have any conflictsof interest that may impair their ability to serve the District.

Costs and Benefits

The costs associated with these recommendations would include minimal staff time devoted todrafting revisions to the Districts policy on the procurement of special and professional servicesand briefing the Board of Trustees on such changes. The benefits of these recommendationswould include a more transparent and consistent method for the District to procure special andprofessional services. The benefits would also include a reduced likelihood of the Districtprocuring services from individuals or firms with perceived or actual conflicts of interest.


35/48


24

2. Assessment of Bidder Qualifications and Experience

The Request for Proposals (RFP) for Inspector General services stated that theprimary services to be conducted would be audits and investigations of theDistricts bond-funded capital program and operation of a whistleblower

hotline. The RFP required that bidders present their experience and

qualifications in these areas along with demonstrated knowledge of audit andinvestigation techniques, procedures, pertinent laws and regulations and

principles of facility planning and development and construction management.

None of the eleven bidders had experience and qualifications in all areasidentified in the RFP.

Though bidders with more relevant experience and qualifications were generallyranked higher by the proposal review and scoring panel, the Districts process

for assessing qualifications and experience was not methodical or consistent. For

example, the

laccd final report

Documents