Loading...
HomeMy WebLinkAbout2020-07-21 Council Appointed Officers Committee Summary Minutes Council Appointed Officers Committee FINAL MINUTES Page 1 of 25 Special Meeting July 21, 2020 Chair Kniss called the meeting to order at 10:06 A.M. via Virtual Teleconference. Participating Remotely: Cormack, Filseth, Kniss Oral Communications None. Agenda Items 1. Discussion and Direction on the Evaluation of Submittals for the Internal Auditing Services Request for Proposals. Chair Kniss began by saying that the Evaluation Team did an Independent Review of the proposals and Staff was going to make a recommendation to start this part of the meeting. Council Member Filseth thanked everyone for coming together in the middle of summer. He wanted to finalize the structure as to when candidate firms were going to be interviewed. The Evaluation Committee reviewed all the proposal responses and had some recommendations as to which firms were going to move forward. Next, the Council Appointed Officer’s Committee (Committee) was going to interview the firms, decide on finalists and then the finalists were going to be interviewed by the full Council. Today the Committee needed to decide: 1) which firms they were going to interview (the Evaluation Committee had a recommendation, but it was up for discussion; 2) finalize the structure of the interviews, such as questions and timing; and 3) discuss how a decision was going to be made amongst the finalists. A key piece of the interviews was timing and what questions were going to be asked. Chair Kniss opened the meeting up for public comment. FINAL MINUTES Page 2 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Nelly Baumb, Deputy City Clerk remarked that there were two members of the public present. Chair Kniss inquired if now was a good time for public comment. Council Member Filseth said yes. Ms. Baumb declared that no one wished to speak. Council Member Filseth commented that this was a public meeting and at this point in time, it was recommended that the actual names of the firms applying for the Internal Auditor position were not to be used, the alpha- numeric reference to each firm was to be used instead. It was conceivable that some of the firms were able to view this public meeting and anonymity needed to be considered. Chair Kniss confirmed the firms were going to be referred to as: Firm A, B etc. Council Member Filseth confirmed. Pete Gonda, Management Partners declared he was coordinating the Request for Proposal (RFP) process for the City of Palo Alto. He wanted to give the Committee a recap of the Staff Report and then he wanted to discuss the rankings of the firms. On July 10, 2020, there were six proposals received by the City for Internal Auditing Services. This was a re-issued RFP, following the rejection by the City Council of the six proposals that were initially submitted under a process that was conducted in April and May, 2020. An RFP process was established by the Committee on April 16, 2020, followed by Council on May 4, 2020. The process called for an initial review by an Evaluation Team if more than three proposals were received, which was what happened. The Evaluation Team was established, consisting of Council Member Filseth, as the Ad Hoc RFP Sub-Committee Member, former Council Member and Mayor, Larry Klein, as the Chair’s appointed Evaluation Team Member and Greg Larson, Partner with Management Partners. Since the initial review was complete, the Committee needed to provide direction for further evaluation. The process today was going to be anonymous, so the firms were going to be referred to as Firm A, B etc. In addition, he suggested that no specific detail of any firm be mentioned, as that indicated who the proposing firm was; such information included home name, office location, home office location or anything that identified the specific geographic region. FINAL MINUTES Page 3 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss mentioned that there were those at the meeting that were just listening; she requested a little guidance. Mr. Gonda agreed. He remarked that the left most column of the information he was displaying showed the proposal reference by alphabetical designation, the second column showed the average score by the rankings of each evaluator, which were shown on the right hand portion, the grey column in the middle represented the overall average ranking scores and the right three columns showed the Evaluators, listed as Evaluator One, Two and Three, including how each of the Evaluator’s ranked each of the proposals. He noted Proposer E received an average score of 93 and had an overall ranking of one, followed by Proposer C, which received an average score of 92 and had an overall ranking of two, followed by Proposer A in third position, with received an average score of 90, followed by Proposer D, with a score of 84, then Proposer B, with an average score of 79 in position five and Proposer F, with an average score of 71, with an overall ranking of sixth. The Evaluation Team met to debrief on the scoring and recommended that the top four proposers advance to the interview process, which were proposers E, C, A and D. He questioned if the Committee was going back to public comment or deliberation. Chair Kniss said no. Mr. Gonda suggested there be a discussion of the overall rankings and then a decision about who was going to be interviewed. Chair Kniss mentioned that due to her calling in, she was not able to see the table referenced; she inquired where it was in the Packet. Mr. Gonda understood and wanted to get that information to the Chair. Chair Kniss confirmed it was called “Evaluation Team Scores and Rankings.” Mr. Gonda said yes. There was an inadvertent omission with the Staff Report in Attachment A, which represented an overview of the Evaluation Process. At some point, the Committee was going to discuss that, which was provided in the Staff Memo; it had the Process Steps in greater detail. Greg Larson, Management Partners said that he followed up with the City Clerk and confirmed it was posted on the City website. FINAL MINUTES Page 4 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Beth Minor, City Clerk relayed that she sent the PowerPoint via email. Chair Kniss reiterated that she was not on Zoom, but her email was working. Council Member Cormack outlined that based on her understanding, what was needed to discuss at this meeting were questions around the scores. She requested confirmation that the ranking was ordinal, and the score was cardinal. Mr. Gonda said yes. Council Member Cormack stated that she was going to focus on the score for the moment. Regarding the nine different scores, she wanted to know if there were any major differences on number four, which was cost. She questioned if that was a factor and wanted to know how the Committee was going to know the different components because there were nine, she thought were particularly important. Mr. Gonda said there was a budgeted number that was given to the proposers, and more or less, the costs came in close to the number, which was $750,000 for the first two years. The cost was considered with regard to the evaluation parameters and the firms were not necessarily affected by price but how effective their service array was. Mr. Larson added that the cost factor was given 15 points and that was only part of two categories that had the greater number of potential points allocated to it. While he was not going to divulge individual evaluator ratings by category of those points, there was focus in the debrief on the cost factors and how those were applied. He spoke for all the evaluators when he said cost was a significant part of the discussion, in terms of what services were offered for the cost given. Council Member Filseth added that while all firms came close to the total number that was specified, there were significant differences in the number of hours, and to a lesser extent, where the hours were put in each task. There were significant differences in what services were proposed for the $750,000. The largest number of hours was D, the smallest was C and the rest were clustered in the middle. Council Member Cormack understood that the dimension of cost was captured. Something she just noticed was the Project Management Tool as said there were different tools used. She wanted to know if the Project FINAL MINUTES Page 5 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Management Tool was captured in the Staff Report under Quality and Effectiveness Solutions, and if that was where the evaluators considered that portion. Council Member Filseth did not think so and did not recall talking about that. Council Member Cormack thought this was a tool that City Staff were going to need. She suggested putting this on a list to think about next time and reiterated that the interaction tool was important for the Committee to think about. She inquired if Specialty Auditor’s that were within the firm was captured in the breakdown in the Staff Report under “Experience.” For example, Palo Alto has their own airport, they run their own website; she questioned whether “Specialty Auditor’s” were captured in the Staff Report under 7.3. Larry Klein, Evaluation Team Member answered that they did not evaluate the various tools because they were limited in their analysis to the four corners of the proposal. They were instructed, for example, to not do supplementary research, such as “Google” on any of the firms listed; the same thing applied to the tools being better or worse than the other tools used. Council Member Cormack understood. Mr. Larson noted that was correctly captured in the “Experience” criteria of each firm’s individual ability to correctly deliver the tools needed. Mr. Gonda noted there was going to be an opportunity during the interview process to clarify items like this. Council Member Cormack inquired if it was fair to say there was needed expertise under California State Law captured. Council Member Filseth did not think anyone of the three spent time trying to parse through these types of inquiries. Their assumption was they were going to comply with State Laws. Council Member Cormack understood. FINAL MINUTES Page 6 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Mr. Larson thought that Council Member Cormack’s inquiry was captured under “Experience” because it implied the firms had to comply with California State Laws and regulations. Mr. Klein added that all applicants had experience with California clients. They had to comply or else they would not have California clients. Council Member Cormack inquired of the Committee’s recommendation of four applicants. Looking at the cardinal numbers, there was a big difference between a 90 and 84 percent rating. It was conceivable that the top three could be interviewed. She requested discussion of the Committee interviewing the top three versus the top four. Chair Kniss commented that was her question as well. Council Member Filseth remarked that the chart really captured the two splits. There was a split between A and D, and another between C and A. He noted that both A and C (or E and C) ended up in everyone’s top three choice, but A and D had one person that gave them a very high ranking, and everyone else ranked them middle to low; there was a gap there. The two points the Evaluation Team discussed were the amount of time to spend with the firm and the range of perspectives because some of the firms were very similar and some very different. Some firms were national, and some were tiny specialty firms. The Evaluation Team thought there was value in the diversity of the firms. Mr. Gonda thought that was a good summary. Mr. Larson suggested that was the driving factor for the Evaluation Team to recommend four firms. It strengthened the pool of candidates the Committee and ultimately the Council had to choose from. Regarding Firms A and D, there was one evaluator that had them ranked first, so that was a differentiating characteristic for those two firms. Council Member Cormack remarked that it was hard to tell the difference between the rankings, it might be miniscule. She was pushing back because if the rubric captured everything, the Committee should be confident interviewing two to three, as opposed to three or four. On the other hand, if the Evaluation Team had other things to contemplate or if this was the first step in the process, then she was willing to consider what the top four candidates were able to provide. FINAL MINUTES Page 7 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Mr. Klein expressed that there should not be too much emphasis on the average or numerical scores because the Evaluation Team varied in hard to easy ratings. Council Member Filseth agreed. Mr. Klein explained that the average between each evaluator had a difference of about five to six points. Council Member Cormack thought that was helpful. Mr. Gonda commented that in his many years of experience, it was rare that the rankings stay the same between the written evaluation period and the interview process. This was why the interview process was so important; to be able to see and hear the team almost always caused the rankings to change. Council Member Cormack said those were all the questions she had at this time and assumed once the Chair was done with her questions, the Committee would discuss, timeline, interviews etc. Chair Kniss had a different line of questioning but first wanted to confirm who attended the meeting (all names were called off.) Chair Kniss wanted confirmation that Larry Klein was in attendance in an official compacity. Ms. Minor confirmed that was correct and added that the Assistant City Manager was also in attendance. Chair Kniss thought the explanation about why four candidates were recommended to interview was helpful but what she thought was interesting was there were some hard and some not so hard graders; she wondered how that would inform the Committee going forward. Mr. Larson answered that was going to be part of the judgement process. Council Member Cormack related that this was the first hurdle. Once candidates were interviewed, the decision was made based off of the interview process, not the numbers from the Evaluation Team. FINAL MINUTES Page 8 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Mr. Gonda said yes. The difference between a hard and a soft grader was it was a way to approach the evaluation; they tried to be as objective as possible, sticking to the criteria, applying the judgement calls only as they related to the established standards. Invariably, evaluators approached things differently. This was done independently, there was no conversation between evaluators, and each brought their own perspective. Chair Kniss understood, the repeated that the interview was really the indicator, even though there was a lot of time was spent on the applications. Mr. Klein thought the interview was crucial, but the firm’s applications could not be ignored. There was information in the application that needed to be considered and should be considered. For example, where a firm was located, how many people they had on their team and the quality of the person they chose as Auditor. These things were most likely not going to be discussed in the interviews, but the information was there, and the nomination of City Auditor was a combination of those things. Regarding the grading, there were categories, like the cost, which was worth 15 points in the evaluation category. He commented that the easy graders might have designated a firm that they thought was best and would have given a score of 15/15, whereas the hard grader might have agreed but only given 13/15, just because they were tough. Mr. Larson added that when looking at the range of low and high scores implemented by each evaluator, the range was 17 as the most compact grader, and 32 where the scores that were broader in range. The difference in range in the evaluations showed the different mind-sets of the evaluators. While the numerical scale was very helpful, a one-point separation by the 17 range evaluator meant something very different from the one-point separation of a 32 range evaluator. This was why both data points were presented because the scores and the ranking both provided intelligence. Ultimately, the Committee’s decision was based off a ranking process, which was why these processes focused on ranking; what was underneath was the score needed to be recognized. Chair Kniss inquired if the potential City Auditor was going to be on-site. Council Member Filseth understood none of them were to be on-site. Chair Kniss repeated that none were to be on-site and asked if that was always going to be the case. FINAL MINUTES Page 9 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth did not know for sure. There was nothing in any of the proposals stating any of the firms were going to be spending a designated amount of time on-site. They did notice a designated location the applicant worked out of; some applicants were in-State and some out of State. Mr. Larson recalled the RFP did specify the applicant to be on-site for any work that needed to be done on-site. He gave an example of a pre-COVID- 19 scenario where office space was provided “as needed,” but there was no RFP requirement for a permanent full-time assignment to the City location. This was due to the prospect of hiring a contractor versus an employer. Two points were: the RFP did specify an expectation but there was also widespread recognition that the current COVID-19 dynamic changed the on- site dynamic. Mr. Gonda recalled there was some question and answer around this topic in the Agenda responses that were provided to the Committee. There were specific questions from perspective proposers about being on-site and what the expectation was, which was already considered and answered by the City. Chair Kniss questioned whether the prospective City Auditor was going to understand the current situation in the Auditor’s Office. Mr. Larson said yes. Prospective firms were provided with links to prior Council and Committee discussion regarding the change of the City Auditor function. Some candidates referenced this in their proposals. Council Member Filseth thought about this with regard to the questions that were to be asked of the candidates in the interview and suggested this be discussed in the interviews. He asked himself what everyone might be most worried about and one answer was the shift from an internal to a private model. Another worry was an adequate level of intimacy with the organization. He suggested asking firms in the interview about how they proposed to handle this shift. Chair Kniss stated she and Council Member Filseth knew this subject intimately and knew how complex the department was, not just with regard to workload, but with regard to interaction and socialization difficulties. She did not see any information about the prospective candidate having to meet with the City Council and Mayor on a regular basis but supposed this could be asked at the interview. FINAL MINUTES Page 10 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth answered that this topic came up as everyone’s biggest concern, in terms of insource versus outsource. It was fair to inquire how the prospective candidate was going to help the City in this regard. Chair Kniss mentioned scenarios, such as Firm D providing more hours and C providing less; she wondered if this was because Firm D was closer to City Hall and was able to provide more hours. Council Member Filseth remarked that each proposer provided a list of how many hours to bill according to which kinds engagements their fee ranges were per hour, and Firm D’s hours were significantly less than Firm C’s. Chair Kniss thought that was interesting. Council Member Filseth continued that the range per hour, depending on whether the City was billing for Associate Staff or the Senior Partners, ranged from $100-$350 per hour; Firm D topped out at $250 per hour. Chair Kniss expressed surprise. Council Member Filseth speculated that Firm D really wanted their business. Mr. Klein remarked that it was difficult to discuss Firm D without revealing their identity. He was not a fan of Firm D because if a person were to just look at hourly rates, it may not get a person to what was necessarily the right answer. A lower hourly rate did not necessarily mean a person was going to receive more services for a cost. He thought this could mean that a firm might be more efficient than a person who was charging more per hour. He worried about Firm D’s depth and abilities. Other members of the Evaluation Team took a different view and gave Firm D more points for having lower hourly rates. He illustrated that there were some that were good at picking apples and there were some that just threw them all in the same basket; he gave more credit to firms that displayed greater expertise, which might be reflected in a higher hourly rate. Others on the Evaluation Team took a different view, as was seen in their rankings. Mr. Larson explained this was why there were multi-parties doing evaluations because they brought different factors to each review. Chair Kniss understood that all applicants had experience with California clients. FINAL MINUTES Page 11 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth said yes. Chair Kniss wanted confirmation that was not a point to differentiate the applicants with. Council Member Filseth did not think so. Mr. Gonda remarked that volume, range and comparability to the City of Palo Alto varied. He repeated that all had California Municipal experience. Chair Kniss really wanted to see someone who worked with a City like Palo Alto. It was not a great differentiator but thought it gave insight as to how they operated in the City. Council Member Filseth relayed that two were located in California. The Evaluation Team did weigh that, not in the cost category but in their ability to deliver and execute. Chair Kniss wanted to know the location of the firms but wondered if she was getting ahead of things. Council Member Filseth thought that was getting ahead of things. Mr. Gonda agreed. It was getting ahead a little, especially in light of how working conditions were changing. Council Member Filseth said two of the six, not two of the four were in California. Council Member Cormack clarified it was not said which of the two were in California. Mr. Gonda questioned if the Committee was ready to look at interview structure. Ms. Minor confirmed she sent the Committee the PowerPoint. Mr. Gonda repeated that the Committee was discussing the potential structure for the interview process. He recommended at least three of the applicants’ key team members be present at the interview, and there should be no more than five that attend, with no sales or marketing folks in FINAL MINUTES Page 12 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 attendance. He suggested the proposed Auditor lead the presentation for a couple of reasons, to demonstrate leadership and to show how he or she utilized the team members. Regarding the format, he recommended questions be prepared in advance, to provide some structure around the process. Next, he suggested each Committee Member be able to ask one question that was not prepared in advance, which addressed some of the important concerns and priorities to the individual interviewees. The process was expected to take about 30 minutes. Regarding some of the information that was to be given in advance to the firms was to ask that they speak about their key differentiators between internal Auditing firms and how these relate or were relevant to the cities Internal Audit program. A second potential question was to indicate any drivers that were to be considered the most important that would bring a cost-effective work product for the City and indicate any lessons learned about maximizing value to the City. A third potential question was how they could ensure accountability when reporting responsibility to the City Council while maintaining a collaborative relationship with the City Administration, including examples on how that was going to be successfully accomplished. This was followed by questions that were not revealed in advance. Council Member Filseth brought up some concerns of going from an internal organization to an external organization and asked how the City was going to make sure the firms’ relationships and knowledge of our organization was addressed; he suggested this be asked at the interview. An example of a concerning question was “tell us why you are the best fit for Palo Alto,” which made sense, but he imagined the reply was that they hire the best people and that was the best thing, which he thought was a predicted generic answer. Replacing Mr. Gonda’s question with a question like “how does the firm plan on making the in-source/out-source environment work, including speaking to organizational knowledge and relationships,” was a big topic of conversation. When the Council discussed structure and how the Auditor’s Office was going to be organized, that was a question that was brought up. Mr. Larson took notes of possible interview questions and said he had four. Council Member Cormack mentioned the system tools used and how the firm was to interact with them. There was also the question of the transition from internal to external Auditor raised by Council Member Filseth. Another question was the onsite staffing, which could be included with the transition from internal to external Auditor. Lastly, there was a question of service delivery to jurisdictions comparable to Palo Alto. FINAL MINUTES Page 13 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth recalled there was another question regarding access to domain specialists. Mr. Larson thought that was a question for him and his team. Council Member Cormack said Council Member Filseth was correct. She noted her question was “was it considered,” and “did that need to be considered.” Mr. Larson sent the PowerPoint to the City Clerk and the Chair. Council Member Cormack did not think question 1 was helpful because it asked the firm to compare them against other people and she was more interested in what they were able to do for Palo Alto then how they were different from other applicants. She thought question 2 was important because question 2A was the reason why they were having this discussion. Question 2B was important too, much like Council Member Filseth’s example on hiring the best people. There were a lot of lessons learned; she was interested in question 2B and thought the applicants might give different answers. Council Member Filseth agreed and was considering question 2A as well. Council Member Cormack repeated that question 2B was important and 2A was a substantive to 2B. The transition question felt like it needed to be separate. There was a lot in question 3 that the Committee talked about, depending on whoever got the job, but the transition was the first step. She expected many firms to have experience making a transition like this, which was where she wanted to extract the following questions: 1) what were one to two things that went well; 2) what was one thing that did not go well; and 3) how did they plan on compensating for that. Collectively she expected some interesting feedback, especially if the presentation focused on the experience. She was not sure how Option B was going to be handled in a Brown Act compliant way because the three Committee members were going to have to come up with questions. She asked for guidance to avoid duplication and to ensure the process was helpful. Mr. Larson remarked that the Evaluation Team did discuss this, which was why he took notes about what he thought Committee members wanted to ask at the interviews. There were two sets of interviews, the Committee interviews and the follow-up round with the Council and the finalists. His recommendation was to either defer to the Evaluation Team or the Ad Hoc FINAL MINUTES Page 14 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Committee, which consisted of Council Member Filseth, and to approve the final questions to be used at the initial interview. He thought the process could include Mr. Larson and Council Member’s Cormack and Kniss. He also suggested incorporating a question which was not revealed to the firm ahead of time, so the panel had a chance to ask one question they thought was on point. Council Member Cormack thought that was helpful. She trusted the Ad Hoc Committee and the Evaluation Team to come up with something appropriate. Next, she inquired about the final decision process and wanted to know where it was appropriate for the full Council to decide what guidelines were in store for that. Mr. Larson said the process was discussed with the Committee last April 2020 and was now taking more shape. The Committee was going to meet next week to interview four firms, presumably. After which, the Committee was going to decide which City Auditor candidates were going to be interviewed by the City Council. The role of the Council in the Closed Session was a personnel appointment, even though it was a contracted service. He emphasized that solely the City Auditor candidate was to be interviewed by the City Council, the second interview did not include the whole firm. This was why Mr. Gonda specified that when the firms interviewed before the Committee, they give a presentation. This ensured the City Auditor candidate played the lead role in the presentation at the first round of interviews. This was important because then the Committee was able to incorporate their feedback into their recommendation to the full Council. He asked the City Clerk to poll the Council regarding availability during the Council Break for Wednesday, July 29, 2020. Council Member Cormack inquired if that date was secured. Ms. Minor said no, she was still waiting for the outcome. Council Member Cormack inquired if Mr. Larson was to compile the interview questions to the Council or maybe it was a more free-form process. Mr. Larson answered it was usually both. His experience was there was a two-phased process, a rubric was used for the first round of interviews, and then there was a more casual discussion. Since the process was different at the Council interviews, depending on how many candidates there were at the second round, he suggested one period of structured questions and then an open period, all taking place in a Closed Session atmosphere. He was FINAL MINUTES Page 15 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 worried that three or four candidates interviewed before the full Council could cause the meeting to be lengthy. Council Member Cormack supported a structured atmosphere at the Council interviews (by recommendation of the Committee and Management Partners.) Chair Kniss requested differentiation between the attendees at the Committee interviews and the Council interviews because the report said three and no more than five attendees from each firm were able to attend. Mr. Larson answered that only the person selected to be the City Auditor from each firm was to be interviewed by the full Council. Most firms proposed an array of resources that they wanted to use for the job, which ranged from quality control, financial staff and IT expertise. He assumed the firms were going to have at least two to three other team members in the room during the interview with the Committee, but they did not want a group 15 people either. The limits were variable, but he also did not recommend a firm be singly represented either, so he suggested a minimum of two for the purpose of internal accountability. It was good to have minimal accountability with the proposers by having the City Auditor and at least one other member of their team present at Committee interviews, but if the Committee wanted to reduce that number, that was fine. He did not think there was a need to go above five because a 30-minute interview did not have the compacity for all of those people to participate. Chair Kniss thought three people was good and five interfered with the interview. She requested confirmation that the applicants had 15 minutes and then they were going to have another 15 minutes. Mr. Larson said yes. The first 15 minutes were going to be structured and known in advance, the second 15 minute period was going to be filled by questions from the Committee that the firms were not going to know in advance. Chair Kniss confirmed the Committee was going to interview the top four candidates. Mr. Larson replied that was his recommendation. Chair Kniss counted a total of two hours and asked if there was time woven in to confer after each interview. FINAL MINUTES Page 16 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Mr. Larson said no, the interviews were going to be back to back with Council direction and discussion afterward. He explained a tool that he used with four candidates to interview: after each interview, the Committee was able to make a numerical ranking. For example, the first candidate was assigned number 1 and ranked number 1, candidate two would be called number 2 but as the interviews proceeded, the candidates were put in order according to their ranking after each individual interview. This process freshened the Council’s discussion and minimized the recency effect of the last interview. This was a good tool for four candidates but with two or three, it was not necessary. It was helpful for the Committee because they could take notes and explain why they put one candidate above another. There was a five minute transition period between candidates. Council Member Filseth liked the approach. Chair Kniss reiterated that the interviews were not going to be discussed until they were all complete. Mr. Larson said right. Chair Kniss inquired how much time was allotted after the interviews. Mr. Larson replied as much time as the Committee wanted. His assumption was, allowing for natural transitions between interviews, he assumed the interviews themselves were going to take about two hours and fifteen minutes. The Committee was not going to make a final decision, they were just going to make a recommendation of who to interview, which he thought would take 45 minutes; he suspected the entire process would take three to three and a half hours. Chair Kniss agreed. She felt the meeting was going to be intense. Mr. Larson said it was a unique role for a Council or Committee to play. Council Member Filseth recalled a discussion on how to structure the Council interviews. He inquired if that conversation was going to take place when they discussed the recommendation of the finalists. Mr. Larson thought any further direction from the Committee was helpful. The interviews were going to be lengthy, so they were able to start earlier, depending on schedules. FINAL MINUTES Page 17 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss was okay with 9:00 A.M. Council Member Cormack was flexible; if the meeting was going to be three and a half hours, it was better to start sooner rather than later. Chair Kniss preferred a 9:00 A.M. start time. Council Member Filseth agreed. Chair Kniss clarified they would allot three and a half hours, but they may not need it. Mr. Larson realized the Council had outside lives. Council Member Cormack remarked this meeting was supposed to end at 11:00 A.M. Chair Kniss alluded to Council Member Cormack’s comment and suggested tidying the schedule. Council Member Filseth said there were suggestions on the questions, such as Council Member Cormack’s remark that question one was not that important, which he thought might make sense. Also, he wondered if the applicants were going to do a short overview of their firm. Mr. Larson answered that the differentiators were their one opportunity to discuss their firm. They were not going to get a five minute self-introduction because the Committee will have read the proposals by then. If the time allotted was longer, it was common to give the applicants time to introduce themselves, however the Evaluation Team’s discussion was to use the differentiator in place of the introduction. He added that dropping the introduction was good because they had already discussed the applicants to determine who was going to move forward with consideration. Council Member Cormack remarked that the differentiator part was hanging her up because it called for a comparison, rather than to bring out the parts that were relevant. She suggested calling a section “strength” and noting that because she thought whatever was listed first was the most important. The first item on the list of questions was “how is your firm better than the others,” but she really wanted to know why they were best for Palo Alto. FINAL MINUTES Page 18 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth clarified Council Member Cormack was suggesting making question number two, question number one. Council Member Cormack thought if the Committee could give the applicants a little space, it was good to give them a moment to introduce themselves. She suggested that context be “what was special about your firm and how was that relevant to what the City was trying to do, exactly;” it could be a component of question number two. Council Member Filseth said that was a merge of questions one and two. Chair Kniss agreed and wanted the applicants to have a chance to introduce themselves. Mr. Larson replied that based on the feedback, he recommended question one be tightened up. He was able to put a clause in the second half regarding how the firm was relevant to the City of Palo Alto, but that could be moved to be first in order, for example: “Regarding the engagement with the City of Palo Alto, what were the key strengths you would bring.” Wording the question like this gave the applicant a little leeway, while not turning the discussion into a 15 minute treatise. Council Member Filseth felt familiar with the discussion, and was not sure that he needed his special question in Section B. If it made sense to surrender that time to allow more time in Section A, it might be a better use of time. Council Member Cormack liked that idea. Chair Kniss agreed and wanted to move forward with that plan. Council Member Filseth was much more familiar with the information; she inquired if that could be accommodated. Mr. Larson said yes. Council Member Filseth asked if that meant the Committee kept question one and two. Mr. Larson said question one was going to be tweaked in the way he suggested, as long as they were okay with that. FINAL MINUTES Page 19 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Filseth said yes. Mr. Larson felt like part of the discussion made him wonder if the question needed to be tightened up to emphasize Section 2B, rather than 2A. The dynamic was the same for an outsourced or Internal Auditor; they reported to the Council but how were they supposed to work with the Administration because not working with the Administration made them ineffective. Chair Kniss felt that was so important. If nothing else was emphasized, that point was really an important one. Mr. Larson suggested adding Question 4 to Section A or they could be given more time to address Questions 1, 2 and 3. The question that received the most discussion was the question regarding transition from an in-source to an out-source, lessons learned and what has not worked. He saw the transition question as being crafted as a new Question A. He considered giving the applicants five minutes per question or he was able to give them 15-20 minutes as a block and let them allocate their own time between the questions. Chair Kniss thought that was a good idea. Council Member Cormack seconded the agreement. She thought it was possible for a person to get through a question succinctly, and then the discussion could move on. Council Member Filseth remarked that conversely, if the firm spent 20 minutes on a generic sales pitch, that gave the Committee insight as well. Chair Kniss reiterated the process and said there was a determined length of time, which was 30 minutes. Mr. Larson concurred and said the firms would know that. Mr. Gonda repeated that the firms were going to know that. It was very clear to them that they will need to manage their time. There was a need to gatekeep time because there was always a tendency to use more. Chair Kniss mentioned that time was over at this meeting as well. Council Member Cormack was okay with the time today. FINAL MINUTES Page 20 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss inquired if there were any other aspects that needed to be discussed, or any other questions that needed to be moved around. Mr. Larson questioned whether there was concurrence about adding a fourth question to the presentation on the transition; it did not make sense to leave it as a surprise question. Council Member Cormack thought that was a good place to incorporate the Project Management Schedule and tool, like an on-boarding of how the City was to work with the firm. Mr. Gonda said one thing that was not covered was the public nature of the interview process. After the interview, he was going to have to ask each successive firm whether they were privy to any information from the prior interview, which was grounds for disqualification. He planned on explaining the process to the firms in the correspondence that was going to be sent out; they should not be watching the other interviews. This applied to the Committee interview process and the Closed Session. Council Member Cormack repeated Staff was going to tell applicants how the process worked and then there was going to be a check to ensure they followed the rules. Mr. Gonda said yes. The process was going to be made very clear to them in the correspondence, letting them know Staff was going to double check as well. MOTION: Council Member Cormack moved, seconded by Council Member Filseth to interview the top 4 candidates: A, C, D, and E on July 28th at 9:00 A.M. with the CAO Committee, and that the questions discussed today are used, with 20 minutes for applicants and 10 minutes for Council Members. Ms. Minor wanted confirmation on who exactly was going to be attending this meeting. Council Member Cormack said the Committee. Ms. Minor confirmed the meeting was going to be on July 29, 2020 at 9:00 A.M. Mr. Larson corrected July 28, 2020. FINAL MINUTES Page 21 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Ms. Minor confirmed. Mr. Larson clarified it was a public meeting. Council Member Cormack said yes. Ms. Minor asked if next week’s meeting was public. Council Member Cormack confirmed it was going to be a public meeting; she anticipated the meeting would take about four hours. Separate from that, they were looking for a Closed Session on the Wednesday after Council. Ms. Minor was going to doodle and check with Council Members. There were three or four Council Members that may not be able to attend if the meeting was before 5:00 P.M. Council Member Cormack asked for Staff and the Chair to work with the Clerk. Chair Kniss asked for the Motion to be read back to the Committee. Ms. Baumb read the Motion aloud. Council Member Cormack wanted “recommended to the City Council” deleted from the Motion. Chair Kniss agreed. This was the Motion. MOTION PASSED: 3-0 Chair Kniss questioned if there was any other business that needed to be discussed. Mr. Larson understood that implicit in the Motion, that the Committee was moving forward with the top four rated firms for interview, coming out of the Evaluation Team, which were Firms A, C, D & E. Second, the Clerk was going to commence polling the Council for a Closed Session on Wednesday July 29, 2020 to potentially follow-up on whatever Action the Committee was to take next Tuesday after the interviews. FINAL MINUTES Page 22 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss confirmed Tuesday, July 28, 2020 at 9:00 A.M. but inquired of the second follow-up meeting. Ms. Minor stated it was the same day, in the afternoon. Mr. Larson clarified it was the next day, Wednesday, July 29, 2020. Ms. Minor confirmed. Mr. Larson’s plan was to get the City Auditor resumes to the Council after the meeting on Tuesday, so the Council Members had a day to review. Chair Kniss inquired if that was sufficient time. Mr. Larson answered that it was not locked in stone, although it was up to Committee direction. The source of the back-to-back schedule was due to the speedy timeline mentioned in the spring. If the Committee wanted to move the interviews to Thursday, that worked as well, depending on Council availability. He suggested the Clerk poll for both Tuesday and Wednesday. Chair Kniss liked the idea of reserving July 29th and 30th, 2020. She reminded the Committee that these meetings needed to happen after 5:00 P.M. Ms. Minor clarified that the Council meeting needed to be after 5:00 P.M. because of the four Council Members that work. She did not have Clerk Staff support on Thursday’s because of the Council Packet. Chair Kniss clarified that the meeting needed to be on Wednesday. She inquired if anyone was on vacation. Ms. Minor mentioned that Vice Mayor DuBois was out of town Saturday and came back the following Friday; he was not able to attend. Chair Kniss hoped these dates were not set in stone. Council Member Cormack remarked that the Budget discussions were held during the day; it concerned her if the meetings needed to be held in the evening. FINAL MINUTES Page 23 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss agreed with Council Member Cormack. Committee Members were available during the day, they just needed the Mayor and two others. She inquired of Council Member Kou’s schedule. Council Member Cormack did not know; there were certain days she worked. She asked the Committee if they believed all seven Council Member’s needed to be present. Chair Kniss did not think so. Council Member Filseth did not think all seven needed to be present, but four Council Members were iffy. Council Member Cormack understood and wanted that point clarified. Regarding the timeline, was it assumed that the new City Auditor was going to be ready in August, 2020 when the Council was back from their break. Mr. Larson said no. The goal of the schedule was to have the contract negotiated, drafted and ready for approval at the August 17, 2020 Council Meeting, as well as a Resolution appointing the City Auditor, per the Palo Alto City Charter. The goal was to have a few weeks’ time before the Staff Report had to go out to negotiate and draft the contract. If July 28th and 29th, 2020 worked, he was inclined to recommend Tuesday, August 4, 2020 in the afternoon. Council Member Filseth wanted to bring this topic up to the Committee. Council Member Cormack observed that the Council was on Summer Break, but the appointment of the City Auditor was an extraordinary work. She anticipated a big Agenda on August 4, 2020 and inquired if there was going to be a meeting on August 24, 2020. If yes, that gave the Council another week; then the appointment of City Auditor still happened in August 2020. She assumed everyone thought August 17, 2020 was the last meeting in August. She asked the Chair for input on a potential Council Meeting on Tuesday, August 4, 2020. Chair Kniss did not think it was a problem but also did not think the Council advanced themselves by having a meeting on August 4, 2020. It was going to be difficult to get more than five Council Members to attend a meeting, no matter when they did it. FINAL MINUTES Page 24 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Council Member Cormack mentioned that it gave people more time to review the resumes. She questioned if the RFP’s were going to be provided to the other Council Members at this time. Mr. Larson said yes, of the candidates going forward. Council Member Cormack observed that the applications were quite lengthy. If she were not on this Committee, it was going to be hard to review the applications in 24 hours and be able to absorb all the required information. Mr. Larson was providing just the resume to the other Council Members because that was the information available to them in the Closed Session, but if there was another week’s time, he thought the full proposals could be given to the Council. Mr. Gonda concurred with that assessment although he was a little concerned about identifying the interviewees and then having a week’s lapse time until the Closed Session. Council Member Cormack asked why. Mr. Gonda explained that having interviews back-to-back allowed for minimal opportunities for fairness issues to crop up. Chair Kniss suggested sticking with the July 29, 2020 date. Council Member Cormack concurred. Chair Kniss planned on working with the Clerk. Ms. Minor said she would work with the Chair. Chair Kniss restated she was going to be working with the City Clerk. She asked if there was a Council Meeting on August 24, 2020. Ms. Minor said yes. Chair Kniss questioned if that was announced yet. Ms. Minor said no as the decision was made recently. FINAL MINUTES Page 25 of 25 Sp. Council Appointed Officer’s Committee Meeting Final Minutes 7/21/2020 Chair Kniss heard that was in the works but had not seen an announcement. Ms. Minor planned on sending a meeting invite and said it was also going to be listed on the Tentative Agenda, which was going to be published Thursday. Chair Kniss asked for the date. Ms. Minor said the Agenda was going to be published Thursday for the meeting for Monday, and the Tentative Agenda was going to include the additional meeting for August 24, 2020. Oral Communications Chair Kniss inquired if there were any public speakers but there were none. Adjournment: The meeting was adjourned at 11:41 A.M.