Detailed Information on the
Examining Compliance with Securities Laws Assessment

Refine and/or enhance the program's surveillance and risk assessment processes for registered firms. These processes include analyzing additional data points, as well as new methods for conducting examinations of dual registrants.

Further refine the agency's Budget and Program Performance Analysis System (BPPAS), an activity-based costing and performance-based budgeting system to develop and present integrated budget, performance, and cost information.

Establish a baseline for its long-term performance measure in 2007.

Continue collecting performance data on its annual measures to ensure the program is achieving its goals.

Initiating risk-targeted exam initiatives in select, key risk areas identified by program-wide risk-mapping exercise and through other means.

Measure: The percentage of exams that resulted in deficiency letters requiring the registrant to take corrective action.

Explanation:Deficiency letters promote compliance with federal securities laws and SRO rules.

Measure: The percentage of exams disposed of within 120 days from the end of field work (excluding risk-targeted exam initiatives). (From 2004 to 2007, this measure tracked the percentage of exams disposed of within 90 days.)

Explanation:Promptly providing notice of findings to registrants results in immediate improvement in compliance and allows exam staff to maintain exam schedule.

Measure: Percentage of attendees at CCOutreach that rated the program as "useful" or "extremely useful" in their compliance efforts. (New measure, added February 2008)

Explanation:The CCOutreach program is a collaborative effort sponsored by OCIE and the Division of Investment Management. It consists of several components that are designed to, among other things, educate, inform, and alert Chief Compliance Officers (CCOs) of pertinent information, issues, or events affecting adviser and fund compliance. At the conclusion of all events in which CCOs participate, CCOs are given the opportunity to rate the usefulness of the information provided as part of this program in assisting them in their compliance efforts.

Measure: Percentage of firms receiving deficiency letters that stated they took or would take corrective action in response to all exam findings. (New measure, added February 2008)

Explanation:Examiners find a wide range of deficiencies during examinations. Some of these deficiencies are more technical in nature, such as failing to include all information that is required to be in a record. Other deficiencies may cause harm to customers or clients of a firm, have a high potential to cause harm, or reflect recidivist misconduct. At the conclusion of examinations, the staff communicates identified deficiencies to registrants in the form of a deficiency letter. Registrants are then given a chance to respond to staff findings and often take action to remedy any problems and potential risks.

Is the program purpose clear?

Explanation: The Office of Compliance Inspections and Examinations (OCIE) conducts examinations to foster compliance with securities laws, to detect violations of the law, and to keep the Commission informed of developments in the regulated community.

Evidence: Congress enacted laws such as the Securities Exchange Act of 1934, the Investment Advisers Act of 1940 and the Investment Company Act of 1940 to maintain investor confidence in capital markets by providing structure and government oversight. These laws authorize the SEC to examine regulated entities - broker dealers, investment advisers, investment companies and self regulatory organizations - to detect violations and promote and enforce compliance. Also, see Mission Statement.

Does the program address a specific and existing problem, interest, or need?

Explanation: The exam program protects investors and promotes the integrity and efficiency of the U.S. securities markets. The disruption of financial markets - whether criminal or non-criminal in means - continues to be a concern given its impact on the overall U.S. economy.

Evidence: In 2003, the volume traded on U.S. exchanges and NASDAQ exceeded $22 trillion and investors entrusted over $23 trillion in assets with investment advisers and companies. 8,535 investment advisers, 982 fund complexes, over 5,900 broker-dealers, 12 Self-Regulated Organizations (SROs), and over 400 transfer agents are subject to SEC exams. When necessary, examiners refer potentially serious violations uncovered during reviews to SEC's Division of Enforcement (i.e. criminal)or, when appropriate, a registrant's SRO for further investigation. SEC examiners also identify areas where regulated entities can improve internal controls and other compliance mechanisms and require entities to take corrective action (i.e. non-criminal).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The SEC is uniquely charged with administering and enforcing federal securities laws. Although SROs and state regulators assist the commisison in the oversight of financial markets, the SEC ultimately is responsible for ensuring compliance with federal statutes.

Evidence: SROs, such as NYSE and NASD, have authority to examine and discipline their member broker-dealers and employees of those members. The SEC, however, is responsible for overseeing the examination, enforcement and other regulatory activities of the SROs and can take away their authority over their members, if necessary. See also 1995 MOU with SROs and State Regulators detailing cooperation.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence to suggest the exam program would be more efficient or effective under an alternative approach. The program is designed to maintain flexibility so that emerging issues affecting the markets are addressed. In addition, the SEC is able to leverage examination resources with assistance from the SROs and state regulators.

Evidence: The SEC is expanding its ability to detect problems earlier, identify novel issues sooner, and more effectively use new sources of information to prevent problems from having a negative impact on the markets. Specifically, OCIE has established a risk assessment process involving all exam staff and regularly produces trend memoranda to ensure that the program is positioned to address risks likely to impact the markets and market participants.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program has an organizational structure and a risk-based approach to examinations that is designed to distribute resources in an effective manner.

Evidence: The organizational stucture is divided by the type of regulated entity to maximize use of the staffs' expertise. In addition, examinations are conducted on a cyclical basis, for cause, to oversight an SRO, and in sweeps targeted through risk-based priorities. The resources devoted through this structure and examination methodology are periodically evaluated to best reflect the needs of the program. This results in a reallocation of staff slots to the various regional and district offices as well as within the various regulatory groups with OCIE at Headquarters. These efforts are all designed to maximize investor protection.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: OCIE has meaningful performance measures consistent with agency goals of: enforcing compliance with the federal securities laws and maximizing the use of SEC resources. One such long term measure was recently developed -- to determine whether registered firms have improved self-compliance by tracking the corrective action(s) firms have taken in response to deficiencies noted in exams.

Evidence: The program utilizes a variety of reporting and monitoring processes to document and track progress towards its long-term performance outcome, including: its dashboard measures, the examination tracking system, STARS and planned improvements for tracking corrective action; individual examination reports, deficiency letters and firm responses to deficiency letters; and exam scorecards for IA/ICs.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term performance measure that OCIE has recently developed will trace the mitigating action(s) firms have taken to mprove their self-compliance with the Federal Securities Laws in the deficient areas noted during exams. As noted, this is a new long-term performance measure; therefore, we are in the process of developing systems to demonstrate progress towards the targets.

Evidence: The program will primarily utilize the STARS reporting and tracking system to monitor progress towards acheiving its long-term target including. Additional documentation includes: examination reports and workpapers; registrant responses to deficiency letters; enforcement referral memoranda; risk-targeted examination sweep memorandums and reports; and evaluations of regional and district offices.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Specific annual measures are used to monitor the performance of the program. These measures include the percentage of OCIE-referred cases pursued by Enforcement and the percentage of firms that take mitigating action after receiving deficiency letters. The program also has an efficiency measure related to the percentage of cases disposed of within 90 days. These measures contribute to the long-term outcome of the program by compelling the regulated community to voluntarily comply with federal securities laws.

Evidence: OCIE strategic plan outlines agency outcomes and the measures used to gauge progress.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Targets are used to assess the program's performance in: the length of time to complete an exam; the resolution of exams; and the distribution of exam findings across core deficiency categories.

Evidence: SEC 2004 Strategic Plan. SEC FY 2006 OMB Budget Request and Performance Plan. OCIE Goals. Risk matrix; Sweep Memos; STARS.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program does not have partners as defined by the question.

Evidence:  

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: GAO regularly evaluates the Exam program, most recently in April 2005. The aforementioned report highlighted concerns with mutual fund abuse and, in part, led Exam to reallocate resources to mutual fund abuse and recommend to the Commissioners changes to underlying mutual fund regulations.

Evidence: April 2005 and June 7, 2005 GAO Audits. Recent SEC Office of Inspector General finding resulted in internal program changes to better target mutual fund exams and improve the referral process to Enforcement.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: While the SEC made progress integrating performance into its budget in its FY 2006 OMB Budget Request, costs could be better integrated into the agency's performance framework. The SEC is continuing to develop a more comprehensive approach to presenting information that links budget data with performance information.

Evidence: The SEC undertook a requirements analysis to determine the appropriate budget and program performance analysis system. Additional strategies for linking budget and performance data will be addressed after this process is complete.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: During 2003 the SEC implemented new management reporting activities including performance dashboards, comprehensive risk assessment practices, and regular organizational reviews targeted at aligning human resource requirements with agency priorities. These efforts continue and the program is implementing parallel practices to further develop its planning and management activities. The commission is encouraged to adopt a limited number of long-term performance goals, however.

Evidence: In 2004, the SEC strategic plan was finalized with participation throughout the commission. In addition, budget development activities include much closer collaboration between budget, planning, and evaluation staffs.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program uses a variety of quantitative and qualitative information sources, including tracking systems and regular management meetings to adjust program priorities, make resource allocations, and take other appropriate management actions. The program maintains a flexible approach to allocating and re-allocating resources to adapt to continually changing exam needs.

Evidence: Data on all exams for Broker-Dealers, Investment Advisers, Investment Companies and Transfer Agents are tracked in the Super Tracking Reporting System ("STARS"), which generates a variety of management reports. The program staff also develops the Dashboard, a monthly compilation of statistical information generated from the management reports in STARS. The Dashboard allows the Commission and managers to evaluate trends and assess performance. The program regularly monitors the number of exams, duration of exams, the number and type of significant findings, the results of the exams, and staff retention figures. OCIE has taken a number of steps in the past few years to enhance its information tracking capabilities, including developing a more comprehensive tracking of significant exam findings (findings are broken out into more than a dozen different types of findings that can be tracked separately). OCIE also has periodically prepared a status report to the Commission on the dozens of ongoing mini-sweep exams.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The SEC implemented a pay for performance program in 2003. The merit-based pay component of this program links annual evaluations to performance. Senior officers and supervisors in the agency are held accountable for performance and program management through performance standards and evaluations.

Evidence: Annual pay for performance policy guidance is provided to managers and staff according to established employee performance review cycles. Performance management materials are also made available to staff on the agency's internal website. Human Resources "Performance Management Process" memo dated April 13, 2005. "Pay for Performance" internal web page.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Most program funding is associated with compensation, benefits, and travel, and is obligated for those purposes. Budget execution of program funds is timely and regularly monitored by senior program and agency officials. The SEC's Human Capital Review Board oversees compenation and benefits. Travel is governed by SEC-wide policies. Budget performance against operating plans are reported on monthly in the Dashboard.

Evidence: Department of Interior payroll data and SEC travel policies.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program adopted an efficiency measure on the timeliness of completing exams and has hired some technology/exam staff so that the program can more efficiently meet its goals. While the program does not currently have procedures in place to measure cost effectiveness in program execution, it is developing alternative approaches for determing program costs to achieve agency stratgeic goals.

Evidence: The program uses a measure and targets to monitor the timeliness of its exams. OCIE recently began conducting more exams utilizing electronic records of the registrant instead of on-site vists which reduces overall exam costs.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program has working relationships with officials at the SROs as well as state, foreign, and other financial regulators. OCIE periodically holds national and regional summit meetings with the SROs and state securities regulators and assists with law enforcement (eg. United States Attorney's Office, Department of Justice) and bank regulation efforts. The SEC also works with foreign regulators including conducting coordinated examinations of jointly registered investment advisers. Finally, OCIE has entered into separate arrangements with the Commodities and Futures Trading Commission and Federal Reserve to increase coordination and information sharing with respect to registered clearing agencies subject to joint or overlapping jurisdiction.

Evidence: The program leads or participates in several joint task forces, including a multi-divisional enforcement referral committee and adviser surveillance taskforce. Additional evidence includes an access manual that contains sharing agreements with other regulators; agendas for meetings with other regulators; minutes summarizing discussions; descriptions in examination reports.

Does the program use strong financial management practices?

Explanation: The agency went through its first financial audit this spring. Although the SEC received an unqualified opinion, the auditor (GAO) reported three material weaknessess including the reporting and recording of disgorgement and penalties; financial statement preparation and disclosure; and information security.

Evidence: FY 2004 Fianancial Audit. GAO report #: GAO-05-691R.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program successfully resolved prior deficiencies and has implemented regular management and performance reviews. New risk assessment practices are being established to help the program direct its resources to those areas that present the greatest potential harm to the public and the industry.

Evidence: The program promptly resolved matters identified by the SEC's Office of Inspector General that required management focus, such as the targeting of mutual fund examinations and referrals from the examination program to the enforcement program. Additionally, activity levels and timeliness of examinations are regularly reported through the Dashboard report. The agency's Human Capital Review Board meets regularly to consider the organizational structure of agency programs to ensure that resources are optimally allocated to meet mission goals. As part of the SEC's risk assessment initiative, OCIE focuses on identifying and addressing risks among registrants.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: OCIE is in the process of developing long term performance goals (see Questions 2.1 and 2.2). Therefore, it is not yet possible to conclude that the program has demonstrated progress in achieving its long term goals.

Evidence:  

Does the program (including program partners) achieve its annual performance goals?

Explanation: OCIE has achieved its targets for some of the annual measures but did not reach its targets for other measures.

Evidence: STARS exam tracking system.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: OCIE has increased the percentage of exams that resulted in deficiency letters requiring firms to take corrective action. OCIE would have reduced the number of days to complete its exams in FY2005, but for the fact that we embarked on a significant redesign of our exam program. OCIE is making refinements to its tracking system to demonstrate improvements to firms' self-compliance by tracking whether firms took corrective action as a result of deficiencies noted during exams.

Evidence: Dashboard Reports and STARS database.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation:  

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: GAO and OIG have conduct regular reviews of the exam program. In general, the reviews do not indicate any significant problems with the program. However, a recent GAO report noted that exam failed to detect mutual fund trading abuses. In response, Exam has taken meaningful steps to address the report's findings.

Evidence: GAO Report #: GAO-05-313.