Embedded Quality is an automated quality review system which allows reviewers to rate case quality using quality standards called "attributes." The use of the same attributes for both managerial (EQRS) and national (NQRS) reviews provides consistency and comparability.

The Embedded Quality Review System (EQRS) is used by managers to document all case related reviews of employees, including closed case reviews, 4502 reviews, workload reviews, etc. Attributes in EQRS are linked to Critical Job Elements (CJEs) to assist managers in performance evaluation. Procedures for EQRS may be found in the BSA Manager's Tool Kit and in BSA EQRS training course #19054.

The National Quality Review System (NQRS) is used by BSA national reviewers to review both sample and mandatory review cases. Sample cases are collected using a statistically valid sample so that results will be representative of the entire BSA case universe.

NQRS data is used by management to assess program performance and to identify training and educational needs to improve work processes.

The NQRS program is administered within the Workload Identification, Selection, Delivery and Monitoring (WISDM) section of BSA Policy.

BSA national reviewers, supervised by a WISDM section supervisor, conduct reviews of both Title 31 and Form 8300 cases and provide data to BSA Headquarters on at least a quarterly basis.

To accomplish these objectives, BSA national reviewer responsibilities include:

An attribute may be defined as a specific aspect of case work that the review will measure. "Attribute" is synonymous with "quality standard." Attributes compare to, and replace, the quality standards formerly used by SB/SE.

Attributes are concise statements of SB/SE BSA's expectations for quality examinations/reviews and are guidelines to assist examiners in fulfilling their professional responsibilities. Each attribute is defined by elements representing components that are present in a quality examination/review.

Attributes provide objective criteria against which case quality is assessed and are grouped into the following five quality measurement categories:

Attributes directly link to the examiner's critical job elements (CJEs) in EQRS.

Attributes are divided into six categories for ease of data input, as follows:

The BSA website and the EQ Home page provide various reference materials to aid the reviewer in rating cases. These reference materials include:

Each attribute and aspect has been worded as a question. This format minimizes the need for interpretation and is scored as:

This scoring system provides maximum flexibility for reviewers, while emphasizing select key elements and aspects that represent priorities for a quality examination.

Key elements and aspects are those selected items that have been determined as a prerequisite to a quality case and are essential to IRS goals.

Every attribute contains one or more key elements/aspects.

If any one of the key elements/aspects of the attribute is rated "No," it will not cause the entire attribute to be rated "Not Met."

CTR Operations at the Detroit Computing Center (DCC) will select a valid sample of closures for NQRS review. The sample size is determined statistically to provide a valid sample at the National level. The sampling plan is determined from the number of closures per the BSA work plan.

CTR Operations will select Title 31 cases and Form 8300 cases separately using the statistically valid sample selection rate determined for that type case.

Survey cases will be included in the closures from which the sample is drawn.

CTR Operations will send all selected cases to the NQRS Section Chief. The Section Chief will assign the cases to the reviewers.

In addition to the closed case statistical sampling from CTR Operations, the following Title 31 cases are also subject to NQRS review as "Mandatory Review" cases:

In addition to the closed case statistical sampling from CTR Operations, the following Form 8300 cases are also subject to NQRS review as "Mandatory Review" cases:

Mandatory review cases are to be sent to review by the group manager with a Special Handling Notice. The "Mandatory Review" block should be checked on the Closed Case Document.

The first step in the Embedded Quality process is the review of the case. The purpose of the national BSA quality review process is to provide management with measures on how well work processes are carried out and how these processes can be improved.

National reviewers are responsible for reviewing BSA cases for quality using five measurement categories:

It is important that reviewers have an in-depth understanding of the EQ attributes and process measures, as well as a good understanding of BSA law, requirements and procedures. Furthermore, it is important that the reviews be complete, accurate and consistent to promote a true and valid sampling that management can rely upon to improve examination processes. See IRM 4.26.18.12. Consistency of Reviews.

Closed case review results are entered into a web-based Data Collection Instrument (DCI) on the National Quality Review System (NQRS). See IRM 4.26.18.11 (below) for detailed input procedures.

Generating Reports: Once the review is completed, the results are included in the raw data from which the EQ system generates reports. Reports will be generated separately for Title 31 and Form 8300 case reviews to ensure separation of the Title 31 and Title 26 data. See IRM 4.26.18.14 (below) for more on reports.

For help in reviewing cases, reviewers may access electronic research portals and reference materials on the Embedded Quality (EQ) home page. Available reference materials include the BSA Job Aid, Attribute Reference Guides and Crosswalks, as well as links to the IRM. The EQ home page can be accessed at http://sbse.web.irs.gov/EQ/default.htm. Reviewers can also access the BSA web page for additional information.

An Embedded Quality Job Aid has been developed to be used as a guide in completing the NQRS database input screens including headers and attributes.

The Job Aid provides operational and job aid definitions, points to consider, reason codes (If available), rating guide explanations, attribute examples, IRM references and CJE links for each attribute in the guide.

Case reviews are input into the web-based system known as the National Quality Review System (NQRS).

Each case review is input using a Data Collection Instrument (DCI). The DCI guides the review process and documents reviewer comments. This documentation can be extracted from the system through statistical and informational analysis reports.

The NQRS database is accessed through the EQ home page at http://sbse.web.irs.gov/eq/default.htm.

Reviewers must complete an On-line Form 5081 to obtain a password to access the NQRS system.

The Data Collection Instrument (DCI) provides the principle documentation for the reviewer's case evaluation and conclusions. A DCI will be completed for each case reviewed in the NQRS system.

Header input procedures: The first input section of the DCI is the data input of headers that capture basic information about a closed case, such as "Who, What, When, Where and Why." This information if categorized into four groupings:

Evaluating and Coding the Attributes: The next step in the case review process is the evaluation of the case by rating attributes that are specific to a Specialized Product Review Group (SPRG). (BSA is one of the SPRGs.) Attributes on the DCI are either required or optional and all required attributes (in bold) must be rated "Yes" or"No" in order for the DCI to be recorded as complete. Other attribute fields should be completed if information is available to ensure an accurate quality case review.

Attribute reason codes: Most of the attributes have reason codes. Once the attributes have been rated, at least one reason code must be selected (where reason codes are present) for each attribute rated as not met ("N" ). The reason code that best applies should be selected. If none apply, "Other" may be selected and a narrative should be entered describing the reason.

Reason Code Narratives: In addition to selecting a reason code on all attributes rated "N," the reviewer is required to write a narrative in the attribute narrative box specifically describing why a case failed a particular attribute. Positive comments may also be entered for attributes that have been met and are encouraged in order to provide a complete and balanced case review.

Narrative Guidelines: NQRS reviewers need to be thorough in documenting case performance by providing clear, concise and specific descriptions of any errors in order to identify problem areas. Positive actions should also be described in specifics. Citing IRM or other references will add validity to the narratives. Narrative remarks that are too general may not offer sufficient detail to allow either specific recommendations for improvement or recognition of good performance.

Completing the DCI: Once the headers, attributes and narratives are completed, the reviewer should select the "Complete Review" screen. At this point, the EQ system notifies the reviewer if there are any required items needed to complete the case review and/or provides suggestions of recommended items. The reviewer will not be able to complete the case review until all required input is addressed.

Since the attribute ratings in NQRS form the basis of quality performance measurement for BSA, it is important that attribute ratings are complete and accurate. Reviewers must strive to be consistent over time in rating similar case actions on all reviews.

Incomplete or inaccurate national quality reviews can lead to a distortion of Business Results - Quality Performance. Furthermore, general narrative remarks may not provide sufficient detail to allow for meaningful drivers for improvement.

Guidelines for consistent reviews:

Consistency checks of national reviews can be performed in several ways:

Consistency checks of EQRS will be done by Territory Managers on a regular basis. National reviewers may be called in to assist in this process.

The NQRS database includes basic pro forma reports designed to assist in the analysis of data collected through case reviews. These reports include:

Reports are generated separately for Title 31 and Form 8300 case work.

NQRS reports can be generated periodically, as needed. At a minimum, these reports will be generated and analyzed on a quarterly basis.

The NQRS Reviewer's Guide should be used when interpreting or analyzing NQRS reports. The definitions of attributes and process measures are narrow and rules for ratings are specific to the individual elements.

NQRS data will be used by management to assess program quality performance as well as to identify training and education needs.

NQRS data will not be used to evaluate individual employee performance. This is the role of the EQRS, where attributes are linked to Critical Job Elements (CJEs).

Commendatory: If the NQRS reviewer identifies a case exhibiting exceptional quality or one reflecting innovative examination techniques, the reviewer will prepare a Form 3990 Reviewer's Report (Commendatory) and forward to the NQRS Section Chief for approval. Upon approval, the form will be forwarded to the group. The case will be closed by the Section Chief to CTR Operations (DCC). The commendatory report information will be reflected as a best practice in the quarterly NQRS reports.

Correction: If the NQRS reviewer identifies a circumstance in a case review that would have an adverse impact on the BSA program (such as the failure to issue a Letter 1112 where it was appropriate), the reviewer will prepare a Form 3990 and forward to the NQRS Section Chief for approval. Upon approval, the entire case file will be returned to the group for corrective action. After corrective actions have been taken, the case will be returned to the Section Chief for processing back to CTR Operations (DCC).

All other completed review cases will be forwarded to the NQRS Section Chief for processing back to CTR Operations (DCC).