For: Digital talking book distribution analysis. Task 1 - Distribution system analysis and selection (Final report : September 16, 2005)
Section 4 - Evaluation of Distribution Options
4.1 Evaluation Factors Definitions and Weights
Evaluation factors are the considerations that should be taken into account when determining which distribution option for Digital Talking Books (DTBs) should be implemented. A model using weights assigned to the factors has been used to force objectivity in the selection process, take into account both economic and non-economic considerations, and consider the impacts of implementation upon both network libraries and NLS (impacts upon the USPS are not considered.)
Weights have been assigned to the identified factors and they have been used to evaluate the two viable distribution options via a "score" developed from their weights. Each non-economic factor is itself composed of several components, or subfactors, that in tandem determine the score for each factor for the given distribution option.
Below is the list of factors, subfactors and weights/maximum scores, with definitions resulting from several meetings held between ManTech and NLS during the course of Task 1 of this project. The schema is based upon a total possible maximum score of 100, with economic and non-economic considerations given equal weight (i.e., costs have a 50% weight, and the total of non-cost factors also carries a 50% weight).
Factor 1 - Total Costs (50% Weight)
Total operations cost on an average annualized basis after the transition to digital technology is completed and steady-state operations have been attained. The cost of operations could be a make-or-break consideration for implementation of an option.
NLS Costs:
Includes, indirectly via contractor performance, labor costs for distribution (including duplication) functions performed at Duplication-on-Demand Centers, Multistate Centers, and for the mass-production of DTBs; facility space costs used by Duplication-on-Demand Centers and mass-producers of books, and facility space for storage of recorded book collections and storage equipment used at MSCs; costs for information system functionality required to support Duplication-on-Demand Center(s) processing operations, mass data storage requirements, and 2-way communications with network libraries; costs for modification to the READS systems to facilitate future distribution operations at libraries, and the PICS system to support both network library and NLS operations; and the costs for flash memory cartridges used for both mass-production and duplication-on-demand of DTBs provided as GFE to contractors. Excludes costs associated with DTB narration/title master production; production, distribution and repair of playback machines for DTBs; the residual RC circulation system; and recorded magazine production and distribution.
Network Library Costs:
Based upon NLS Life-Cycle-Cost (LCC) model with FY 2004 updates. Includes labor costs for distribution-related functions (receiving, inspection, repair, putaway, shifting, weeding, disposal, pulling, packing and shipping recorded books); costs for facility space used for storage of recorded book collections and storage equipment used therein; costs for information system functionality sufficient to support library-level distribution operations, communicate on a 2-way basis with Duplication-on-Demand Center(s) systems, and assimilate circulation information from the same; costs for training staff involved in distribution operations in order to implement DTB distribution at the library level; and costs for media and supporting materials used that are not provided by NLS. Excludes costs not directly associated with distribution operations and/or that are not expected to vary under different DTB distribution options; for READS system modifications; for operation of the residual RC system; for playback machine operations; and for Braille distribution operations.
Factor 2 - Impact Upon Patrons (25% Weight)
A very important consideration since the free national library program exists for the purpose of serving eligible patrons. The Impact Upon Patrons factor is composed of six subfactors.
Subfactor 2.1 - Timeliness of Service (6% Weight):
is a function of both the timeliness of distribution processing operations at network libraries and/or distribution-on-demand centers, and the length of time books spend in the mail.
Subfactor 2.2 - Quality of Product (6% Weight):
attributes of the DTB and associated containers and the relative desirability of those attributes. Excludes "Timeliness of Service" which is considered separately.
Subfactor 2.3 - Availability of Copies (6% Weight):
the relative availability of copies of titles being circulated to patrons, however the circulation is generated or the distribution is provided.
Subfactor 2.4 - Walk-In Support (1% Weight):
thedegree of support provided by the distribution system to support walk-in circulation.
Subfactor 2.5 - Rush Orders (3% Weight):
the degree of support provided by the distribution system to support rush-order circulation.
Subfactor 2.6 - Complexity (3% Weight):
the relative complexity of obtaining, using and returning books to the appropriate service location(s).
Factor 3 - Complexity of Operation (18% Weight):
A strong consideration in the option selection process regardless of an option’s relative standing in other evaluation factor areas. The Complexity of Implementation factor is composed of 9 subfactors.
Subfactor 3.1 - Contracting (3% Weight):
applies to NLS only and refers to the requirements for obtaining mass-production and/or duplication-on-demand services for DTBs via contractor performance.
Subfactor 3.2 - Duplication Labor, Equipment and Supplies (2% Weight):
For both the NLS and libraries, refers to equipment (including supporting software) requirements for the duplication of DTBs. "Supplies" applies to network libraries only and refers to the availability and cost of supplies necessary for local duplication of both NLS and locally-produced titles.
Subfactor 3.3 - ADP/Telecommunications Systems (3% Weight):
For libraries, this refers to the ADP systems necessary to facilitate circulation of books under the new system (whether locally and/or centrally distributed). For NLS, it refers to ADP systems necessary to support duplication-on-demand and mass-production operations, the READS system, and the PICS system.
Subfactor 3.4 - Training (1% Weight):
the complexity of the training requirements necessary for relevant personnel in order to effect implementation of the future distribution system.
Subfactor 3.5 - Copy Allotment (2% Weight):
the complexity of the process for determining total production quantities for titles and the allocation of copies to libraries.
Subfactor 3.6 - Media Reuse (2% Weight):
the complexity of the process for determining the excess numbers of copies in libraries for titles, the transfer of copies among libraries from locations of relative excess to locations of relative deficiency, and the reuse, recycle or disposal of copies deemed in excess systemwide.
Subfactor 3.7 - Circulation System (3% Weight):
the procedures used to distribute DTBs (separately considered from ADP Systems).
Subfactor 3.8 - Network Expectations (1% Weight):
the expectation that apparently already exists among some network librarians that fewer titles will have to be stored in and distributed from libraries in any future system than is currently the case.
Subfactor 3.9 - Cooperation (1% Weight):
the degree to which network libraries and NLS will need to cooperate under the selected option.
Factor 4 - Risk/Vulnerability (7% Weight):
Operational risks associated with the distribution options are very important and must be considered in the evaluation process. The Risk/Vulnerability factor is composed of two subfactors.
Subfactor 4.1 - Loss or Reduction of Circulation Output (4% Weight):
the relative vulnerability of the distribution system to perils other than "Disasters" that result in either unwanted reductions in circulation or a complete cessation of circulation.
Subfactor 4.2 - Disaster Impacts (3% Weight):
relative ability of the system to withstand disruptions due to disasters and continue to provide circulation.
4.2 Application of Evaluation Factors to DTB Distribution System Options
The evaluation factor weights described in Section 4.1 were then applied to the baseline scenario for the All Mass Duplication option and the Hybrid option for mass-duplication being performed in the range cited in Section 3 as being the most favorable for implementation. These results are shown in the evaluation matrices in Appendices 63 through 68 for 25%, 30%, 35%, 40%, 45% and 50% of new titles mass-duplicated, respectively. Within these operational bounds, the non-economic factor weights for options are the same.
Factor 1 - Total Costs:
Composite of NLS and Library Costs. All Mass Duplication score ranges from 36.1 to 39.6, while the score for the Hybrid is 50. The All Mass Duplication score is calculated as 50 * (Hybrid Costs/Mass Duplication Costs), so e.g., if the Mass Duplication cost was twice that of the Hybrid it would receive a score of 25 (one half of the maximum) rather than 50.
Factor 2 - Impact Upon Patrons:
Composite of scores for 6 factors below. Mass Duplication score is 23.5 and Hybrid score is 22.
Subfactor 2.1 - Timeliness of Service:
Hybrid has a score of 5, and Mass Duplication has a score of 6. Timeliness of service will be only very slightly better for Mass Duplication than for Hybrid. While books will experience 1-2 days longer average transit times in USPS stream, DOD operations will equal service response times in the better libraries and exceed response times in others.
Subfactor 2.2 - Quality of Product:
The Hybrid has a score of 5.5 and Mass Duplication has a score of 6. There would be almost no difference in quality of product (sans delivery time). DTBs would use the same cartridges, labeling and audio recording produced to identical quality levels. However, there would be no container labeling on copies shipped from the DOD Centers.
Subfactor 2.3 - Availability of Copies:
Hybrid has a score of 6, while Mass Duplication has a score of 4.5. Hybrid delivery should improve overall availability to a moderate degree, rather than either very slight or significant degrees.
Subfactor 2.4 - Walk-In Support:
Mass duplication has a score of 1, while Hybrid has a score of 0.5. Walk-in support under Mass Duplication option would be superior to that under Hybrid.
Subfactor 2.5 - Rush Orders:
Hybrid has a score of 2.5 and Mass Duplication has a score of 3. There would be almost no difference in the service response times for rush orders (sans delivery time) under the two distribution options.
Subfactor 2.6 - Complexity:
Mass Duplication has a score of 3, while Hybrid has a score of 2.5. Patrons would have to return books to two locations under the Hybrid, but only one (MSCs excepted) under the Mass Duplication option; other than this, there should be no other differences.
Factor 3 - Complexity of Operation:
Composite of scores for 9 factors are below. Mass Duplication score is 15.5 and the Hybrid score is 9.5.
Subfactor 3.1 - Contracting:
Mass Duplication has a score of 3, while Hybrid has a score of 2. The Hybrid will require contracting for both mass-duplication and DOD operations, while the Mass Duplication option would require contracting with only mass-duplication contractors. However, the DOD contractor may also turn out to be the/a mass-duplication contractor, which to some extent simplifies matters. Also, the number of mass-duplication contractors would possibly be greater under the Mass Duplication option than under the Hybrid option.
Subfactor 3.2 - Duplication Labor, Equipment and Supplies:
Mass Duplication has a score of 2, while Hybrid has a score of 1. Duplication equipment required for the Hybrid option will be more complex than that possibly required for the Mass Duplication option, but both require development work.
Subfactor 3.3 - ADP/Telecommunications Systems:
Mass Duplication has a score of 3, while Hybrid has a score of 0.5. ADP systems for the Hybrid option are more complex for both libraries and the NLS DOD contractor(s) than the systems required for libraries and mass duplication contractors under the Mass Duplication option.
Subfactor 3.4 - Training:
Mass Duplication has a score of 1, while the Hybrid has a score of 0. More training would be required for the Hybrid than the Mass Duplication option because procedures/processes are slightly more complex.
Subfactor 3.5 - Copy Allotment:
Mass Duplication has a score of 2, while the Hybrid has a score of 1. Copy allotment workload would be slightly greater under the Hybrid option than under the All Mass Duplication.
Subfactor 3.6 - Media Reuse:
Mass Duplication has a score of 0.5, while the Hybrid has a score of 2. Media reuse workload would be greater under the Mass Duplication option than under the Hybrid option because more titles are managed by the libraries.
Subfactor 3.7 - Circulation System:
Mass Duplication has a score of 3, while the Hybrid has a score of 1. The circulation system used by the Hybrid option would be more complex than that used by the Mass Duplication option because of the necessity for interface with the DOD Centers.
Subfactor 3.8 - Network Expectations:
Mass Duplication has a score of 0, while the Hybrid has a score of 1. Some network libraries are anticipating labor and/or facility space savings under the Hybrid option relative to that under the Mass Duplication option.
Subfactor 3.9 - Cooperation:
Mass Duplication and the Hybrid each have a score of 1. Cooperation requirements would be about the same under the two options.
Factor 4 - Risk/Vulnerability:
Composite of scores for 2 factors below. Mass Duplication score is 7 and Hybrid score is 3.
Subfactor 4.1 - Loss or Reduction of Circulation Output:
Mass Duplication has a score of 4, while the Hybrid has a score of 2. Risk is more diversified under the Mass Duplication option, but the operation of two rather than one DOD Center substantially reduces this risk. Risk could be further reduced if Centers have capacity to handle workload in one shift (could then implement a second shift if other center’s operations were disrupted).
Subfactor 4.2 - Disaster Impacts:
Mass Duplication has a score of 3, while the Hybrid has a score of 1. Risk is more diversified under the Mass Duplication option, but the operation of two rather than one DOD Center reduces this risk. Risk could be further reduced if Centers have capacity to handle workload in one shift (could then quickly implement a second shift if the other Center’s operations were disrupted).
Total Score:
The Mass Duplication option ranges from a total score of 82.1 to 85.6, while the Hybrid option has a constant score of 84.5 for the six mass-duplication scenarios shown in Appendices 63 – 68 (this is because it has the lowest total cost in all cases so it receives the maximum Total Cost score, while that for the All Mass Duplication option is scaled relative to the Hybrid’s cost). For mass-duplication levels of 25%, 30%, 35% and 40% of new title production, the Hybrid slightly outscores the All Mass Duplication Option, and it slightly underscores the All Mass Duplication Option for mass-duplication levels of 45% and 50%. ( appendix 63 appendix 64 appendix 65 appendix 66 appendix 67 appendix 68)
4.3 Recommendations
- DTB distribution should begin with the mass-duplication option only, probably for the first two or three years of the transition period, but then evolve to a Hybrid distribution option of some type thereafter. The specifics of the transition period are to be addressed later in this project. This is recommended due to both the lower volumes of mass duplication that will be required during the years in which RC duplication will also continue (possibly title as well as copy production), and so that DTB flash memory cartridge duplication technology and library ADP systems and procedures can be developed in order to facilitate DTB DOD.
- The Hybrid option is clearly more economically efficient than the All Mass Duplication option. Its implementation will result in significantly lower costs for DTB distribution for the system as a whole and for network libraries relative to the Mass Duplication option. NLS costs in the apparently most efficient new title mass-duplication range should be equal to or less than NLS costs for the All Mass Duplication option.
- The Hybrid option is a superior alternative to the All Mass Duplication option when non-economic factors are considered in tandem with costs using the Evaluation Factors Matrix for the range of 25% - 40% of new titles being mass-duplicated, and only very slightly less than the All Mass Duplication option for mass-duplication levels of 45% - 50%. While the Hybrid will be somewhat more complex to implement and will have more inherent operational risk than will the All Mass Duplication option, its impact upon patrons will be almost the same as that for the All Mass Duplication option and it is more cost efficient.
- There are several ways in which the Hybrid option can be implemented vis-a-vis new title production, and/or library collection weeding and/or title migration, and/or a combination of both the "title-based" and "time-based" Hybrid strategies. Such variations upon the common theme of the Hybrid option will be closely examined and analyzed further in this project, and the most favorable approach identified and developed, in subsequent study tasks.