Introduction

Industry and government has a long tradition of purchasing and subcontracting for products and services.  This type of purchasing and subcontracting is currently called sourcing or outsourcing. The Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations in the health care industry and the Sarbanes-Oxley Act’s Section 404, requires the management assessment and audit of all public companies Internal Controls as an integrated part of their financial audit [AICPA and HIPAA].  Following from these regulations, the AICPA’s Professional Ethics Executive Committee is exploring issues surrounding outsourcing to third-party providers.  Outsourcing control methodologies are therefore becoming an essential element of organizations required internal controls.

This paper will present a proven outsourcing internal control methodology that has been used for several decades in the information technology arena, since the primary functions of a modern Information Systems organization, except for strategic planning, can be either performed in-house or outsourced to development, processing, networking or consulting providers/vendors.  The evaluation of these providers/vendors is usually based on some type of cost-value analysis to rank and select providers.  A basic method for such cost-value analysis is the computation of a worth index.  Since almost all outsourcing proposals are required to provide a technical and managerial proposal and a separate cost proposal, the worth index is computed as:

                Worth Index = (α*Technical Score + β*Managerial Score) / Life Cycle Cost

This paper includes a proven methodology for computing the technical score, managerial score and life cycle costs for a Worth Index using both RFP and RFQ approaches.  The Worth Index methodology presented in this paper is applicable to functional sourcing opportunities in six IS/IT areas: the full IS organization (excluding strategic planning), IS development projects, IS data center production, IS technical support, tele­communications, and architecture planning support. These functional sourcing opportunity areas exist at both the enterprise and department/ workgroup levels.

Quantitative Evaluation Methodology

A fabricated comparison, based on several actual selection projects, between an in-house and three external potential vendors of an applications software package will be used to illustrate this papers proposed quantitative worth-index based process.  The quantitative evaluation process is diagrammed in the following model. 

Worth Index Computation Process

This paper will first discuss an overall methodology for sourcing evaluation based on popular RFP/RFQ approaches, second present the qualitative method used for the numerator, third a quantitative approach for the denominator, and fourth present some interesting approaches for presentation of the results of the analysis.
 

RFP/RFQ Methodologies

 

The process shown in the prior chart and described in the following sections of this paper are RFP based.  Equal weight is placed on quality and cost.   Most advanced systems and products use this RFP methodology. 

 

The RFQ approach shown in the following typical chart is used for projects with well known methodologies such as construction and sometimes consulting or shrink-wrapped package selection.  The RFQ method uses a Fail/Pass approach to evaluation of technical and managerial factors.  The sealed cost proposals are then opened for only the highly qualified bidders.  The low bidder is then the winner.  Note the Minimum Score column in the European Union procurement evaluation form following (The European Commission 2001).  That column will not be used in the detailed IS example in this paper, since it is primarily RFQ oriented.  

A Typical Management and Technical Proposal Evaluation Form

(EU “User-Friendly Information Society” Project

 

Each vendor is evaluated and scored.  The scores are multiplied by the weights and summed to give a weighted total score for those vendors that exceed all minimums.  The cost proposals are then opened and the lowest bidder is awarded the contract (Halifax 2000).  Note: This method does not work for high technology projects and one-of-a-kind projects or products. 

 

The RFP oriented methodology that follows in this paper, therefore applies to most Information systems projects.  The following chart outlines the procedure for computing the worth index.  A management proposal evaluation team and a technical proposal evaluation team are formed at the time of RFP issuance.  They follow the evaluation methodology in the next sections of this paper.  The weighting of each proposal (α and β) is normally specified by executive management. 

 

Deriving Technical and Managerial Scores and Life Cycle Costs 

A four step process for deriving the worth index follows. 

Step 1: Define Value of Specific Evaluation Criteria 

The specific criteria used in this illustration include: 

• Functionality
   

• Package capability related to functional requirements as a percentage of a perfect match. 
   

• Platform Utilization
   

• The forecasted utilization of a current processing platform as a percentage of maximum feasible capacity. 
   

• Survival Probability
   

• The forecasted probability, shown as a percentage, that the vendor package will maintain or expand its share of market over the planning horizon of this application. 
   

• Initial Cost
   

• Front end cost in $ of software, support, training, conversion and taxes. 
   

• Annual Cost
   

• Continuing costs in $ of maintenance and support. 
   

• Annual Benefits
   

• Estimated cost reductions or profit increases in $ due to converting to the new system.

More details on scoring these criteria can be found in the following section – “Sourcing Evaluation Criteria”.

A typical result of the application of this step is shown in the following table.

 

Step 2:  Compute Life Cycle Costs and ROI

Computing a return on investment (ROI), requires (in addition to initial and continuing costs), an estimated life of the project[1].  Currently many investments in applications software involve a planning horizon that is twice the platforms technology cycle, while most investments in platform alternatives involve a single technology cycle planning horizon.

Therefore assuming a ten year planning horizon (twice the mainframe five year technology cycle) with no adjustment for inflation, an ROI computation using the internal-rate-of-return methodology follows.

 

Step 3:  Compute Qualitative Criteria Index

Combining the three illustrated technical criteria requires that their relative importance be determined.  This type of importance ranking methodology (called the Delphi Method when first presented by Rand Corporation during the 1950's) includes the use of expert's rankings which are then normalized into a weighting scale running from 0 to 1.  Applying this approach to the illustration results in the following table:

The weighted value columns are the product of the weights assigned by the experts times the evaluation criteria scores contained in the table from Step 1.

Step 4:  Compute Worth Index

The computation of a quantitative worth index for the illustrative evaluation is now straight forward.

Based on the worth index, vendors B and C are approximately equal from an objective (quantitative) viewpoint.  The decision between them would be based on subjective criteria such as competitive issues and control

The worth index can be computed in three forms, using the ROI as shown in the illustration, using net present value (NPV), and using life cycle costs.  The formulas for each follow.

• Using ROI

                                WORTH = SCORE X ROI

• Using NPV

                                WORTH = SCORE X NPV

• Using Life Cycle Costs

                                WORTH = SCORE ÷ COST

The next section will discuss and structure the subjective and objective evaluation criteria relevant to scoring decisions.

Sourcing Evaluation Criteria

The evaluation criteria used in selecting sourcing alternatives can be divided into two major categories:

Objective Criteria

                These can be quantified through costing.

Subjective Criteria

These require intuitive weighing and are used for score individual criteria.  They can also be used for screening unacceptable approaches prior to the formal evaluation discussed in this paper.

The objective criteria used to compute Life Cycle Costs & ROI are discussed in a later section of this paper.  The subjective criteria evaluated through scoring are discussed in this section.

The scoring of criteria can often have different forms when applied to in-house and external vendors.  When relevant, these differences are highlighted.

Criterion 1 - End User Deliverables Functionality

When relevant, this functionality criterion evaluates the quality, from the view of the user, of the application/product/service deliverables to be provided by in-house or vendor organizations.

Criterion

What is the quality of the deliverables in terms of meeting end user defined functional requirements.

Scoring

The evaluation measures for developing a score for meeting functional requirements is completely dependant on the type of deliverable (eg. application system, processing capability, image system, strategic plan, etc.).  A small portion of a multi-page functional evaluation follows as an example of the type of approach often used.

Deliverables Functionality Example - Applications Software

REQ  7  7.1  7.2  7.3  7.4

Generate Monthly Reports   Yield Analysis   Arrears Trends   Loan Growth   Rate of Return TOTAL POINTS AVERAGE POINTS

Essential (1)/ Desired (.8)   D E D  E 3.6

Standard (1)/ Custom (.5)   C S S  C 3.0

Points     .4  1.0   .8   .5  2.7   .75

Deliverables Functionality Example - Data Center

REQ  5  5.1  5.2  5.3  5.4

Help Desk Capability   Automated Task Status   Automated Report Status   Automated Input Status   Rescheduling Capability                                                                 TOTAL POINTS AVERAGE POINTS

Essential (1)/ Desired (.8)   E E E  D 3.8

Standard (1)/ Custom (.5)   C S S  C 3.0

Points     .5  1.0  1.0   .4  2.9   .76

 

Criterion 2 - Product/Service Life

When relevant, this criterion is used during the evaluation of products where continuous enhancement is needed over the planned life of the product or service.  Enhancement requirements can be due to such items as evolving user/legal requirements and evolving technologies.

In-House Supplier Criterion

        In-house suppliers are often assumed to have an indefinite life.  This can be very misleading if the internal enhancement skills required to maintain the product or service are not within the mainstream of IS activities.

               A.         What is the probability that the skills needed for support of the product/service will be available over the project/service life cycle?

External Vendor Criterion

                 B.         What is the probability that the firm supplying support will maintain or improve its competitive position over the project/service life cycle?

                  C.         What is the probability that the firm supplying support will still be providing adequate support over the project/service life cycle?

Criterion Applicability

                                HARDWARE:

                                                Processing                            A,C

                                                Network                 A,C

                                SOFTWARE:

                                                Applications                         A,B

                                                Systems                 A,C

The scoring of this criterion is subjective and normally based on the number of years that in-house capability has been maintained or on the number of years that a potential vendor has been supp