Section 1 The Need for Contract Evaluations
Section 2 Contract Evaluation Processes
Section 3 Movement for Best Value
Description: The purpose of this unit is to enable you to describe how post contract evaluations can be used to strengthen client relationship and give examples of the information that can be fed forward to future client relationships.
Author: Gates MacBain Associates
Section 1 The Need for Contract Evaluations
Aims and Objectives
At the end of this section you should be able to:
- Identify the initiatives that brought about change in the construction industry.
- Recognize how performance measurement and evaluation extended to every aspect of the project delivery process.
The construction industry has been in a period of significant change since the early 1990’s when the need to raise the quality of its products and customer services was accepted. The whole process of procurement, design and construction had to deliver products with greater certainty of cost, quality, time and customer satisfaction and a central driver would be performance measurement and evaluation.
Post contract evaluations at the time revealed that 73% of contracts exceeded their tender price and 70% exceeded their time estimates. Either lessons were not being learnt or actions were not being taken in a timely manner. The clear need was for the use of performance measurement and evaluation throughout the project delivery process, which indicated:
- Project evaluations at each of the main decision points before confirming progression.
- Performance evaluation against each component of the project delivery process to confirm progress as planned.
- Post project evaluation of original objectives and project outcomes.
- Project by project evaluations to confirm continuous improvement.
This unit will look at the initiatives that brought about change and how clients were made responsible for its implementation. It will look at its extension to the whole project team and the move from post-contract to continuous evaluation. Finally it will outline the development of the evaluation process to consider collaboration, benchmarking and client relationships.
The Need for Change
In his report, entitled ‘Rethinking Construction’ (See link below), Sir John Egan recommended changes to improve the efficiency and quality of construction works. His recommendations focused on product development, project implementation and integrating the supply chain. In response, working groups were established to determine the indicative performances to which the industry should be working (performance indicators) as well as a means for making comparisons across the industry (benchmarking). Other groups looked into how best practice could be recognized and shared with others (demonstration projects) and into how innovation could be encouraged and adopted.
Achieving Excellence in Construction
Central government also took up the mantle and several initiatives transpired. Two such initiatives by the Office of Government Commerce (OGC) entitled, ‘OGC Gateway’ and ‘Achieving Excellence in Construction’ required government clients to introduce processes to measure the performance of construction projects at specific stages in their progress (OGC Gateway) and to compare the outcomes with others using performance indicators (Achieving Excellence).
Achieving Excellence in Construction also required the use of tools to assess the true value of a project (using whole-life costing to determine value for money) and procedure to better manage the risks associated with the whole procurement process. Guidance on the requirements of Achieving Excellence in Construction is offered through a set of 3 core guides, 8 supporting guides and 2 high-level guides; see web reference to ‘Achieving Excellence in Construction’.
Central and local government clients involved with the procurement of construction projects are required to follow the requirements of Achieving Excellence in Construction. Thereby the performance of construction projects, in terms of their design, construction procedure and operation and the effectiveness of their project management has to be rigorously measured and evaluated at every stage in their development. The natural extension of this requirement was to engage the whole of the construction delivery team in the process with the quality and acceptability of the final project being the focus throughout.
The whole of the construction industry is engaged in the pursuit of continuous improvement and relationships that add value. Accordingly most construction project procurement contracts require performance measurement, benchmarking and objective evaluation. But generally, the lessons learned will be passed on only if the team secures the next project. Therefore clients, architects and contractors have entered into longer term contracts, framework contracts and contracts that shift traditional risks between the parties (eg. Private Finance Initiative). Through these, performance evaluation becomes an on-going process with improvements and innovation being incorporated into subsequent projects.
Measuring the degree of success with which a project is delivered requires an evaluation of outcomes versus planned objectives. But outcomes will include in-process as well as end results and individual as well as team performances. They will include fundamental aspects such as cost and time and their predictability as well as the many aspects that describe quality. They will include the communication of instructions as well as their observance and application. They will include not only the functionality and serviceability of the delivered project but also an assessment of the satisfaction of the project users. But there are also questions to ask:
- Was the project brief applicable, properly understood and transferred to an appropriate design?
- Was the design conceived and developed with whole-life costing as the influence?
- Were the project constraints, strategies and objectives appropriate or prohibitive and how was its value perceived?
- How does the project and its participants compare with others?
- Were lessons learned from earlier projects and is continued learning evidenced?
A post project evaluation covers the full scope of the project and is analytical, performance based and a feature of government thinking. It is an essential part of client procurement strategy as well as a tool for building relationships through evidenced continuous improvement. In other words it is about doing good business and retaining good business.
The measurement of performance is essential to determine if the planned improvements have been achieved and to learn lessons for future projects. In essence they are required to:
- Ensure that planned improvements in quality, cost and time are achieved
- Compare with similar projects
- Do things better with future projects
- Determine how integrated supply teams compare with others
There are many different types of construction contract and they can be used to construct a single project or several projects, to order works of construction, maintenance and refurbishment or to commission project design, operation and hand-over. They include measurement, cost-reimbursement, term and DBFO contracts amongst others, and all require performance evaluation processes to assess their end-product and/or in-progress value. The nature and timing of the evaluations may differ but their purpose will be the same. That is to ensure that planned improvements have or are being achieved, that lessons have or are being learned, that integrated teams have or are performing better and that performance is comparable within the industry.
The successful completion of any construction project requires clients, architects, contractors, suppliers, manufacturers and other parties to work co-operatively towards the achievement of the stated objectives. Several contracts will link these individuals and the successful completion of the duties and responsibilities assigned by these contracts require good working relationships.
The degree of success with which the project is delivered will greatly depend on the effectiveness of the relationships between the parties whether linked by contract or not. Conversely, the degree of success with which the project is delivered will have a significant bearing on the continued health of those relationships: Hence the importance of establishing and quantifying the project outcomes and gauging them against the original objectives; a post-contract evaluation.
- List the key aspects introduced by Achieving Excellence in Construction and describe what is meant by performance measurement.
Section 2 Contract Evaluation Processes
Aims and Objectives
At the end of this section you should be able to:
- Identify the headline indicators of performance and the elements of performance measurement used to construct a KPI
- Describe the use of KPIs within project evaluation procedure and be able to locate industry values to enable comparison.
Contract evaluation is about undertaking a review of the contract outcomes or performances relative to the original objectives or comparable contracts. Whether a post-completion or in-progress evaluation it must be based on evidence including, amongst other things records, reports, product data and assessments of indicative performance (performance indicators). To enable and encourage ready comparisons, known as benchmarking, the construction industry maintains information on construction performances (termed key performance indicators) that are representative of the industry or examples of best practice.
A performance indicator is the level of performance achieved relative to an original objective. It can be determined by the measurement of a specific outcome or the systematic assessment of data or performances. It can be compared against an established project objective or pan-industry values that are researched and updated annually. Above all, performance indicators can be a tool for assessing ‘value’ and encouraging continuous improvement.
Traditionally, a project’s successful delivery has been assessed by considering the final values for time, cost, quality and client satisfaction. These values would have been compared with the stated objectives of the project and a report compiled accordingly but a positive result did not automatically indicate best value, value for money or best practice. Indeed the construction industry had a poor record in this respect such that the government of the day asked Sir Michael Latham (1994) and later Sir John Egan (1998) to consider how the industry should change. Their recommendations initiated evolutionary change across the industry. Part of those recommendations contained ten headline performance indicators by which they suggested the industry and individual companies should be assessed.
Key Performance Indicators
The ten areas to be represented by headline or key performance indicators (KPI) reflected either project or company performances.
- Construction cost
- Construction time
- Cost predictability
- Time predictability
- Client satisfaction in the product
- Client satisfaction in the service
Clients who promote projects and services for use by others (eg. central and local government) generally add a further measure of customer satisfaction.
Performance Measurement – Headline Indicators
The measurement of performance means actively checking performance against the above objectives throughout the life of the project. This promotes learning and enables comparisons with the industry as a whole or on a project by project basis within a single client’s portfolio or organization. The scope of these comparisons will be:
- Pan-industry: time and cost predictability, quality (presence of defects), safety (accident frequency rate), client satisfaction.
- Project by project: final cost versus estimate, product (number of design changes), customer satisfaction
Performance Measurement – Individual Indicators
Key performance indicators are closely aligned with the project or service objectives and are essentially a headline indicator of performance. But each KPI will be the product of separate measurements or assessments of different aspects of the indicator. For example, the KPI for safety could be a combination of measured performances such as LTIFR (Lost Time Incidents Frequency Rate), RIDDOR (Reportable Incidents, Diseases and Dangerous Occurrences Rate), AFR (Accident Frequency Rate), recorded near misses, safety audit results alongside assessed performances of safety training, safe working practices, public safety, supply chain management and the application of CDM (Construction Design and Management Regulations).
These measurements will obviously reflect (or are influenced by) different aspects of safety performance. Some will reflect on how robust the H&S (health and safety) processes are, some on how effective the H&S management is and some on how successfully the management of public safety has been achieved. In addition, it is clear that some of the above are required by law, some required by internal safety procedures and some are a reflection of following good practice by being a Considerate Constructor (a construction site registration scheme to improve the image of construction - see web reference). Project teams do not necessarily record all of these aspects on every project so it is important to agree what is to be measured and how from the outset.
Constructing a Performance Indicator
As outlined above, the performances and outcomes can be measured or assessed and they can relate to different aspects of the headline indicator. However, to be incorporated into a single performance indicator calls for criteria that ensures consistency of measurement and the combination of the related aspects though different units of measurement cannot be automatically combined. For example, LTIFR (number of lost time incidents per 1000,000 working hours) cannot be added to RIDDOR (number of reportable incidents per 100,000 working hours) and AFR (number of accidents per 1 000 employees) even though they all relate to H&S Processes. Therefore a means of scoring the performance that they collectively represent, termed the level of satisfaction, is required.
A Sample Scoring Mechanism is shown below.
| Satisfaction Level|| Evidence of Performance|| Score|
| Fully Satisfied || All requirements in place to full satisfaction|| 10|
| Well Satisfied|| Full satisfaction with most items but some just satisfactory|| 8|
| Satisfied|| Some items fully satisfactory, some satisfactory, some not satisfactory|| 5|
| Dissatisfied|| Some items satisfactory, some less than satisfactory|| 2|
| Fully Dissatisfied|| All items dissatisfactory|| 0|
To give effect to the above table a written description must be in place of what, in respect of H&S processes, correlates to each level of satisfaction in the scoring system. For example, to be Fully Satisfied the table could read as follows:
| Well Satisfied|| LTIFR, RIDDOR, AFR = zero Safety Audits satisfactory, no remedial actionsAll Near Misses reported and acted uponAll workers informed of and following safety requirementsFull on-site training and consultation programme in place Full adherence to safety requirements by Supply Chain Welfare facilities appropriate and well maintainedSite given Considerate Constructor award (or similar)|| 10|
Similarly, descriptions are required for each of the other levels in this satisfaction scale for H&S Processes.
In a similar way, descriptions would need to be prepared for each level of satisfaction in the scales for H&S Management and Public Safety.
Therefore, with each aspect of safety performance being assessed in the range of 1 to 10 then to report an overall performance under the headline Safety KPI just requires a simple average of the assessed performances for H&S Processes, H&S Management and Public Safety.
Each headline KPI was given equal importance in the Latham and Egan reports but there may be reasons to apply a form of weighting depending on what are seen as the critical aspects of the project. But caution must be exercised in pursuing such weighting as there is a danger of people focusing on the critical aspects to the detriment of the others.
The project sponsor or client instigates a project evaluation process to check on how well a project is performing. This process involves both in-progress evaluation as well as post-completion evaluation and will concentrate on:
- assessment at major decision points (gateways) if the project can proceed
- on-going assessment of performance against planned outcomes
- assessment of what is working well and any opportunities missed
- assessment of problem solving or innovative techniques
- assessment of how well the project was managed
- comparison of performances against KPIs
- assessment of the value of team working (including full supply chain
- assessment of successes and failures in a blame-free culture
As part of the agenda for change, Constructing Excellence developed KPIzone, an array of key performance indicators that offer a benchmarking tool to the industry, see web reference. By using this tool, builders and contractors are able to measure their performances in a consistent way, demonstrate improvements in cost, time and quality and make comparisons with their peers. The Centre for Construction Innovation developed their KPI Engine, see web reference. This is a similar tool but which was extended to meet the desire of clients to measure and compare the performances of contractors and consultants and to drive continuous improvement within best practice benchmarking clubs.
There are four key stages in the benchmarking process:
- Planning: confirming the activity and performance to be measured and the partners with whom to compare
- Analysis: identifying differences between the existing process and that of the partner and agreeing targets for improvement
- Action: planning to achieve the improvements and monitoring progress
- Review: evaluating the process once the improvements have been introduced and rectifying any shortcomings identified
Further information on the use of KPIs can be found on the websites below.
- In constructing a performance indicator under the Safety Headline KPI, outline the levels of safety performance that would be described as ‘Fully Satisfactory’, ‘Satisfactory’ and ‘Unsatisfactory’ in relation to Public Safety on a construction project.
Section 3 Movement for Best Value
Aims and Objectives
At the end of this section you should be able to:
- Recognize how the drive towards best value has changed the industry’s approach to business.
- Recognize how best value has extended across the whole supply chain.
- Link the development of good working relationships with the effective use of in-progress and post contract evaluations.
In 2002 the Office of Government Commerce updated its initiative 'Achieving Excellence in Construction' to introduce a requirement for central and local government clients to seek 'best value'. Therefore, all clients who procure construction projects had to develop procedure for focusing the project delivery team on the project outcomes (ie. the client's business objectives) and finding solutions for problems long-associated with construction projects. The organisation foremost in this was The Highways Agency (HA) which describes best value as, “delivering their business objectives at the lowest affordable cost whilst achieving continuous improvement.”
The principles that underpin the client business objectives are:
- To have fewer defects
- To have predictable final costs and completion time
- To spread good practice
- To be more innovative
- To achieve better value for money and speed of construction
Further developments have included a drive towards the total integration of the supply chain. This aims to encourage ‘design for construction’ that delivers process efficiency in conjunction with project quality. Total integration also ensures that architects, builders and contractors collaborate with manufacturers and suppliers to bring best value through the optimization of installation costs and delivery performance. It has required clients to introduce extensive procedures for vetting designers and contractors before being engaged and rigorous in-progress performance measurement and post-project evaluations once engaged. All parts of the construction industry, including housing, environment and civil engineering are involved.
The initiative is supported by the Department for Business, Innovation and Skills which commissions Constructing Excellence to publish KPIs for pan-industry comparison. These are wide-ranging, covering such aspects as housing, non-housing, environment and infrastructure. They are also enhanced by Specialist KPIs (construction consultants, M&E contractors and construction products) and Design Quality Indicators (DQIs) that are for assessing the quality of buildings by those involved in the production of the built environment.
Clients within construction are adopting holistic models to continuously evaluate the performances of themselves and those they engage. One good example is the Highways Agency who have published processes for the pre-engagement assessment of consultant and contractor organizations and for in-progress assessment during their commission. Their ‘Capability Assessment Toolkit’ and ‘Motivating Success’ processes are outlined here to highlight the essential features of such an holistic approach. The thoroughness and purpose of this approach is mirrored by many major construction clients.
‘Capability Assessment Toolkit’ is a framework by which an organization can be assessed in terms of its potential to deliver the Highways Agency’s objectives. ‘Motivating Success’ is a toolkit for performance measurement that requires the whole supply chain to be engaged in value engineering as well as seeking innovation and opportunities for adding value. These contributions are recognized and included in the scoring of their in-progress and post contract evaluations. Both toolkits can be accessed through the web references below.
Everyone in the construction industry is striving for improved efficiency in design and construction. This includes better use of time, IT, systematic techniques, standardized products, unified design and construction, updated skills and systematic learning, innovation and improvement. But not every contractor has the capability, connections or capacity to engage in this full programme. So a client must be confident that a contractor’s organization and approach to business will deliver results in line with their declared policies, objectives and the requirement to deliver best value.
Research has shown that there is a clear correlation between having effective management systems and continually producing improved organizational results. Several business models emerged from the research and any of these can be used to continually assess an organization, its operations and individual project performances. Therefore, many clients are encouraging contractors and consultants to adopt such business models and to share their assessment results so that comparisons of capability can be made with other contractors and consultants.
One such business model, the European Foundation of Quality Management Excellence Model (EFQM Excellence Model) evaluates what an organization does and what it achieves. It has been adapted by the Highways Agency to produce their Capability Assessment Toolkit (CAT3), a framework of 24 indicators that are aligned with excellent business performance; see web link to indicators. To meet the exacting criteria, a contractor must have a clear vision, measurable goals, effective leadership, systematic procedures, appropriate results, periodic self-assessment and evidence of constant learning and benchmarking.
Meeting the requirements of the Highways Agency’s initial audit assists the partaking contractor’s qualification for tender invitation. Once engaged in the contracting process, the contractor will feed the results of ‘Motivating Success’ back into the company’s business model as evidence of continuous movement towards business excellence. This in turn has a significant effect on a contractor’s chance of being invited to tender for further works or to negotiate for repeated works.
Consultants wishing to be commissioned by the Highways Agency and to be invited to undertake further commissions will undergo the same procedure.
Supply Chain Management
Constructing Excellence is leading the radical change in the construction industry with a focus on value for money, profitability and reliability through best practice and innovation. It also recognizes the need for totally integrated supply chains and modern procurement strategies. They have produced industry guidance for effective ‘Supply Chain Management’, see web link. This expects contractors and suppliers, in addition to their normal business strategies, to share a long-term objective of delivering added value to clients. In line with this, each contractor is required to select suppliers only from the list of those with proven capability and capacity and who have demonstrated a commitment to openness and collaborative working towards the stated goal.
Therefore, contractors generally have a very restricted list of suppliers with the status of ‘preferred’ who have shown that they meet the competence criteria and share the collaboration ethos. Suppliers granted this status will only maintain that position by participating in six-monthly or annual reviews (1800 or 3600). The results of these can instigate further improvement and also feed back into the contractor’s business model and so demonstrate to their client that they exercise positive supply chain management.
More information on supply chain management can be found in the constructionsite unit shown below.
Best value requires all parties to look at every element of cost in the complete production cycle, to consider its necessity and to investigate ways of eliminating or minimizing its effects. This requires all parties to be open about their costs, their basis and to declare their required margins. Clients seeking best value solutions should accept the need of contractors and suppliers to produce reasonable margins and to work with them to eliminate the risks that endanger them. By collaborating actively in this process clients can help reduce the whole–life cost of their projects by creating conditions that remove contingency costs and minimize or eliminate risk, waste and over-engineering.
Best value exercises require careful judgment and constant monitoring to ensure that assumptions are appropriate throughout the project and applicable for use with subsequent projects. Accordingly best value exercises are closely linked with performance indicators and post contract reviews and their success is clearly linked with fruitful long-term relationships between client, contractor and supply chain.
It can be seen from the above that clients do not evaluate their projects by cost, time and quality alone. They want predictability of time and cost and to be confident that an organization exercises strong leadership and effective management based on systematic procedure. They need KPI comparisons to be assured of value for money, best value and best practice performances as well as clear evidence of continuous improvement and innovation. They want to be kept informed at every stage of the project and to be assured that their own customers will be satisfied in the project and how it will be delivered. Above all, clients want contractors and their supply chain partners to engage with them in delivering their agenda and to take responsibility for the services they mostly influence.
It is easy to see how good working relationships will develop by meeting this agenda and how valuable co-operation and collaboration are in the process. It is also possible to recognize those elements of a post contract evaluation or an on-going evaluation that will help set the standards for the next project and further strengthen those relationships.
- Describe the benefits of having a fully integrated supply chain.