List of FAQ

 

0.        General

0.1.      What does CAF – Common Assessment Framework – mean

0.2.      Why should I use the CAF

0.3.      What are the advantages of self assessment for an organisation and its stakeholders

0.4       Are there any "footbridges" between CAF and EFQM

0.5       Is it useful to use CAF in an organisation that is ISO 9000 certified

0.6       In how far can we adapt CAF to the specific context/ environment of the organisation
To which extent are the examples compulsory

0.7       Is applying CAF enough to improve the performance of the organisation

0.8       How long does it take and what is the cost of a self-assessment process based on CAF

0.9       How can I obtain information about CAF in my country

 

top

          

1.        Using the tool in your organisation

1.1.      Which organisations can apply CAF

1.2.      When is the right time to apply CAF

1.3.      Is the implementation of CAF feasible for a public organisation confronted with serious problems in its functioning

1.4.      How to convince colleagues and managers

1.5.      Are there success factors and what are the pitfalls

1.6.      How much and what type of external support are necessary/ needed
What is the role of external assessment

1.7.      What does consensus mean and how to achieve it

1.8.      Is CAF an objective instrument for self-evaluation

1.9.      Can CAF be applied in a unit? If so who is considered the leadership (Criterion 1)?

1.10     Can I eliminate examples and sub-criterions in the self assessment process

1.11.    Criterion 8: Society Results

 

top

   

2.        Scoring

2.1.      What is the role of scoring

2.2.      Do I have to score on every sub criterion

2.3.      Do I have to score on every example

2.4.      Should we score the sub criterion (like a global score) or should the sub criterion score be the result of the examples average

 

top

          

3.        Bench learning

3.1.      We have applied the CAF and wish to compare with other organisations. How do we proceed

 



0. General

 

0.1. What does CAF – Common Assessment Framework - mean?

 

CAF is a common European quality framework that can be used across the public sector as a tool for organisational self assessment.  It has been jointly developed under the aegis of the Innovative Public Services Group (IPSG), an informal working group of national experts set up by the Directors General in order to promote exchanges and cooperation where it concerned innovative ways of modernising government and public service delivery in EU Member States.

 

top

 

0.2.Why should I use the CAF?

 

CAF is an excellent starting point for an organisation in its way to quality. Being a self-assessment tool it also has the advantage of involving the people of an organisation and of bringing forward their own views.

Furthermore, it shows the following benefits:

-          it is relatively easy to handle,

-          it demands less resources (the use of the model is free of charge) and

-          it introduces a common language to organisations wishing to benchmark using CAF.

 

top

 

0.3.What are the advantages of self assessment for the organisation and its stakeholders?

 

The procedure of self assessment with the CAF is a mean to help public sector services to use quality management methods, in order to improve – effectively, efficiently and continuously – their performance. It provides an assessment focused on evidence as well as a tool to achieve consistency of direction and consensus among all the stakeholders on what needs to be done to improve the organisation. 

 

Additionally, the CAF is a mean of measuring progress over time through periodic self assessments and provides a link between goals and supportive strategies, in order to focus improvement activities where it is most needed. Finally, the CAF is a mean to create enthusiasm and inspiration among the civil servants by involving them in the quality improvement process.

 

top

 

0.4.Are there any "footbridges" between CAF and EFQM?

 

The basic design of the CAF was developed in 1998 and 1999 on the basis of joint analysis undertaken by the European Foundation for Quality Management (EFQM), the German University of Administrative Sciences Speyer (which organises the International Speyer Quality Award for the public sector in the German-speaking European countries) and EIPA. The CAF is thus based on the Excellence Model owned, developed and promoted by EFQM but self-assessment using CAF is a less rigorous process than a full assessment against the EFQM Excellence Model. For public organisations (or any organisation with a public mission) it is therefore perfectly possible and even recommended to start with CAF – especially designed for public sector organisations – and to move to EFQM at a later stage.

Furthermore, EFQM rewards organisations having used CAF and being able to provide evidence of improvements being implemented through its first level C2E “Committed to Excellence”

 

top

 

0.5.Is it useful to use CAF in an organisation that is ISO 9000 certified?

 

It is certainly possible to use the CAF for self-assessment in that context. CAF is a quick and relatively easy-to-use tool which can be used alone or in conjunction with other quality management tools as a "health check" or as a means of identifying areas for improvement on all aspects of the functioning of an organisation. CAF is the bracket around all other quality management tools in use. CAF asks if your organisation runs process management, ISO, agreement on objectives etc. In the case of an organisation that is ISO 9000 certified, the use of CAF could broaden the view on the organisation and also help to identify possible partners for benchmarking projects.

 

top

 

 

0.6. In how far can we adapt CAF to the specific context/ environment of the organisation?

- To which extent are the examples compulsory?

 

The basic structure (9 criteria, 27 subcriteria, and scoring system) should be respected by the CAF users, but the examples and the process of self-assessment are free/ flexible. „Translation“ and adaptation“ of CAF to the organisation or the sector that is promoting self-assessment are therefore not only possible but desired and helpful.  In several Member States such initiatives[VS1]  have been taken.

The purpose of the examples is to help an organisation finding relevant elements for self-assessment. Even though in some approaches (e.g. Germany) scoring is done for each and every one of the examples, the suggested way of proceeding is to use the examples as a source of inspiration: you use those examples you consider relevant and you add examples who are especially relevant for your specific context/ environment/ organisation (see also 2.3. and 2.4.)

 

top

 

0.7. Is applying CAF enough to improve the performance of the organisation?

 

CAF will help you to establish strengths and areas of improvement and is a powerful instrument to initiate a process of continuous improvement. It gives you an initial impression of how the organisation performs, it allows for the improvement actions to be focused where they are most needed and it shows how the organisation improves over time, if the CAF is applied a second time.

It should therefore be seen as the start of a continuous improvement process and will not in itself improve the performance of the organisation. The most important thing to be done as a result of the self-assessment is to take measures. If CAF should make sense it is necessary to make an action-plan on the basis of the self-assessment results and to take measures!

 

top

 

0.8. How long does it take and what is the cost of a self-assessment process based on CAF?

 

The survey on the use of CAF held in 2005 amongst 131 organisations shows that 2 to 3 days seem to be rather short to do a decent self assessment whilst 2 working weeks or more is too long. The ideal time schedule for a CAF self assessment does not exist because too many variables are at stake such as the objectives of the management, the time and resources invested, the availability of data etc. But to the majority of organisations a duration of 2 to 5 days looks appropriate.

The very far majority ended the CAF application in 3 months, including the preparation, the self assessment itself, the drawing of conclusions and the formulation of an action plan.

Three months seem to be an ideal lapse of time to stay focused. Taking more time raises the risk of a weakening engagement of all parties involved. Furthermore, the situation might have changed in between the start and the end of the self assessment. In that case, the diagnosis isn’t accurate anymore.

 

top

0.9. How can I obtain information about CAF in my country?

 

On these pages, you will find a list of the national CAF correspondents.

Furthermore, EIPA gathers information on CAF users in Europe and has carried out two surveys on the use of CAF in Europe (one in 2003 and one in 2005. All this information is made available on the website.

If you wish to register yourself, please contact the CAF resource centre to obtain a login and password.

 

top

 

1.      Using the tool in your organisation

 

1.1. Which organisations can apply CAF?

 

CAF is an excellent starting point for any (public) organisation in its way to quality. Previous experience with / knowledge of „quality management“ is not required. A certain level of maturity however helps to deal with quality issues and to be critical about oneself.

 

top

 

1.2. When is the right time to apply CAF?

 

It is always the right time to apply CAF.  It can indeed be used at any time for different purposes: from simple „health check“ to a full action plan / internal reform programme.

The CAF should however not be used for „nasty“ purposes (cutting back on finances or staff) or in periods of serious „turbulence“.

 

top

 

1.3. Is the implementation of CAF feasible for a public organisation confronted with serious problems in its functioning?

This is not the recommended starting point (see also 1.2.)

 

top

1.4. How to convince colleagues and managers?

 

One should be conscious of the fact that convincing everyone is difficult or impossible (at least in the beginning).

Key players – a critical mass – need to be won over (e.g. personnel managers; financial managers; „leaders“).   Without the full support of the management, you should not consider starting such an exercise.

A way to obtain their engagement is to focus on “What is in it for people / management”: find convincing arguments. Subsequently support can develop/grow at a larger scale (quality rarely starts in the whole organisation....).

Please note that support for quality very often evolves from a sceptical approach/ attitude to a certain degree of enthusiasm at the end of the self-assessment.

 

top

 

1.5. Are there success factors and what are the pitfalls?

 

An obvious success factor is the decision by management to introduce self-assessment, with a view to dealing with the results and for the subsequent follow-up. If not based on a clear decision, it is doubtful that self-assessment will be a long-term success. But precisely because of the process, i.e. a continuous improvement cycle, and through concrete realisation, quality management becomes noticeable and evident. The CAF can create and promote an appreciation for quality.

Through self-assessment, communication and information problems often come to light which generally can be cleared up very quickly.   Indeed, a certain administrative culture is a condition for meaningful self-assessment with the CAF instrument.

Openness, tolerance, trust and the certainty that those assessing and those being assessed do not have to be afraid of disadvantages and restrictions on account of what they said during the assessment rounds, is essential.

Nearly every change meets with resistance. Positive changes are often blocked, even if they entail advantages for the staff concerned. Resistance usually arises from fears that people do not (dare to) express. Any change, however small, is at first considered a threat. It can only be dispelled through open communication and by creating the greatest possible transparency. That is why providing detailed information about the purposes from the very beginning is of great importance if self-assessment is to be a success.

 

top

 

1.6. a) How much and what type of external support are necessary/ needed?

 

CAF is a self-assessment tool, but experience tells us that assistance and support are needed in most organisations. Many administrations need support throughout the whole process (not just initial training). For support you might contact your national CAF correspondent (see also 0.9).

If you go for more rigorous and precise assessment, external assessment will without any doubt be required, unless you have sufficient in-house expertise in the field of TQM.

 

top

1.6. b) What is the role of external assessment?

 

As a neutral person, an external consultant can lend unbiased and impartial support throughout the entire self-assessment process. Qualifications: methodological know-how (e.g. presentation skills, project management etc.) and experience with the CAF instrument.

 

top

 

1.7. What does consensus mean and how to achieve it?

 

Consensus means above all shared understanding. It can be achieved through for example

-          intense group discussions until you reach agreement

-          arbitration by the chairperson of the self-assessment group

The self-assessment group can focus on

- the background to different views/assessments

- the evidences/indicators used by different group members to support their assessment

- the scores attributed by different group members

There is no best way to achieve consensus but one should not put the major focus on the scores.

By reaching a consensus, the self-assessment group creates an added value to the individual appreciations and puts them on a higher level.

 

top

 

1.8. Is CAF an objective instrument for self-evaluation?

 

CAF contains both an element of objectivity (fact-based approach) and subjectivity (individual knowledge and experience count). Balancing and bringing both together is the key!

 

top

1.9. Can CAF be applied in a unit? If so who is considered the leadership (Criterion 1)?

 

Yes, CAF can be used in a unit if the unit is some sort of autonomous and manages all 9 criteria of the CAF (e.g. HRM, Strategy and Politics etc.). Before starting the self-assessment it is necessary to define who is considered to be the leadership (leader of the unit or leader of the whole organisation). 

In addition, it is also necessary to define the different stakeholders such as the citizen/customers, the partners, etc.

 

top

 

1.10. Can I eliminate examples and sub-criterions in the self assessment process?

 

The basic structure (9 criteria, 27 subcriteria, and scoring system) should be respected by the CAF users, but the examples and the process of self-assessment are free/ flexible. See also Q 0.6.

 

top

 

1.11. If a territorial based organisation, like a local authority, has in their core business all of the initiatives established on criterion 8 how to evaluate this criterion?

 

In that case, the results should be evaluated in the sub criterion that evaluates the goal achievement: 9.1

 

top

2.  Scoring

2.1. What is the role of scoring?

 

We have noticed that there is a strong focus on scoring in most administrations although some decide not to use scores. Furthermore, many administrations tend to score too high.

Methodological support may help to overcome these pitfalls. The score-system is according to the PDCA-cycle. The score is asking for steps of implementation, not for marks.

The scoring system is meant to be simple (for example in comparison with EFQM) but there are many grey areas and uncertainties. The scoring system is therefore one of the issues at stake in the current process of CAF revision (to be finalised in 2006).

 

top

2.2. How do I score (1 or 1.1 or 1.25)?

 

Often it is considered useful to use one decimal behind the comma. For example if you want to adjudge a score for the results criteria, and you have many examples in depth, you would adjudge a score of 2.9, whilst if you have only a few examples, you would opt for 2.1 or 2.2

 

top

 

2.3. Do I have to score on every sub criterion?

 

The basic structure with 9 criteria and 27 subcriteria should be respected. In that perspective, it is considered that every sub criterion has its place in the holistic view of the organisation and should therefore be taken into consideration. (cf. Q.1.9. and 0.6.)

 

top

 

2.4. Do I have to score on every example?

 

The purpose of the examples is to help an organisation finding relevant elements for self-assessment. The suggested way of proceeding is to use the examples as a source of inspiration: you use those examples you consider relevant and you add examples that are especially relevant for your specific context/ environment/ organisation. The discussion on the evidences related to these examples will help you to establish a score for the sub criterion as a whole. We therefore do not recommend you to score on the individual examples but it is possible.

 

top

 

2.5. Should we score the sub criterion (like a global score) or should the sub criterion score be the result of the examples average?

 

The score of the sub criterion is based on the evidences found when considering the different examples relevant for your organisation. This should allow you to give a global score. You can also choose to score every example and the average then generates the sub criterion score.

See also above (2.4.)

 

top

 

3. Benchmarking

 

3.1. We have applied the CAF and wish to compare with other organisations. How do we proceed?

 

The CAF certainly is not a rigorous instrument that could be used for a hard performance benchmarking. Also, in most cases the use of the CAF will be without external support, so the self-assessment is not a 100% reliable for comparison. However, the CAF will support a process of learning - and the focus is indeed on sharing experiences and on learning, and not on benchmarking against a given standard or against the "best in class". Organisations will mainly have the opportunity to find partners that they would like to share experiences with in order to mutually improve their operations and results. We will also include on our CAF website some "best practice organisations" that have used the CAF and have had an external evaluation - so these self-assessments will be more reliable. It is also hoped that over time we will learn more about benchmarking with the CAF and that we will be able to offer a "standard procedure" for benchmarking with the CAF. This should indeed be added to our website in the near future.

 


top
17-Jul-07