Introduction
Evaluation of educational programs
When we talk about “programme evaluation,” we are referring to a measure that is made up of two parts: the EVALUATION and the EDUCATION PROGRAMMES.
The evaluation means to make judgments or assessments in order to make decisions based on them.
An educational programme can be described as a collection of personal, methodological, material and temporal resources, for an educational or competence objective of persons in a concrete social reality.
Programme evaluation can therefore be understood as:
Planned and systematic measures applied to a targeted educational programme. The intention is to provide an assessment of the value of part or all of the development, planning or impact of this programme, and to obtain useful information for decisions on the effectiveness, continuity or, if necessary, adaptation of the programme.
Reasons to evaluate
Implementing a programme requires the participation of people, investing time and resources, and influencing our tight social environment.
Carrying out a programme therefore means making an investment, mobilizing resources that are not available for other activities during the remaining time. It is important to ensure the long-term impact of the programme and to forecast the effects and, of course, to constantly review them.
An evaluation makes it possible to identify the benefits of the programme both for persons (observers, pupils, teachers, supporters, etc.) in the institutions involved and, in the environment, / network. It is also a valuable tool to justify the investments made.
In summary, one can cite the following reasons for carrying out an evaluation:
- Efficiency of public funds
- Faced decision-making basis
- programme improvements
- Review and document short-term and long-term effects
- sustainable competence development among clients and students
- Opportunities for adaptation to other educational contexts
The evaluation identifies aspects that do not work at the expected level. It also allows you to identify and highlight the issues that work correctly in the programme. Finally, even the problems that were not included in the actual programme can be identified, but they can be helpful in improving the effectiveness of the programme.
The evaluation can thus be applied to:
- Identifying students’ needs at the beginning of the programme.
- It allows the observation of student performance during the performance of the programme.
- Allows the evaluation of the goals achieved by the students.
The evaluation also makes it possible to justify the investments made in the context of effectiveness. This aspect is essential if one assumes that projects and programs are financed by public money.
A clear and transparent evaluation, which proves the efficiency and effectiveness of a programme, makes it easier to reach the support of public authorities.
Evaluation plan
The process of programme evaluations depends to a large extent on the characteristics of the programme itself and the context. It thus affects the chosen evaluation model, which is used as a reference. However, it is most likely in most evaluation procedures.
- approach
- design
- information collection
- analysis
- report
In order to be able to carry out an evaluation, some questions must first be clarified
Imagine that you are asked for an evaluation of a programme in your working context. There may be questions like:
- Who should carry out the evaluation?
- Why do you want to rate the program?
- What exactly should be evaluated (the whole programme or a part of it)?
- Which side and boundary conditions exist?
- Can the evaluation be carried out under the current conditions?
Other questions that may arise at the beginning are:
- Which elements must be considered?
- Are there any indicators for action?
- Which quality criteria should be used?
- What information is available beyond the context of the program?
These are some of the issues that need to be resolved before starting the evaluation.
Realisation of the programme
When implementing a programme, we can describe two situations. First, that the programme itself has planned or includes the evaluation, and second, that this is not the case.
In the first case, it will be possible to implement an evaluation based on the principles set out in the actual programme. This evaluation can be complemented or improved by a more general approach, which is not included in the programme but can be used to improve the assessment. In the second case the evaluation should be based on the characteristics and the development of the programme, although it is not an actual part of the programme itself.
Suppose a programme is started in the immediate vicinity to increase the rate of waste recycling. In the same programme, there was a phase on impact assessment and improvement that can be achieved with the programme. At this stage it will
be expected to increase the recycling rate by 10% after applying the programme. This could be regarded as unsuccessful and even considered as a failure in other parts of the city, as the considered neighbourhood has its own characteristics. However, in this case, 10% can be regarded as a success, because in the district under consideration, which is subject to face social tensions, such improvements in urban development have never been achieved.
With such an interpretation, the evaluator sets the process the effectiveness criteria in an appropriate context, which is also influenced by aspects outside the programme. If this context does not materialize, distorted and unrealistic valuations can arise. Therefore, the programme has the advantage in its actual setup the evaluation is included.
Regardless of whether it is included in the programme or not, evaluations always have to consider which solutions are proposed and which alternatives are proposed. This gives you the opportunity to choose the most effective way.
The success of an evaluation
The success of an evaluation depends to a large degree on process planning and organization. It is essential that the gathering of information becomes notorious in planning. The evaluation team must have the data collection deadline (both initials and, if appropriate, later) as soon as possible, and ensure that the data collection procedure (questionnaires, observations, tests, etc.) is in place and authorized to do so.
Once the information has been gathered, the evaluator has sensitive data that is subject to confidentiality. As with any other type of personal information, the data, legal privacy policies and certain ethics codes are also subject to evaluation by evaluators. An adequate storage system together with the absolute guarantee of confidentiality are two essential requirements that the evaluator needs in terms of personal data.
Immediately after receiving the information, the analysis will begin. This is done in compliance with the evaluation goals and the requirements for the evaluation. The type of analysis and the technical resources used depend on the characteristics of the data, the characteristics of the programme, the resources available for evaluation, and so on.
In a programme evaluation, the data collected can be both qualitative (textual) and quantitative (numerical). It is also possible to carry out the analysis qualitatively (evaluation of the information quality) or quantitatively (evaluation of the measures determined). If both options are crossed, four analysis categories are named:
- qualitative analysis of qualitative data
- qualitative analysis of quantitative data
- quantitative analysis of qualitative data
- quantitative analysis of quantitative data
Reporting
Finally, the team of evaluators must translate all their work into the appropriate report. Depending on the scientific needs of the evaluators or other audiences for whom the report is being produced, it will be written in one format or another. Accordingly, one and the same report is not written for, for example, the parents of pupils, teachers’ committees, supervision teams or politicians of the ministry of education. Despite the different ways in which the report can be adopted, as well as the different expressive styles, the reports can be grouped together in a general structure that includes a cover sheet, a summary, information on the programme being evaluated, the method of evaluation and the results achieved. The discussion, which can be deduced from the results, ends with a series of conclusions and recommendations.
« Exercise!
Tell your parents (or your partner). Imagine how you are asked what you do in this class. And yet you have the idea to tell them about a programme evaluation, right? If you do, he / she will stare at you and ask, please explain more exactly what an evaluation is.
Try to explain that he / she has never heard of evaluations and has no relation to them. Write down the explanation.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License
The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.