Reliable and accurate data are needed to raise awareness about the magnitude of road traffic injuries, and to convince policy makers of the need for action. Robust data are critical in correctly identifying problems, target populations, and prioritising risk factors. It is the foundation of monitoring and evaluation. Road safety data will be used by a variety of stakeholders (including policy makers, the police, transport departments, civil society organisations and the media) and is paramount in persuading political leaders that road traffic injuries are a priority issue.
Often data are weak, particularly in low- and middle-income country settings. Visualising the journey to school requires localised data which is often not immediately available. The data tools presented here account for these weaknesses, and provide an evidence base for targeted interventions. Given constraints on resources, data gathering exercises must be affordable, a key premise of this toolkit.
Schools and communities will often have an awareness of the level of road traffic injuries and fatalities among their children. Teachers and parents will often know where children are facing risks from road traffic. Such community level testimony can help in identifying priority areas for interventions.
Official data on injury and fatality localized to a school area is often unavailable, weak or incomplete. This is where both school area assessments (such as the SARSAI methodology) and the ‘Star Ratings for Schools’ (SRfS) app prove effective. They identify the level of risk and the exposure of children to road traffic as they go to and from school. These tools provide a basis for protecting children on their journey to school. The data section in this toolkit provides step-by-step guidance for assessing risk, establishing a baseline before implementation, and for data-led monitoring and evaluation.
Identification of 'high risk' schools
Whilst crashes can occur anywhere and any given time, some locations have more crashes than others. Tackling road safety in these “hot spots” is one of the most traditional approaches to road safety and is known to reduce casualties. Identifying and focusing on these locations makes it possible to set priorities and maximise the impact of limited resources. On average, targeting high risk locations results in an 18% reduction in casualties, and in most cases is cost-effective.
Lists of all schools in the area can usually be obtained from local authorities ideally in spreadsheet format. By analysing secondary data and speaking with the head teacher of schools, identify schools where there is a high rate of road traffic injuries and deaths. Examples of secondary data that may be of use in identifying hot spots are police data, hospital registries, newspaper reports, and school registries.
School Area Road Safety Assessments and Improvements (SARSAI) projects have used the following steps to identify schools with high rates of injury:
Appropriate schools for Amend-type interventions:
SARSAI focuses on schools where minor infrastructural improvements can lead to major reductions in road traffic injuries. If the school is near a major highway, it may not be a suitable candidate for SARSAI, because larger infrastructural changes such as overhead footbridges or major intersection alterations are not within the scope of SARSAI-based projects.
For more on Amend and SARSAI visit www.amend.org
At each school site visit, make a note of surrounding roads, school entrances (whether they are on a main road or side road), footpaths, existing traffic-calming measures, etc to assess whether the school is an appropriate choice for the project.
During the site visit, observe student flow at drop-off and pick-up times. Where are the heavy foot traffic areas? What hazards are the children exposed to on their journeys to and from school? If necessary ask permission to follow students as they travel by foot and make a note of any areas where accidents are likely.
Key features to look out for during sites visits are:
Interview and focus groups (pupils, teachers, parents, community around school)
These interviews and discussions build a picture of what a student’s journey to school looks like. What are the challenges faced by the students, parents and school staff? Details of how to carry out in-depth interviews and focus group discussions will be available soon.
Photovoice is a qualitative method of community-based participatory research which documents real life situations and commentaries surrounding a particular topic through the medium of photographs. Participants are asked to express their points of view surrounding a research topic, in this case road safety, by taking photographs, combining them with short narratives to give more context. Photographs are collaboratively interpreted and discussed by groups and can be used to highlight specific issues of concern by the participants. This then enables researchers as well as other stakeholders to identify the issues faced by the participants and promotes dialogue to develop effective solutions that address specific needs.
Save the Children have developed a 10-step guidance document on how to involve children in needs assessments using Photovoice:
Step 1: Sensitize the community and recruit participants
Step 2: Conduct a workshop to introduce participants and facilitators in order to establish group dynamics
Step 3: Set the project goals (remember objectives should be SMART)
Step 4: Get the children to focus on a specific thematic area (in this case the route to school)
Step 5: Introduce the use of cameras and photography to the children
Step 6: Provide guidance to children on how to “speak out through photography”
Step 7: Discuss strategies for taking pictures with the children
Step 8: Discuss ways to capture stories and develop captions for pictures
Step 9: Plan to exhibit or showcase the children’s pictures
Step 10: Go public through meetings, workshops, media, etc.
Please be aware of the ethical and safety issues surrounding the involvement of children in projects of this nature. Be sure to obtain the necessary permissions at all levels before embarking on such a project.
Safe Kids Worldwide used Photovoice effectively to document the pedestrian-related issues facing children, parents and teachers through a project entitled: “PHOTOVOICE: Children’s Perspectives on Road Traffic Safety”. The project successfully captured on film the environments children faced while walking in Brazil, Canada, China, India, Korea, Mexico, Philippines, Thailand, the United States and Vietnam. A total of 5,743 photographs were taken by students aged 9-14 in the 10 countries. The subsequent discussions and accompanying narratives enabled participants to effectively advocate for permanent infrastructure changes which led to a safer journey to school for the students. Safe Kids Worldwide provided a number of grants to these communities permitting the implementation of permanent infrastructural changes to the risks identified through the photographs. A sample of before and after shots from this particular project can be found here.
Traffic volume studies are conducted to determine the number, movements, and classifications of roadway vehicles at a given location. They can help identify critical flow time periods, determine the influence of large vehicles or pedestrians on vehicular traffic flow, or document traffic volume trends. Similar to pedestrian counts, these are conducted around peak school times at the beginning and end of the school day and at 15 minute intervals at various locations around the school. Detailed instructions how to carry out a traffic volume count can be found here.
Assessment of traffic speed around the school will be vital in determining high-risk areas and the effectiveness of traffic calming measures in reducing speed. Detailed instructions how to carry out a speed survey can be found here.
Monitoring and evaluation is an important aspect of any project and are often overlooked. Monitoring refers to the systematic collection of data regarding the performance of a road safety programme or intervention during or after its implementation. Evaluation involves the analysis of this data to determine the effectiveness of the program.
Monitoring and Evaluation are fundamental aspects of good programme management:
It is useful to build the monitoring and evaluation framework alongside planning the intervention and base it around the main objectives of the project. Early M&E planning allows for preparation of adequate time, resources and personnel before project implementation. It also informs the project design process itself as it requires people to realistically consider how practical it is to do everything they intend to measure.
An agreed M&E framework is essential in order to carry out monitoring and evaluation systematically.
This framework serves as a plan for monitoring and evaluation, and specifies:
Linking project objectives and activities with indicators that can be used to measure processes is a fundamental aspect of planning M&E. This exercise can be aided by using simple templates such as the logic framework (see below) which subdivides project objectives into: inputs, activities, outputs, short-term outcomes, long-term outcomes, impact. In reality the pathway is unlikely to be linear as there maybe multiple outcomes and feedback loops along the way but visualising the project in the form of a change pathway can be useful.
|The long-term change to which the project will contribute, e.g. reduction of child injuries / deaths / disabilities from road traffic crashes, in city X. The aim is usually a general statement about what the project hopes to achieve.|
|Objectives are the specific steps that need to be taken in order to achieve the aim of the project. In other words, the “how” of the project. Objectives, therefore, always start with a verb, e.g. to collect, to test, to produce, to analyze, etc. They should be as specific as possible (remember SMART objectives: Specific, Measureable, Achievable, Reachable, Timebound).|
|These are the key resources needed to support the project. They include human resources, funds, equipment, etc.|
|These are the interventions (or actions) that will be taken that will lead to the outputs. They can include workshops, trainings, infrastructural changes, enforcement, supervision, etc.|
|These are the tangible results of the project such as the number of people trained, the number of workshops conducted, the number of zebra crossings painted, the amount of material distributed, etc. They should lead to the outcomes.|
|The outcomes are the changes that result from the intervention. They may be short, medium or long-term. They should directly relate to the aim/objective of the programme. They can include changes in knowledge and attitudes (short-term), changes in behaviour (medium term) and finally reductions in injuries, deaths and disability.|
Choosing the right indicators to measure inputs, outputs and outcomes will depend on what questions are needing to be answered so having a clear aim fofr the evaluation is essential. The breadth of an evaluation will always be limited by the resources available, but a well-designed, simple evaluation can be as useful as a more complex and costly one.
An indicator is a variable that measures some aspect of a program that is directly related to that program’s objectives. Indicators can be either quantitative or qualitative and will measure magnitude and changes. They should be:
Avoid vague indicators such as “number of people who “know” about speed reduction in the area.” Better to use “proportion of parents who can correctly state the new speed limit surrounding the school” as measured through a parent survey.
Examples of indicators for each part of the framework for the evaluation of a road safety program in a school are given below:
Generally you are likely to need information to track and assess what has changed (both intended and unintended) and understand the reasons for changes - i.e. what factors/organisations/individuals have facilitated/constrained change (including your contribution). Secondary data sources such as police data, vital registries, school registers or hospital data may be available and often provide a good source of outcome data. However, as secondary data is not designed specifically for program needs, it is important to avoid the trap of using irrelevant secondary data just because it is available. Check the relevance of secondary data for:
There are many different types of studies that will help in collecting the necessary data for your indicators – they can be both qualitative and quantitative. Qualitative data such as in-depth interviews and focus group discussions can be used in the formative evaluation to provide information on why an intervention may or may not have worked and whether the program went to plan. There is a hierarchy of quantitative methods for examining the effectiveness of a program ranging from randomised-controlled trials to before-after studies with no control group. Whilst the former is more rigorous and is the gold standard by which evidence is assessed, it is also the most expensive type of evaluation to carry out. Conversely, the before-after study is cheaper but it can only provide weak indicative evidence as to a program’s effectiveness. Using a quasi-experimental design such as a controlled before-after study is often more practical and involves measuring the outcome of interest e.g. vehicle speeds before and after the intervention for both the target population and an control population A control group allows trends that may have been occurring in the population separately from those happening as a result of the programme to be taken into account.
Types of studies that can be carried out are:
|Written questionnaire designed to measure indicators||Systematic, can be given to a large number of people||Not possible to explore answers in greater detail|
|Ask specific information regarding indicators||Flexible, can be done in-person or by phone||Requires interviewing skills so as not to ask leading questions|
|Skilled facilitator mediates a conversation with 6-12 people||Cheap, allows opinions from a larger group at one time||People influence one another, feelings may not be shared honestly|
|Roadside observations of risk factor behavior
e.g. speed, helmet wearing.
|Relatively cheap. Allows estimation of risk factor behavior such as speed, seatbelt wearing rates, helmet wearing etc.||Not always accurate and relies on observation skills of researcher leading to potential bias. Time consuming|
|A self-administered questionnaire which collects data on usual mode of travel to school and road injury||Reliable and valid method of data collection on mode of travel and attitudes surrounding road safety||Time consuming and potentially costly. Need trained researchers. Large sample size.|
The methods used for the evaluation will depend on the aim and the budget for the evaluation. However there may be ways to reduce the costs of your evaluation. Ask yourself the following questions:
To ensure M & E is relevant to your stakeholders it is important that you consider their information needs, as well as your own. You will therefore need to identify the key internal and external stakeholders, and decide how to involve them in the design, implementation, analysis and/or communication of findings. Set up regular project team meetings to discuss M & E results.
This will depend on your evaluation aims and your resources. At the very least you need to be collecting data on indicators at the very beginning of the project (to build a baseline) and at the end of the project. Collection of process and input indicators should be carried out throughout the implementation of the project. Ideally for observation studies it is useful to collect data at the beginning, part way through, at the end of the intervention.
Groups can learn from each other: what was useful, what worked, what tools can be shared, etc. Sharing of problems encountered and how these were overcome can be benefit others and avoid time wasting in future work. Dissemination is important in enabling the PI to connect with other individuals, groups or organizations conducting similar projects (create a network).
Results can be shared through:
School registers can provide a fast, simple and cheap means of injury surveillance. They may already possess an electronic or paper-based database of student attendance and collect data pertaining to the absence.It is worth approaching the headmaster to see if there is an existing system of reporting child absences. If there is none, it is good to ask about the feasibility of implementing one (or augmenting the existing one) with the help of school administration staff. It could be as simple as a logbook or a simple excel spreadsheet with drop down menus to record the following variables for any children reported as absent:
Speed data from specific sites like school zones is very hard to find. Often police data pertains to major highways, or if there is a speed camera within a school zone, harnessing that data from the relevant authorities may be problematic. Speed observation studies are a reliable way to obtain such information and can be used to help evaluate speed management to reduce child pedestrian injuries.
During the area review, the investigator should note the following elements:
Timing of observations
Speed observation forms are relatively easy to create as a word document and printed on A4 paper for data collectors to use on-site. An example from AMEND is shown below:
Amend, in partnership with the United States Centers for Disease Control and Prevention, conducted a multi-year population-based control study impact evaluation of SARSAI and found that the SARSAI programme results in a statistically significant reduction in the number of children injured in road traffic. For every 286 children whose schools are part of the SARSAI programme, one RTI is prevented per year. This is the first known road safety programme of any type proven to reduce RTI among children in sub-Saharan Africa. The methodology consisted of before and after household surveys in catchments areas of both intervention and control schools allowing for a more rigorous evaluation. Details of this evaluation can be found here.
Traffic Conflict Analysis requires carrying out an observation using video device or manually to detect and record interactions and conflicts between pedestrians and vehicles on roads. A conflict is an instance when two or more road users approach each other in space and time to such an extent that a collision is imminent if their movements remain unchanged.
Objective: To assess the conflicts/interactions between pedestrians and vehicles in a given road segment.
Note on the method: The methodology given below is simplified and non-technical and borrows from various methods that are deployed to assess traffic pedestrian interactions on roads.
The method presented here should be understood as a general method that can be tailored to suit local context. Further, different local authorities in different countries, have different versions of Traffic Conflict Analysis. Thus speaking in advance with local and national partners would help harmonize conflict analysis methods.
Mode: Video recorded observation
Frequency of measure:
Pre- intervention: 2-4 weeks before intervention
Post-intervention: 2-4 weeks after intervention
Duration: Duration refers to the time observed on the video recording. The recording should capture times when the traffic flow and pedestrian flow is high. For areas around schools, times when children come to school or leave from school are likely to be times with the highest pedestrian volume.
Record Traffic Volumes
|Peak Hour||Southbound||Northbound||Total||Peak time|
Record Pedestrian volume
|Peak Hour||Southbound||Northbound||Total||Peak time|
If a camera is used:
|Situation 1||Vehicle passes directly in front of a pedestrian who is still crossing|
|Situation 2||Vehicle clearly slows down or stops on the approach to the pedestrian crossing|
|Situation 3||Vehicle passes immediately behind a pedestrian who is still on the zebra crossing|
|Situation 4||Vehicle changes of speed or course sharply to avoid pedestrian|
|Situation 5||Pedestrians running through the road due to small safe gaps|
|Situation 6||Pedestrian waiting in the middle of the roadway|
|Level I||Potential conflict situations where there is mere breaking of road traffic rules by a single participant)|
|Level II||Conflict situations when one or more participants are restricted and the traffic flow is affected. It does not result into sudden change of speed or course but certain change is evident in response to the other road user|
|Level III||Conflict situations when one or more participants are endangered i.e. the situation when a collision can be avoided only by a prompt manoeuvring (sharp braking or sudden turning)|
|Level IV||Traffic crash|
In September 2018, Amend, with a local NGO partner and local engineer, worked in collaboration with the Mayor of Lusaka's office to implement road safety infrastructure improvements around Northmead Primary and Northmead Secondary Schools in Lusaka, as part of the Partnership for Healthy Cities programme. Average and 85th percentile speeds of 38km/h and 45km/h respectively were recorded outside the schools before any interventions, and thirteen road traffic injuries with two fatalities had been reported by head teachers in the previous year.
The infrastructure improvements comprised a safe school zone with speed reducing measures (speed humps and rumble strips), two raised zebra crossings, footpaths, bollards, pedestrian gates and signage. After the improvements, average and 85th percentile speeds dropped to 12km/h and 14km/hr respectively. The total school populations of over 3,500 at Northmead Primary School and 2,300 at Northmead Secondary Schools, as well as the wider community have benefited from the road safety improvements.