Measurement should be the first stage in any new project, and an ongoing process to support continuous listening and improvement programs. The purpose of assessment, or the use of questionnaires and surveys, is ultimately to gather data to inform your decision-making process.
Every school has numerous stakeholder groups including staff, students, parents and carers, and members of the wider community (e.g., prospective families) who are all valuable sources of information and feedback. Furthermore, you may need to use assessment in other ways, for example, to ensure that you are hiring the right people for your school.
Before commencing a project which includes measurement, it is important to decide whether you have the necessary expertise to design and administer the measure (or choose the best available) and analyse the resultant data. While industry experience is important in understanding stakeholders and strategic directions, being an industry subject matter expert is not enough to ensure that your measure meets the goals set for the project and gives you the data you need. Measure design is a process requiring specialised skills, specific expertise, and training.
Common Measures Used in a School
There are many applications for measures in a school environment. For example, you may use pre-employment screening measures such as a competency based personality assessment when hiring for a leadership role, to ensure you are getting the candidate who will fit best within your school and its culture. You may also need to assess leadership effectiveness and could do so using a custom designed 360-degree feedback process to inform succession planning, promotional opportunities, and professional development requirements.
Alternatively, maybe you have concerns about absenteeism or turnover, or want to better understand how engaged your employees are to provide them with a better working environment. You might be interested in identifying your current and ideal organisational culture or discovering how ready your stakeholders are for upcoming change.
You may want to gather feedback on current or proposed programs, offerings, or changes by better understanding the attitudes and perceptions of students and their families or to better understand the needs of current and prospective families. Finally, you may be interested in collecting data from exiting students to inform change, or better understand graduate experiences and evaluations.
Six Questions to Ask Your Consultant
Before committing time and resources to a consultant for your measure design or assessment project, it is integral to ensure that the data you gather will be fit for purpose so that any decisions you make based on it will actually help you to achieve your goals. Ask them about their process, expertise, expected outcomes, and return on investment. Some of these may only be relevant for specific types of measures and projects, however, they are a good starting point for discussions.
1. What experience do you have in measure design? If they are a subject matter expert in education as them about their experience in measure design. Do they understand the fundamentals of reliability and validity? Are they able to justify the response scale they use and why? Can they explain what sort of data you can expect to gather from the measure, how this can be analysed, and what it can and cannot be used for?
2. How will you decide what needs to be asked when designing our measure? You want to be sure that the measure will be contextual, taking into consideration your school and its needs and goals. If they use a standardised measure ask them why? What are the benefits of this approach? If they use their own/custom measures as what theories, models or previous research this is based on? Ask about reliability and validity, and whether the measure has been factor analysed. Ask them if they benchmark your results against similar schools or not and why.
3. How will the measure be distributed, and data collected? Do they use a free survey software? Will you be able to have your colours, logo, branding etc., on the survey or are these things not customisable? Are the links distributed via email, sms, QR code? How do they manage the issue of multiple submissions from the same individual?
4. How do they manage anonymity, confidentiality, and data security? Does the software they utilise conform with minimum data security standards? How is the data stored once it has been collected? If it is stored unsecured, is the data de-identified prior to being stored? How long will they retain your data files and backups for? Will you be able to access this or ask that it is deleted? How many people have access to the storage locations/devices? Do these individuals agree to protect the anonymity and/or confidentiality of respondents?
5. How will the responses and/or results be analysed? What software will be used to analyse the data and what sort of information will be provided to you at the end? It is important to consider the limited utility of having basic results analysed in Excel. Average scores tell you very little, especially if they are made up of items which are unrelated to an overarching construct (idea/grouping). Similarly, knowing whether one group or individual is higher or lower on something may be of little use (depending on the purpose and design of the measure). Your consultant should be able to explain to you how your data will be analysed, what results you can expect, and how these can used. For example, whether these can be used as the basis for specific decision making or interventions, projects, and programs.
6. How will you be able to use the information gathered to support the goals of your school? How will the data be presented to you? Will you have access to raw data? If not, will you only be given a final report/presentation or will you have access to pertinent data slices. Will they assist you with action planning or does their support end with a presentation of results? If they assist with next steps, are their proposed interventions shown to improve scores? Are these interventions standardised or are these designed collaboratively to best suit your context? Are they evidence-based?
If the service provider you are considering is unable to answer these questions with ease it may be worthwhile looking elsewhere for your assessment needs.