Deeper Dive Create survey items

Once you have operationalized parent involvement and created a framework for your survey, you need to create the items you will include in your survey. When creating survey items, always keep in mind the original purpose for your parent involvement data collection activities: What questions do you want to answer?

When developing a new survey or modifying existing survey items, be sure to follow these important principles:

Use clear and simple language

Use clear and simple language when writing survey items to increase the likelihood that the people completing your survey will understand how to answer each question. Figure 11 illustrates the process respondents follow when answering a survey question, and shows that if respondents cannot understand the question, or if the answer respondents want to give is not one of the response options, they will not be able to respond accurately. (Or they may not even try to answer at all!) [45] Goto footnote

Figure 11

The Process of Answering a Survey Question

  1. Respondent stores information
  2. Respondent comprehends question
  3. Respondent retrieves information
  4. Respondent chooses to answer question
  5. Respondent is able to fit answer with response options

presents some characteristics of a good survey item, with examples of questions that ensure that the items are written in a way that facilitates (or does not facilitate) understanding.

Table 26
Characteristics of a good survey item, with examples of not good and good items

Table 26

Characteristics of a good survey item, with examples of not good and good items
Characteristic Example: Not Good Example: Good
Uses clear terminology and plain language How has your active engagement enhanced your child’s academic or social-emotional performance? In what ways does your involvement help your child succeed in school?
Only asks one question at a time (avoids double-barreled questions) To what extent do you think the afterschool workshops and Parent Nights have increased your access to information about how to be more involved in your child’s education?
  • To what extent do you think the afterschool workshops have increased your access to information about how to be more involved in your child’s education?
  • To what extent do you think the Parent Nights have increased your access to information about how to be more involved in your child’s education?
Contains a clear threshold for answering “yes” Have you met with your child’s teacher? Have you met with your child’s teacher to discuss your child’s progress achieving his or her IEP goals?
Provides a timeframe How often do you receive information from the Parent Information Center? How many times in the past month have you received information from the Parent Information Center?
Provides a timeframe appropriate to the topic How many times did you attend IEP meetings? How many times did you attend IEP meetings last year?
Gives exhaustive and mutually exclusive response options

In which situations are you given opportunities to provide input on your child’s education?

  • During meetings with school staff
  • During informal communications with school staff

In what situations are you given opportunities to provide input on your child’s education?

  • During IEP meetings
  • During other regular in-person meetings with your child’s teacher
  • During other meetings with related services providers assigned to work with your child
  • Through emails or other written communications with your child’s teacher or related services providers
  • Through regularly scheduled phone calls
  • Through unscheduled phone calls initiated by you or by staff
  • Through informal communications with staff, such as chats before school
Avoid double-negative questions It is not an insignificant burden for me to attend IEP/IFSP meetings. Please indicate the degree to which you agree with this statement: It is a significant burden for me to attend my child’s IEP/ISFP meeting.

Other ways you can work to use clear and simple language include

  • Keeping questions brief
  • Ensuring that the items are written using language no higher than an eighth grade reading level
  • Avoiding potentially unfamiliar terms and abbreviations whenever possible
  • Defining any unfamiliar terms or abbreviations

Select the correct item format and type

Another part of creating a good survey item is to select the correct item format. presents three types of item formats. [46] Goto footnote

Table 27
Item formats

Table 27

Item Formats
Item Format Works Well When Example
Closed, or structured, items – Provide predetermined response options
  • You have many respondents.
  • You have (or can create) a list of possible response options.
  • You want to expend fewer resources collecting and analyzing data.
  • You need to calculate reliability of the data.
  • You would like to conduct quantitative analyses of the data.

In the past year, how many times have you met with your child’s teacher about your child’s academic progress? (Select one)

  1. Never
  2. 1-3 times
  3. 4-6 times
  4. 7-9 times
  5. 10 or more times
Open, or unstructured, items – Allow respondents to provide answers in their own words
  • You have a limited number of respondents (or you have extensive resources and staff to collect and analyze the data).
  • You do not have (or do not know how you would create) a list of possible response options.
  • You would like unthought-of categories to emerge.
  • You want spontaneous, or in-depth, responses.
  • You are exploring response options for future surveys.
  • You have skilled qualitative analysts who have time to analyze the data.
 Over the past year, what would you say has been the most important thing your child’s school has done to facilitate your involvement in your child’s education? (Write your answer)
Mixed format items – Combine elements of closed and open formats
  • You would like to have the advantages of a closed format question (e.g., having the respondent pick from specific response options, minimizing costs) while giving respondents an opportunity to either enter their own response or elaborate on their answer.

In the past year, in what ways has your child’s school made it easier for you to participate in your child’s education? (Select all that apply)

  1. Communicated with you regularly about your child’s progress on IEP goals.
  2. Given you choices about the services provided to meet your child’s needs.
  3. Offered you a variety of ways to communicate with teachers/school staff.
  4. Provided information on the options you have if you disagree with a decision of the school.
  5. Other, specify

Additionally, you can choose from a number of types of closed items, as shown in .

Table 28
Types of closed survey items

Table 28

Types of closed survey items
Item Format Description Example
Dichotomous Respondents choose one of two response options

Has your child received special education services during the current school year? (Select one)

  1. Yes
  2. No
Categorical

Respondents choose one response from among multiple categories

Note: These are treated as nominal (or categorical) data in analyses.

What grade is your child in? (Select one)

  1. under K-2
  2. 3-5
  3. 6-8
  4. 9-12
Forced choice: Single response Respondents choose one of several response options

I receive communications from my child’s school… (Select one)

  1. Not often enough
  2. Often enough
  3. Too often
Forced choice: Multiple response Respondents choose more than one of several response options

How do you communicate with your child’s service providers? (Select all that apply)

  1. In-person
  2. Phone
  3. Email
  4. Other

Note: The “Other” response could also include a space for respondents to write in their answer, making it a mixed format item, as shown below.
Ranking Respondents rank items in a list in order of preference or importance.

Please rank the following methods of communication from most effective to least effective, with a 1 = least effective and 5 = most effective

  • In-person meeting
  • Phone call
  • Text message
  • Email
  • Other (Specify):
Rating Respondents select a single rating along a continuum.

Please indicate your level of agreement with the following: I feel involved in my child’s special education services.

  • Strongly disagree
  • Disagree
  • Agree
  • Strongly agree

Note: This item does not offer respondents an option to enter a “neutral” response. This might be beneficial when you want to force respondents to give an opinion about a topic. However, it might discourage respondents who really do not have an opinion from responding to the question.

Grid Questions

If you have a number of items that share a common set of response options (e.g., Yes/No, Strongly Agree to Strongly Disagree), you can group them into a grid or matrix, as illustrated in Table 29 and Table 30.

Table 29

Grid question example 1

Table 29: Grid question example 1
For each of the questions below, please check the box that corresponds to your answer. Yes No Unsure
1. A support network for parents of students with disabilities is available to me through my school district or other sources.
2. I am involved in a support network for parents of students with disabilities available through my school district or other sources.

Table 30

Grid questions example 2

Table 30: Grid questions example 2. Please check the box that indicates to what extent you agree or disagree with each of the following statements.
My child’s school: Strongly Disgree Disagree Agree Strongly Agree
1. Communicates with me regularly about my child’s progress on IEP goals.
2. Gives me choices about the services the school provides to meet my child’s needs.
3. Offers me a variety of ways to communicate with teachers/school staff.
4. Provides information on the options I have if I disagree with a decision of the school.

Grid questions can reduce the amount of time it takes for respondents to complete a set of items. [47] Goto footnote [48] Goto footnote However, it is important to structure these items well and make clear how you want respondents to complete the questions. Consider these two examples:

  • In Table 29, the response option “Unsure” was not offered for item 2, because the survey designers decided that parents should know whether or not they are involved in a support network for parents of students with disabilities. If the “Unsure” response option had been available for item 2, then the data the state obtained from that item might have been unclear, since the state would not have known if the parents were responding “Unsure” because they weren’t involved in a network or if they were unsure if the network was “available through my school district or other sources.”
  • In Table 30, the instructions tell respondents to check the box that indicates to what extent they “agree or disagree with each of the following statements.” If the word each had not been included in the instructions, then parents might have thought they did not have to respond to each of the items. Similar instructions were included in Table 29 (i.e., “For each of the questions below”) to ensure that parents answered both items.

When using grid questions, keep in mind the following suggestions:

  • Keep grid questions simple
  • Minimize the number of items in the grid
  • Provide direct instructions as well as visual clues to guide parents on how to respond
  • Repeat headers of grid questions on any new pages

Since different item formats and types work best to gather different types of data, when choosing item formats and types, think carefully about

  • The questions you want to answer
  • The resources you have available to design and administer the survey and collect and analyze the data
  • The analyses you want to conduct

This will help you choose the right format/type to obtain the data you need. Talk with an experienced survey developer, with your evaluator, or with your IDC State Liaison if you have questions.

Offer good response options

How can stakeholders help?
Stakeholders can be a good source of information during survey development! As you think about possible survey items and corresponding response options, ask stakeholders to offer suggestions and give feedback to ensure respondents will be able to understand and interpret survey items as intended and the survey includes appropriate questions and response options given the particular state context.

It is important to carefully consider how you will present response options for closed (and mixed format) items because the types of options available will affect respondents’ willingness to respond to the item and shape the conclusions you can draw from your data. For example, if respondents have a “neutral” opinion about some experience but do not see “neutral/no opinion/neither agree nor disagree” as a response option, they may skip the survey item.

When developing response options for your parent involvement survey, it may be a good idea to talk with key stakeholders to get their input. They are in a good position to know what possible responses might be appropriate for a particular question, given the state context.

Here are some important general tips to remember when creating your response options:

Link response options back to the questions your state wants to answer.

Be sure that the response options you provide will actually produce the data you are trying to collect. Consider this example:

Your state wants to know whether schools are actually using a specific strategy to facilitate parent involvement that was taught in a statewide professional development workshop. The strategy is providing families with a planning guide for IEP meetings to facilitate two-way planning and communication. Look at the two examples for Question 1 that follow and think about which is more likely to produce the data your state needs to answer this question and why.

Question 1, Example A

In what ways does your school facilitate your involvement in your child’s education?

Question 1, Example B

Please indicate whether your school uses any of the following strategies to facilitate your involvement in your child’s education:

Example B has clear advantages because it specifically lists the strategies for increasing parent involvement taught in the state PD workshop. This ensures that parents provide responses related to the use of those specific strategies. In contrast, Example A allows parents to decide what to talk about in their response, which may or may not provide data related to what the state would like to know.

Make sure response options are clear and mutually exclusive.

Do not offer response options that overlap or that different respondents might interpret differently. Consider the three examples for Question 2 that follow. Which of the examples offers clear and mutually exclusive response options?

Question 2, Example A

How often do you communicate with your child’s teacher about your child’s IEP progress and goals?

Question 2, Example B

How often do you communicate with your child’s teacher about your child’s IEP progress and goals?

Question 2, Example C

How often do you communicate with your child’s teacher about your child’s IEP progress and goals?

Example A is vague and response options b and c are overlapping terms. Example B provides clearer response options, but choices b and c overlap. Example C responses are clear and mutually exclusive.

List all of the potential answers to a particular question in your response options.

When writing questions, be sure to include the full range of responses that might be possible. For example, if you are asking parents to report on their ethnicity, use all of the federal racial/ethnic group categories. See the two examples for Question 3 that follow. Which is the better choice and why?

Question 3, Example A

Which method do you most prefer that your child’s school use to notify you about upcoming IEP meetings?

Question 3, Example B

Which method do you most prefer that your child’s school use to notify you about upcoming IEP meetings?

Example B is the better choice. Example A gives parents very few response options—what happens if parents prefer another type of communication? Example B provides more response options and also includes an “Other (Specify):” option with a space for parents to write-in a response. This gives parents even more ways to tell the state how they prefer to receive communications. However, be careful when including the “Other (Specify):” option, or you will end up with large amounts of qualitative data that you will then need to analyze.

Finally, for questions that might not apply to all respondents, you may want to include a “not applicable” response option. Be careful when offering a “not applicable” response, however, because it might encourage parents to choose that response to avoid having to provide an answer to the question.

Use fully labeled scales. [49] Goto footnote

Labelling all of the points on the scale helps to ensure that everyone has a similar understanding of the response options. For example, you should label every point along the scale rather than labeling only certain points of the scale (e.g., the endpoints) and using numbers or dashes to represent the other points on the scale. Consider the examples presented that follow. How might parents’ responses differ depending on which response scale was provided? How should the state interpret the different responses for Question 4, Example A?

Question 4, Example A

Question 4, Example B

Present response options in the order that respondents would generally expect.

Consider the two examples for Question 5 that follow. Since many parents only meet annually with their child’s IEP team, it might make more sense to present the response options as shown in Example B.

Question 5, Example A

How often do you meet with the IEP team to plan and discuss your child’s educational progress?

Question 5, Example B

How often do you meet with the IEP team to plan and discuss your child’s educational progress?

Additionally, whenever possible, we recommend that you present scales horizontally, moving from left to right, as shown in the scale that follows.

Footnotes

  • 45. Fiore et al. 2012.
  • 46. Carlson and D’Agostino 2015.
  • 47. Couper, Traugott, and Lamias 2001.
  • 48. Tourangeau, Couper, and Conrad 2004.
  • 49. Toepoel and Dillman 2008.

Next Page:

Survey Development

Design the survey to maximize responses

Continue