Does Timing Influence Survey Results?

With the growth of internet research and its promise to deliver speed, it is often debated among market researchers if time of response and reduced field times result in biased data.

Studies have shown that differences do occur between day and evening respondents, as well as those who respond early versus late during the field period. However, the differences are not sharp enough to impact overall business decisions. When fielding internet-based research, it is still important to consider field length and how it may impact results and responses from various demographic groups.

With that in mind, we put together the following recommendations:

• For research programs that require quick field periods of two days or less, specific attention should be given to quota management to allow for a balance among region, employment age and gender in order for fair representation for all cells that are pertinent to the research program.
• When fielding surveys among a younger population, we recommend an extended field in order to maximize response rates and to encourage balance among region, income and employment within each age group.
• When designing a test for absolute comparisons it would be wise to leave the survey in the field for longer periods of time to ensure adequate demographic representation occurs that balances out demographic biases occurring with early versus late responders. We recommend allowing at least three field days.
• When designing experiments such as a multi-cell concept tests where absolute comparisons are required, effort should be made to ensure early versus late responders are balanced by directing each new respondent to a rotating cell as they enter the survey location. The same consideration for concept rotation should be used for day versus evening respondents.
• Finally, when conducting programs of research where concept or product scores are compared in a database, standard field times and sample specs should be as closely matched as possible across projects.