Instead of the heuristic randomization methods to design split questionnaires that are currently used in applied and academic research, we develop a methodology to design the split questionnaire to minimize information loss using estimates from a first wave or pilot study. Because the number of possible questionnaire designs is exponential in the number of questions, we apply the Modified Federov algorithm, using Kullback Leibler Distance as a design criterion, to find the optimal splits. We use Markov chain Monte Carlo procedures to impute missing values that result from the design. First of all, we illustrate the efficiency of the Modified Federov Algorithm on a small synthetic questionnaire, which enables the enumeration of all possible designs for comparison. Second, we compare the efficiency of split questionnaires generated with the proposed method to multiple matrix sampling, incomplete block designs, and a heuristic procedure based on principal components analysis, using synthetic and empirical Web survey data. We generate split questionnaire designs selecting either entire blocks of questions (between-block design) or sets of questions in each block (within-block design). Finally, we illustrate that due to reduced respondent burden the quality of data using split designs increases, compared to a full questionnaire in a field study.

The Design of Split Questionnaires / Adiguzel, Feray; Wedel, Michel. - Proceedings 59th ISI World Statistics Congress, (2013), pp. 317-322. (ISI World Statistics Congress

The Design of Split Questionnaires

ADIGUZEL, FERAY;
2013

Abstract

Instead of the heuristic randomization methods to design split questionnaires that are currently used in applied and academic research, we develop a methodology to design the split questionnaire to minimize information loss using estimates from a first wave or pilot study. Because the number of possible questionnaire designs is exponential in the number of questions, we apply the Modified Federov algorithm, using Kullback Leibler Distance as a design criterion, to find the optimal splits. We use Markov chain Monte Carlo procedures to impute missing values that result from the design. First of all, we illustrate the efficiency of the Modified Federov Algorithm on a small synthetic questionnaire, which enables the enumeration of all possible designs for comparison. Second, we compare the efficiency of split questionnaires generated with the proposed method to multiple matrix sampling, incomplete block designs, and a heuristic procedure based on principal components analysis, using synthetic and empirical Web survey data. We generate split questionnaire designs selecting either entire blocks of questions (between-block design) or sets of questions in each block (within-block design). Finally, we illustrate that due to reduced respondent burden the quality of data using split designs increases, compared to a full questionnaire in a field study.
978-90-73592-34-6
Kullback Leibler distance, missing data, modified federov algorithm, multiple imputation, questionnaire design
File in questo prodotto:
File Dimensione Formato  
ISI2013Proceedings_Split questionaire design for massive surveys.pdf

Open Access

Descrizione: Conference full paper
Tipologia: Documento in Post-print
Licenza: DRM non definito
Dimensione 188.37 kB
Formato Adobe PDF
188.37 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11385/169510
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact