Questionnaire development
Sanneke Schouwstra at CITO heads the SurveyLang Questionnaire development team.
There were four contextual Questionnaires which have been specifically designed for the European Survey on Language Competences. The first Questionnaire was the Student Questionnaire which was completed by the students who also took a language test. The second Questionnaire was for language teachers; the third for school principals and fourth for National Research Coordinators who completed a country level Questionnaire.
The purpose of the Questionnaires
The European Commission has stated that the following is the key purpose of the overall Survey: "to facilitate a more productive comparison of language policies, and language teaching methods between Member States, with a view to identifying and sharing good practice" (Communication of the Commission on The European Indicator of Language Competence, 2005, p. 5). Additional European Council policy objectives include stimulating Member States to provide a sound basis for the life-long learning of foreign languages through teaching at least two foreign languages from an early age, promoting teacher training, and stimulating the creation of environments (both in school and at home) where different languages are heard and seen, where speakers of all languages feel welcome and language learning is encouraged.
The Questionnaires help achieve and find out more information about these objectives by providing additional contextual information that supplements the analysis of the Language Tests.
Content of the Questionnaires
Students answered questions about themselves, their family, their home environment, their school environment, their foreign language lessons and learning experiences. In the second section of the questionnaire, students completed a short series of Can Do statements, self-assessing their ability in the foreign language tested.
The Questionnaires for teachers, Principals and National Research Coordinators focused on those aspects of educational policies, language policies and language teaching methods that may have varied among regions and countries. This helps to identify those areas of good practice within foreign language teaching.
Countries participating in the survey also had the opportunity to add five contextual questions relevant to their country to each of the four Questionnaires. This enabled them to use the Survey to gain insight into aspects of language learning and education specific to their country, but perhaps not relevant to other Member States.
Design and development of the Questionnaire
The first phase of the development process involved the construction of a conceptual framework of policy issues. Another key task was to examine the different educational systems of the Member States, noting those areas where differences may affect the comparability of data across countries.
The second developmental phase involved writing the actual Questionnaire items and testing them in cognitive labs to ensure the questions functioned as intended and that the target age group understood the question wording.
Throughout both of these phases, extensive consultations and discussions were carried out with Member States and other stakeholders.
Countries translated the English source version of the Questionnaires and adapted them for use in their country. Rigorous quality procedures were put in place for this translation process.
Following the completion of the Field Trial, the Survey data were closely analysed and any necessary amendments to the wording of questions incorporated into the version of the Questionnaires used for the Main Study. The workability of questions across all countries and the workability of the translations was scrutinised as part of this review.
After the Main Study concluded, data from the Questionnaire was analysed with data from the Language Tests to determine how these contextual factors might affect students’ language learning, both in the individual Member State and across all the participating countries.