Field Trial
SurveyLang conducted a Field Trial between February and March 2010, one year prior to the Main Study. The Field Trial involved a sample of 40 schools for each of the two most widely taught foreign languages in that country (from English, French, German, Italian and Spanish) from each participating country. Approximately 18,500 students in 16 adjudicated entities took the Field Trial tests. The data from these students, as well as from the teachers and principals that completed questionnaires, was analysed and all SurveyLang processes reviewed, and where necessary, amended prior to the Main Study.
Purpose
The purpose of the Field Trial was to provide information on the quality of the test items (both Language Tests and Questionnaires) and in-country translations of important SurveyLang documentation (including Questionnaires and important manuals for in-country use). This Field Trial allowed any arising issues in terms of the test items and translations to be resolved before the Main Study. It also provided an opportunity for participating countries to familiarise themselves with all the tasks required of them.
SurveyLang Development
Because the Field Trial was a dry run of the actual survey, all test materials and systems had to be developed before the Field Trial. The lead up to the Field Trial was therefore a very intensive development period for SurveyLang.
All of the language test material had already been developed in preparation for pretesting which was only four months prior to the Field Trial. The three contextual Questionnaires were also developed and were translated by countries.
The computer-based software system contains all of the Language Test material and Questionnaires. Both the paper-based and computer-based testing material was provided by this system. It produced kits containing thousands of uniquely-produced booklets and audio files that were dispatched to NRCs and their network of Test Administrators.
National Research Coordinator tasks
Pre-Field Trial was also a busy time for participating countries. National Research Coordinators had a number of tasks to complete before the Field Trial could take place in their country. These tasks predominantly related to the SurveyLang areas of translation and sampling. As the time schedule was very tight, demanding, and had many interdependencies, SurveyLang and the National Research Coordinators maintained regular communication to ensure the latter understood all tasks, received any support they needed and were equipped to progress as required by the schedule. Training sessions were also provided by SurveyLang in the build-up to the Field Trial which focused on test administration, marking issues and use of the testing tool. National Research Coordinators were responsible for cascading this training to staff in-country. Post-Field Trial, they submitted detailed reports of how materials and processes had worked in-country and then worked with SurveyLang to build on these experiences for the Main Study.
Main Study
The Main Study was conducted between February and March 2011. For the Main Study, over 50,000 students in over 2,000 schools in 16 adjudicated entities took the Main Study tests. The results of the Main Study are written up in the Final Report which is released in 2012.