Computer-based software systems
A unique software system has been developed to manage the design, translation, test administration and data collection of the survey instruments (the language tests and context questionnaires).
Jostein Ryssevik, from Ideas2Evidence, works closely with Gallup Europe and is Team Leader for this exciting area of SurveyLang’s work, which creates new approaches to test development.
The platform is fine-tuned to the specific set of requirements of the Survey and is designed to support the delivery of both the paper-based and the computer-based tests.
These are just some of the aspects of this system:
Item authoring tool
The language test items for the ESLC survey were developed by an expert team of over 40 Language Testing item writers located across Europe. Items moved through the cycle of authoring, editing, vetting, adding of graphics and audio, pre-testing, Field Trial and Main Study, with each stage involving different tasks, roles and responsibilities.
The Test Item Authoring Tool supported this process. It allowed non-technical language-testing item writers to create tasks by means of templates for the various task types used. At any stage in the development, a task could be previewed and tested to allow the author to see how it would look and behave when rendered in a test. The authoring tool also supported the capture and input of all the metadata elements associated with a task.
Test Item Databank
The Test Item Databank is the hub of the central system, providing long-term storage, version control and management of the language test items, graphics and sound files. Test items were uploaded to the databank by the item writers to be seen and shared by others in this process. The databank included a version control mechanism and a secure authentication system, ensuring only authorized personnel could see or change tasks.
Test assembly tool
The assembly of test items into complete test sessions was managed by the language testing team.
In order to allocate students to tests, the test assembly tool also needed to process detailed information about every sampled student; especially the student’s proficiency level (from the routing test), the tested language as well as the student’s national or local language.
The assembly tool used this information to randomly allocate students across the group available test sequences matching their respective proficiency level. Finally, the test assembly tool produced the individualized test material for each student, both for computer-based and paper-based testing.
What makes this an innovative approach is the fact that both computer and paper-based tests were served from the same source and the same system. This reduced manual work and manually-induced errors.
Test rendering tool
Security is one of the critical challenges related to computer-based delivery of the Language Tests. Protecting the security of the test material and isolating students from external influences such as access to the web or dictionaries are both vital to the integrity of the test results.
SurveyLang developed a test rendering tool that booted the test computers from USB memory-sticks or CDs. This ensured SurveyLang had full control of the local desktop environment, including blocking access to the Internet and other resources.