Each part is played twice: authenticity, artificiality and anxiety in listening assessment

In this article, Cathy Jones, Assessment Development Specialist, answers the question about why each part is played twice in the LANGUAGECERT Academic Listening test.

Everyone in their working or professional life will have been told at some point that the solution to a problem is ‘to get the balance right’. To me this is another manifestation of ‘get cape, wear cape, fly’, which I wrote about in a previous article flying is the difficult bit. As is deciding what the balance is, putting it into practice and maintaining it.

In the world of language testing, the balance is the Manichean struggle between authenticity and artificiality.

In the real world

A LANGUAGECERT Academic score represents a test taker’s ability to communicate effectively in an academic environment. The test is designed to reflect the language skills used in the real world of higher education. But no test can ever be an exact mirror. If it were, it would have to be as long as a degree itself.

In Jorge Luis Borges' short story On Exactitude In Science, the cartographers of an empire create a map of a province the size of a city, then a map of a province the size of the empire, and finally a map of the empire the size of the empire. Only for the following generations to see that the 'vast map was Useless'.

No high-stakes admissions test can be an exact replica of all possible language use scenarios at university. A driving test can’t simulate every vehicular eventuality. The map cannot be the territory in any form of assessment.

But a test must be authentic.

Authenticity, or context validity in assessment parlance, is the requirement that the skills assessed and task scenarios are as representative as possible of the target domain, the real-world language situations for which the test is intended.

The LANGUAGECERT Academic Listening test measures the ability to understand and interpret academic vocabulary and follow the conventions of academic discourse in lectures, tutorials and seminars. The listening materials, to reflect the real world of higher education, include a range of accents from Australia, New Zealand, South Africa, the USA and the UK.

The Listening test has four parts and the test taker hears:

• Seven unfinished conversations in academic contexts and chooses the appropriate completion or continuation for each from a choice of three options
• Five conversations in academic contexts and answers two three-option multiple-choice questions for each
• A lecture or podcast (single speaker) on an academic subject and completes seven gaps in a notepad with no more than three words each time
• A three-speaker group discussion or debate on an academic subject and answers six three-option multiple-choice questions

Each part is played twice.

“What? Pardon?” I hear you say. “How can a listening test where the audio is heard twice be authentic?” Well, as my hypothetical (arguably) strawman interjection shows, conversations can be interrupted and clarifications sought in any academic scenario, except for listening to lectures. Although just as listening tests no longer require a tape recorder (showing my age), technology means that lectures are available online and can be recorded live and played back.

The test environment, be it pen and paper in a large hall, computer based at a centre or online at home, is artificial by its very nature.* It is easy to confuse this artificiality with the authenticity of skills being assessed. Our Listening test is not a test of short-term memory or the ability to understand an announcement about which platform a train is leaving from. Each part is heard twice so that the right listening skills at the right level for higher education can be elicited, demonstrated and assessed.

In Part 4 of the Listening test, the test taker listens to an extended discussion between three speakers to identify:

• the central topic, main ideas and supporting arguments
• specific information, facts and details
• speaker attitudes, points of agreement and disagreement and emphasis
• how speakers support, challenge and elaborate on arguments according to the conventions of academic discourse

The first listening allows the test taker to build an initial understanding of the general context and content of the discussion, and the second helps them to focus on specific and supporting details. Playing the extract twice means we can use longer passages, delivered at a more natural pace with a wider range of advanced language, thereby making the test a more authentic representation of the real world. None of this would be feasible if each extract was heard only once. By playing each part twice we build up a better picture of what the test taker can do in the real world, and playing each part twice also accounts for what is going on in the test taker’s mind.

In the mind of the test taker

If a test assesses language skills required in the academic domain, it must do so by engaging the cognitive processes that underlie the demonstration of these skills. In language testing this is referred to as cognitive validity, which is the corollary of context validity.

In the real-world context of higher education, where English is the medium of instruction, learners inevitably encounter diverse accents. The inclusion of accents from Australia, New Zealand, South Africa, the USA and the UK in the Listening test is not only important for context validity but also cognitive validity. Psycholinguistic accounts of speech processing suggest that variability in input places different (and cognitively valid) demands on listeners. The listening comprehension task is based on a three-way discussion. It engages higher-level cognitive processes of building understanding, which in turn brings us to the very heart of language: the co-construction of meaning.

In the test taking environment

The test environment is naturally artificial and unfortunately anxiety is an inescapable part of the test experience. This is another reason each part is heard twice in our Listening test. The design of LANGUAGECERT Academic is holistic and human centred. Not only do we account for what is going on in the test taker’s mind, but we also care about what is on their mind and how they are feeling when they take the test. It is easy to forget the anxiety associated with taking a test (time is a great healer). I give a lot of presentations but the first time I was interviewed live for a webcast for the first five minutes I could feel the occasion getting to me before I hit my stride. I would have loved to have been able to redo those first minutes (and adjust the back of my chair to a fixed position). Hearing something once and not catching it is never the best way to begin a listening test and this can dent a test taker’s confidence.

In the Borges' story the following generations discard the map because it is useless and not fit for purpose. It is dumped in the desert and becomes the home of animals. For a listening test to be fit for purpose the balance must be sought between the artificiality of the test-taking experience and the dual authenticity of real-world language demands and actual cognitive processes. While being ever watchful that the test taker is a human being with feelings, whose ambitions and life chances must never be discarded for a test provider's convenience.

* LANGUAGECERT Academic is computer based with in-person, at our global network of high-stakes test centres, and secure online at-home options. Offering test takers flexibility, accessibility and choice.