What is Validity?
by Evelina Galaczi
July 17th, 2020

The fundamental concept to keep in mind when creating any assessment is validity. Validity refers to whether a test measures what it aims to measure. For example, a valid driving test should include a practical driving component and not just a theoretical test of the rules of driving. A valid language test for university entry, for example, should include tasks that are representative of at least some aspects of what actually happens in university settings, such as listening to lectures, giving presentations, engaging in tutorials, writing essays, and reading texts.

Validity has different elements, which we are now going to look at in turn.

Test Purpose – Why am I testing?

We can never really say that a test is valid or not valid. Instead, we can say that a test is valid for a particular purpose. There are several reasons why you might want to test your students. You could be trying to check their learning at the end of a unit, or trying to understand what they know and don't know. Or, you might want to use a test to place learners into groups based on their ability, or to provide test takers with a certificate of language proficiency. Each of these different reasons for testing represents a different test purpose.

The purpose of the test determines the type of test you're going to produce, which in turn affects the kinds of tasks you're going to choose, the number of test items, the length of the test, and so on. For example, a test certifying that doctors can practise in an English-speaking country would be different from a placement test which aims to place those doctors into language courses.

Test Takers – Who am I testing?

It’s also vital to keep in mind who is taking your test. Is it primary school children or teenagers or adults? Or is it airline pilots or doctors or engineers? This is an important question because the test has to be appropriate for the test takers it is aimed for. If your test takers are primary school children, for instance, you might want to give them more interactive tasks or games to test their language ability. If you are testing listening skills, for example, you might want to use role plays for doctors, but lectures or monologues with university students.

Test Construct – What am I testing?

Another key point is to consider what you want to test. Before designing a test, you need to identify the ability or skill that the test is designed to measure – in technical terms, the ‘test construct’. Some examples of constructs are: intelligence, personality, anxiety, English language ability, pronunciation. To take language assessment as an example, the test construct could be communicative language ability, or speaking ability, or perhaps even a construct as specific as pronunciation. The challenge is to define the construct and find ways to elicit it and measure it; for example, if we are testing the construct of fluency, we might consider features such as rate of speech, number of pauses/ hesitations and the extent to which any pauses/hesitations cause strain for a listener.

Test Tasks – How am I testing?

Once you’ve defined what you want to test, you need to decide how you’re going to test it. The focus here is on selecting the right test tasks for the ability (i.e. construct) you're interested in testing. All task types have advantages and limitations and so it’s important to use a range of tasks in order to minimize their individual limitations and optimize the measurement of the ability you’re interested in. The tasks in a test are like a menu of options that are available to choose from, and you must be sure to choose the right task or the right range of tasks for the ability you're trying to measure. 

Test Reliability - How am I scoring?

Next it’s important to consider how to score your test. A test needs to be reliable and to produce accurate scores. So, you’ll need to make sure that the scores from a test reflect a learner's actual ability. In deciding how to score a test, you’ll need to consider whether the answers are going to be scored as correct or incorrect (this might be the case for multiple–choice tasks, for example) or whether you might use a range of marks and give partial credit, as for example, in reading or listening comprehension questions. In speaking and writing, you’ll also have to decide what criteria to use (for example, grammar, vocabulary, pronunciation, essay, organisation in writing, and so on). You’ll also need to make sure that the teachers involved in speaking or writing assessment have received some training, so that they are marking to (more or less) the same standard.

Test Impact - How will my test help learners?

The final – and in many ways most important – question to ask yourself is how the test is benefitting learners. Good tests engage learners in situations similar to ones that they might face outside the classroom (i.e. authentic tasks), or which provide useful feedback or help their language development by focusing on all four skills (reading, listening, writing, speaking). For example, if a test has a speaking component, this will encourage speaking practice in the classroom. And if that speaking test includes both language production (e.g. describe a picture) and interaction (e.g. discuss a topic with another student), then preparing for the test encourages the use of a wide range of speaking activities in the classroom and enhances learning.

Adapted from: https://www.cambridgeenglish.org/blog/what-is-validity. Acesso em: 15 dez. 2023.

Read Text I and answer the fourteen questions that follow it

 Text I The “literacy turn” in education: reexamining 

what it means to be literate

In response to the phenomena of mass migration and the emergence of digital communications media that defined the last decade of the 20th century, the New London Group (NLG) called for a broader view of literacy and literacy teaching in its 1996 manifesto, A Pedagogy of Multiliteracies: Designing Social Futures. The group argued that literacy pedagogy in education must (1) reflect the increasing cultural and linguistic diversity of the contemporary globalized world, and (2) account for the new kinds of texts and textual engagement that have emerged in the wake of new information and multimedia technologies. In order to better capture the plurality of discourses, languages, and media, they proposed the term ‘multiliteracies’.

Within the NLG’s pedagogy of multiliteracies, language and other modes of communication are viewed as dynamic resources for meaning making that undergo constant changes in the dynamics of language use as learners attempt to achieve their own purposes. Within this broader view of literacy and literacy teaching, learners are no longer “users as decoders of language” but rather “designers of meaning.” Meaning is not viewed as something that resides in texts; rather, deriving meaning is considered an active and dynamic process in which learners combine and creatively apply both linguistic and other semiotic resources (e.g., visual, gesture, sound, etc.) with an awareness of “the sets of conventions connected with semiotic activity [...] in a given social space” (NLG, 1996, p. 74).

Grounded within the view that learning develops in social, cultural, and material contexts as a result of collaborative interactions, NLG argued that instantiating literacy-based teaching in classrooms calls on the complex integration and interaction of four pedagogical components that are neither hierarchical nor linear and can at times overlap: situated practice, overt instruction, critical framing, and transformed practice. […]

Although the NLG’s pedagogy of multiliteracies was conceived as a “statement of general principle” (1996, p. 89) for schools, the group’s call for educators to recognize the diversity and social situatedness of literacy has had a lasting impact on foreign language (FL) teaching and learning. The reception of the group’s work along with that of other scholars from critical pedagogy appeared at a time when the field was becoming less solidly anchored in theories of L2 acquisition and more interested in the social practice of FL education itself. In the section that follows, we describe the current state of FL literacy studies as it has developed in recent years, before finally turning to some very recent emerging trends that we are likely to see develop going forward.

(Adapted from: https://www.colorado.edu/center/altec/sites/default/files/ attachedfiles/moving_toward_multiliteracies_in_foreign_language_teaching.pdf)

O texto seguinte servirá de base para responder às questões de 11 a 22.

Archaeological Areas of Pompei, Herculaneum and Torre Annunziata

(1º§) The World Heritage property includes three different archaeological areas: the ancient towns of Pompeii and Herculaneum together with the Villa of the Mysteries (to the west of Pompeii) and the Villa of the Papyri (to the west of Herculaneum), and the Villa A (Villa of Poppaea) and Villa B (Villa of Lucius Crassius Tertius) in Torre Annunziata. The vast expanse of the commercial town of Pompeii contrasts with the smaller but better-preserved remains of the smaller Herculaneum, while Villa A in Torre Annunziata gives a vivid impression of the opulent lifestyle enjoyed by the wealthier citizens of the early Roman Empire.

(2º§) When Vesuvius erupted in 79 AD, it engulfed the two flourishing Roman towns of Pompeii and Herculaneum, as well as the many wealthy countryside villas in the area. Pompeii was buried largely by a thick layer of volcanic ash and lapilli and Herculaneum disappeared under pyroclastic surges and flows. These sites have been progressively excavated and made accessible to the public since the mid-18th century. However, in the case of Herculaneum large areas of the ancient town still lie under the modern town and have only been explored and surveyed by the network of 18th-century tunnels that drew the attention of Grand Tour visitors, the basis still today for visiting the Herculaneum's underground ancient theatre. These areas are mostly not currently included in the World Heritage property.

(3º§) Pompeii, with its well-preserved buildings in an excavated area of 44 ha, is the only archaeological site in the world that provides a complete picture of an ancient Roman city. The main forum is flanked by a number of imposing public buildings, such as the Capitolium, the Basilica and temples and within the city there are also many public bath complexes, two theatres and an amphitheatre.

(4º§) In Herculaneum several impressive public buildings are well preserved, including a spacious palaestra accessed through a monumental gateway, two sets of public baths, one of which (Central Thermae) is monumental and vividly decorated, the College of the Priests of Augustus, and a theatre of standard form. The Villa of the Papyri, outside the city walls, is an opulent establishment. The town is also noteworthy for the completeness of its shops, still containing equipment such as enormous wine jars.

(5º§) Herculaneum's urban districts and seafront display a higher level of preservation with noteworthy conservation of upper floors thanks to the pyroclastic material that buried the town. Organic matter was often carbonized by the high temperatures and exceptionally preserved finds include everyday objects such as foodstuffs, architectural elements and wooden furniture.

(6º§) Both Pompeii and Herculaneum are renowned for their remarkable series of residential and commercial buildings, built along well-paved streets. The earliest is the atrium house, entirely inward-looking with a courtyard at its centre: the House of the Surgeon at Pompeii is a good example. Under Hellenistic influences, this type of house was enlarged and decorated with columns and arcades and equipped with large representative rooms. In its highest form, this type of Roman house, known from towns all over the Empire, developed into a veritable mansion, richly decorated and with many rooms, of which the House of the Faun and the House of the Chaste Lovers are outstanding examples.

(7º§) The suburban villas across the Vesuvian area are perhaps even more exceptional in terms of the scale of their buildings and grounds, as well as their lavish decorations. The Villa of the Mysteries is an enormous residence just outside Pompeii's city walls, developed from a modest house built in the 3rd century BC, named from the remarkable wall paintings in the triclinium, which depict the initiation rites ('mysteries') of the cult of Dionysus.

(8º§) The two villas in Torre Annunziata are both extraordinary examples of suburban buildings in the countryside of Pompeii. The villa A, so-called "of Poppaea", is a huge maritime residence built in the middle of the 1st century BCE, enlarged during the Imperial period and under restoration at the moment of the eruption. It is especially well known for its magnificent and well-preserved wall paintings, one of the most important examples of Roman painting with their superb illusionistic frescos of doors, colonnades and garden views. On the other hand, villa B is an excellent example of villa rustica provided with rooms and spaces designated for market activities such as storage of amphoras and trading of locally produced foodstuffs, especially wine.

(9º§) There were many changes to these buildings over time in response to changing circumstances of the owners; these include repairs and adjustments that were a response to the seismic events that led up to the AD 79 eruption and reflect a community living with changing environmental and economic conditions.

(10º§) A special feature of Pompeii is the wealth of graffiti on its walls. An election was imminent at the time of the eruption, and there are many political slogans scrawled on walls, as well as others of a more personal nature, often defamatory. At Herculaneum, the volcanic deposits preserved hundreds of wax tablets, some of which conserve legal documents, and more than 1,800 papyri scrolls containing Greek philosophical texts were found at the Villa of the Papyri.

(11º§) The diverse range of literary sources available in Pompeii and Herculaneum provides a picture of the final decades of these ancient cities and the image of socially complex and dynamic communities, representing exceptional evidence of typical ways of life in Roman society in the first century AD and the importance of texts in political and private life.

(12º§) Other important sources of archaeological evidence are the human remains of those who died in the eruption. Pompeii witnessed an early archaeological experiment when plaster was poured into voids found in the volcanic material and which allowed casts to be made of the forms of the human and animal victims and other organic material. At Herculaneum, on the other hand, about 300 skeletons were discovered along the ancient shoreline. The study of these significant samples of victims from the towns provides insight into their health, lifestyles and death and a chance to compare the two data sets. The casts themselves are important resources as they contain both skeletal remains and evidence of 19th- and 20th-century archaeological practice.

(13º§) Another important legacy of the twentieth century was the presentation of Herculaneum to the public as an 'open-air museum', perhaps Europe's first, with buildings reconstructed based on archaeological evidence and displays of original objects within the archaeological site. This concept of 'open-air museum' had already been adopted in some buildings in Pompeii, as a medium to communicate the meaning of ancient spaces, at the end of the 19th century.

(14º§) The impressive remains of the towns of Pompeii and Herculaneum and their associated villas, destroyed and yet preserved by Mount Vesuvius, provide a complete and vivid picture of society and daily life at a specific moment in the past that is unparalleled elsewhere. The rediscovery and history of these places as archaeological sites has captured the collective imagination century after century, shaping archaeological, art historical, conservation and interpretation practices in Europe and beyond.

(adapted) https://whc.unesco.org/en/list/829/

Text 6 (for questions)

Things to do in Ireland

*laver: a type of seaweed / seaweed: alga marinha

(Available at: www.nationalgeographic.com/travel/article/10-best-things-to-do-ireland – text specially adapted for this test).