Projects are presented within the three areas of emphasis of our programs in applied linguistics– language assessment, language learning and teaching and linguistic analysis. They were categorized on the basis of their primary current emphasis even though most projects are influenced by and will ultimately contribute to more than one area.
1. Language Assessment
Projects in language assessment explore issues in the development use and validation of assessments for a variety of purposes
- Carol Chapelle and Amy Hutchison are conducting foundational research informing potential future development of English language tests using mobile-technology. The goal of the research is to complete a domain analysis paper focusing on the new language practices afforded by tablet technologies in K-12 classrooms and to develop task framework, task specifications and prototype assessment tasks in collaboration with ETS staff. (funded by Educational Testing Service; 2013-2015)
- Carol Chapelle is working on the integration of concepts argument-based validity into the mainstream of language assessment through projects that make more transparent the links between past research and aspects of the validity argument. One project analyzes similarities and differences among authors’ use of argument-based validity concepts in applied linguistics and a second presents an analysis of the intersection of multiple issues to do with rating spoken and written language and validity argument.
- Elena Cotos‘ and Yooree Chung’s project on validating the interpretation of TOEFL iBT® Speaking Scores investigates functional language use in the lecture, recitation, and lab curriculum genres of the academic ITA contexts to determine whether the language functions elicited by TOEFL iBT® Speaking tasks are representative of language functions used in the target domain. Implications of relevance to theory and research are also expected, as the results may inform task design, test use, and further investigations of the validity of TOEFL iBT® for instructional settings. This work is funded by Educational Testing Service.
- Elena Cotos is working with Hyejin Yang to validate the diagnostic uses of R-PLAT, a web-based rating platform used in the Oral English Certification Test (OECT) at ISU. R-PLAT captures multiple aspects of language performance that can serve as diagnostic evidence. Empirical insights are sought to support the judgmental assumption that the diagnostic descriptors recorded via R-PLAT are indicative of target speaking ability levels used for placement into proficiency-based levels of the course.
- Bethany Gray’s project on the longitudinal development of grammatical complexity in speech and writing analyzes data from the TOEFL iBT. The project documents changes in the frequency of use and accuracy of phrasal complexity features (which are more common in written language) and clausal complexity features (which are more common in spoken language). The study contributes to the validity argument for the TOEFL iBT, demonstrating how language learners develop over time and how that development relates to language assessments.
- Bethany Gray and Volker Hegelheimer are building and analyzing a corpus of test responses to study test takers’ performance on the writing section of the English Placement Test (EPT). The goal is to examine the use of grammatical features associated with language complexity and language development in the EPT data, to investigate how these features are used differentially across placement levels, student level (graduate vs. undergraduate), and different prompts. The data analyzed in this project will be used to study language development, and inform the interpretation, development, and validation of the EPT.
- John Levis and Elena Cotos are working on a word stress intelligibility study using Indian and Korean graduate students who were rated at different levels on the Oral English Certification Test.
- Gary Ockey and Volker Hegelheimer are leading a team of graduate students in developing a performance-based oral communication assessment with the aim of using it as part of the English Proficiency Test, which is used for placement at Iowa State University. This project involves the development of test tasks, rating scales, rater training procedures, and setting cut scores. Imbedded in this project are a number of graduate student and faculty research studies, including investigating the construct of academic oral communication ability, investigating the use of generalizability theory and Multi-faceted Rasch measurement to aid in developing rating scales, and examining new task types for better assessing oral communication.
- Gary Ockey and Volker Hegelheimer are working with a group of graduate students to develop new writing tasks for the English Proficiency Test, which is used for placement at Iowa State University. These integrated skills tasks leverage technology and current understanding of the academic writing construct to better assess the academic writing skills that are needed by university students. Graduate student research embedded within this project includes investigating different techniques for developing cut-scores for a high stakes examination.
- Gary Ockey is conducting research on the issues involved in developing a rating scale for strength of accent. This project involves using human judges to rate the strength of a speaker’s accent based on how different it is perceived to be from the judge’s own accent. This study will be published in a book Co-Authored by Elvis Wagner and Ockey: Emerging Issues in the Assessment of Second Language Listening (John Benjamins, 2017).
- Sowmya Vajjala is working on developing computational models for automated scoring of language proficiency in test taking scenarios. She has worked in the past on automated assessment for written English and Estonian, and is currently working on a project that aims to identify native accents in spoken language using speech processing, and the relevance of non-linguistic, speech signal level features in the automated scoring of speech.
2. Language Learning and Teaching
Applied linguistics faculty are working on projects to investigate new approaches to language learning pedagogy as well as the construction and evaluation of learning materials.
- Gulbahar Beckett is working on several research projects including investigation of English as a medium of instruction policies and practices in EFL contexts through document analyses, classroom observation, and interviews with students and faculty. Findings of these studies are expected to contribute to language policy work.
- Gulbahar Beckett is working on meta-analysis of project-based language learning research with Dr. Tammy Slater. Findings of this study are expected to contribute to experiential learning and technology integrated second language acquisition research.
- Dr. Gulbahar Beckett is co-editing, with Dr. Tammy Slater, a book currently titled “Theory, Research, and Models of Technology-Infused Project-Based Language Learning and Teaching: Focusing on Form and Function,” a sequel to Beckett & Miller (2006).“Project-based second and foreign language education: Past, present, and future.”
- Carol Chapelle is investigating the selection of Québec cultural content in beginning-level French language textbooks with particular emphasis on political content. The sample includes textbooks used in the United States over the fifty year period of 1960 through 2010, whose image and textual content is analyzed from a social semiotic perspective. The chronological analysis examines choices made by textbook producers as well as the relationship of these choices to contemporary pedagogy in foreign language teaching and politics in Québec (forthcoming, Palgrave Macmillan).
- Evgeny Chukharev-Hudilainen and John Levis are developing, in conjunction with Texas A & M University and funding from the National Science Foundation, a pronunciation training system using “accent conversion” (the language learner’s own voice combined with the segmentals and suprasegmental features of a matched native speaker voice. The resulting voice is called a “Golden Speaker” voice. And it will be tested with Korean learners of English.
- Elena Cotos is leading the continuous development and evaluation of the Research Writing Tutor (RWT) [https://vimeo.com/90669213], a unique automated writing evaluation tool for teaching and learning research writing in the disciplines. This project is interdisciplinary, stimulating a fruitful collaboration among faculty and graduate students in Applied Linguistics, Computational Linguistics, Human-Computer Interaction, Computer Science, and Education. This work was funded by the Computation Advisory Committee, Graduate College, College of Engineering, and College of Liberal Arts and Sciences.
- Elena Cotos’ projects with Sarah Huffman and Stephanie Link investigate the effects of RWT modules integrated in learning environments. RWT’s affordances are examined from the perspective of data-driven learning and revision processes with a focus on their potential to foster the transfer of genre knowledge to genre writing. Currently, we are examining fundamental processes specified in cognitive writing theories by focusing on how novice writers interact with multi-level rhetorical feedback and scaffolding during revision and in what ways this interaction may create conditions necessary for enhanced metacognition and writing improvement.
- Elena Cotos and Sarah Huffman are evaluating a new model for academic writing support to graduate students and post-doctoral associates implemented by the Center for Communication Excellence. The objectives are to analyze the discipline-specific research communication needs of graduate students and post-doctoral associates; to investigate the impact of the writing support to meet these needs; and to assess the effectiveness and efficiency of the approach to training Graduate Peer Mentors.
- John Levis is finishing a book for Cambridge University Press, on the central role of pronunciation’s in intelligibility and what that implies for language teaching.
- John Levis is developing a nonnative version of the Arctic Corpus (from Carnegie Mellon University) including speakers from varied first languages (Chinese, Arabic, languages from India, Spanish, and Korean). This project is being carried out in conjunction with Texas A & M University and with funding from the National Science Foundation.
- John Levis and Charles Nagle (World Languages and Cultures) are planning a collaboration on a longitudinal study of L2 learner pronunciation (for advanced users of English and for beginning learners of Spanish). The project is being done with PROG Pronunciaiton Group), PhD students and others interested in pronunciation research.
- Jim Ranalli is studying how students engage with Automated Written Corrective Feedback (AWCF); specifically, the feedback provided by the popular writing-enhancement tool Grammarly. This project investigates whether students’ engagement with AWCF conforms with conditions that support writing-skills development, L2 development, or both, as well as the individual, task, and contextual factors that influence students’ engagement with AWCF. The methodology includes the use of eye-tracking to enhance the veridicality of retrospective verbal reports.
- Jim Ranalli is investigating how use of tools for Automated Written Corrective Feedback (AWCF) affect students’ revision behaviors and features of their written output (grammatical accuracy, complexity, and writing quality) on timed writing assignments. The study compares student performances across two AWCF tools that provide feedback in real-time (i.e., while students write): the popular web-service Grammarly and the built-in proofing tools in Microsoft Word, with the latter representing a “business-as-usual” condition. The revision analyses are based on keystroke logging and screen-capture data.
- Jim Ranalli is experimenting with a new approach to second-language writing instruction called Self-regulated Learning of L2 Writing. The approach is based on a highly successful L1 writing intervention used with young learners and adults with learning disabilites—groups who, like L2 students, are challenged in their attempts to monitor and manage their engagement in complex writing tasks because of shortfalls in attentional capacity. The approach emphasizes the role of formative feedback on students’ cognitive and metacognitive activities, with the former achieved through process-tracing via the revision-history feature of GoogleDocs, and the latter by means of reflection tasks timed to coincide with key junctures in the self-regulated learning cycle.
- Tammy Slater and Barbara Schwarte are conducting research into how ESL (English as a second language) teachers make sense of, value, and apply linguistic concepts addressed in their preparatory programs. While having a strong foundation in linguistics remains a core requirement in many ESL teacher programs, there has been a lack of research on how teaching linguistic concepts to pre-service ESL teachers impacts their teaching experience as full-time ESL teachers. The investigation of this topic will provide insight into (1) the beliefs that ESL teachers have about how to use and apply linguistic concepts in their instruction and (2) the variation in these beliefs among pre-service (both early and late program) and experienced ESL teachers. In other words, the research sets out to understand what beliefs and linguistic concepts have/have not transferred from the teacher education program to the actual teaching experience, and which of these appear to be the most important to those learning and using the concepts.
- Tammy Slater is working on an edited book that looks at research and practice into language-and-content integration using Mohan’s (1986) knowledge framework, a theoretical perspective based on Systemic Functional Linguistics. The collection focuses on Mohan’s theory as it has influenced work in higher education.
- Sowmya Vajjala is working on automated assessment of linguistic complexity in reading materials using natural language processing and machine learning. Beyond computational models, she recently completed a project which involved conducting a reading study to understand how linguistic complexity affects reading comprehension. She collaborates with researchers in University of Tübingen, Germany in the Reading Demands project which aims to analyse and model linguistic complexity in German school textbooks. (URL for Reading Demands: http://www.uni-tuebingen.de/en/research/core-research/lead-graduate-school-and-research-network/research/intersection-4-language-and-learning/reading-demands-in-secondary-school-a-comparison-of-the-linguistic-complexity-of-schoolbook-texts.html)
3. Linguistic Analysis
Projects in linguistic analysis produce knowledge about language use that contribute to the scientific goal of understanding human language in a manner that can contribute to language teaching and assessment.
- Sowmya Vajjala is working on the development of a new tool to support fine grained linguistic analysis for text corpora. Along with simple word sequences, the tool currently supports a wide range of language features that look at part-of-speech, phrase chunks, and other syntactic properties of text. It is under development and the source code is publicly available (link to:https://github.com/nishkalavallabhi/textFeaturesExtraction). From a corpus analysis perspective, Sowmya is working on quantitative and qualitative methods of studying the writing patterns in l2 English among learners from different L1 background, with the aim of clustering them by their L1.
- John Levis and Greta Muller Levis are analyzing the kinds of errors that occur in the teaching presentations of international teaching assistant from various technical fields, and how these errors impact intelligibility.
- Bethany Gray and collaborators Jesse Egbert and Doug Biber (both Northern Arizona University) are working on a volume empirically exploring corpus representativeness. In the project, they investigate how corpus design decisions (such as size and sampling method) influence corpus representativeness. The goal of the project is to raise awareness of the importance of corpus representativeness, and propose methods for empirically evaluating the representativeness of language corpora.
- Bethany Gray is working on how to identify ‘text types’ within the research article register by looking at the interaction of discipline and research method. She will use dimension/factor scores from the a previous multi-dimensional analysis as the dependent variables for the cluster analysis, which groups together texts which behave similarly (i.e., which use these linguistic features in similar ways). The groups are then analyzed to identify why they use these features in similar ways (e.g., one possibility is that history writing, and qualitative political science writing, are similar in some ways, and that theoretical philosophy and theoretical physics articles are similar in other ways). The goal of the analysis is to uncover underlying factors which result in linguistically similar and dissimilar texts, without arbitrarily saying that texts are similar just because they belong to the same discipline or research type.
- Bethany Gray is building and analyzing additional subcorpora for the Academic Journal Register Corpus, a collection of 270 quantitative, qualitative, and theoretical research articles in philosophy, history, political science, applied linguistics, biology, and physics. The new subcorpora will add 180 texts representing articles with an evaluative purpose (rather than reporting primary research), such as forums, state-of-the-art synthesis, and article commentaries and author responses. Situational and linguistic analyses are carried out on the resulting corpus, adding to our knowledge of disciplinary variation beyond primary research reporting, and exploring how these articles contribute to knowledge-building and debate within their respective disciplines.
- Bethany Gray and her collaborators are investigating quantitative methods for characterizing discontinuous lexical frames, a type of formulaic language where words form a ‘frame’ around a variable slot (e.g., the * of the, in the * of). The project explores a range of measures of predictability and variability such as type-token ratios, proportions of most frequent fillers, mutual information, entropy, and ∆p (delta p) for c. 500 4-word frames) occurring at least 40 times per million words in large corpora of conversation and academic writing (c. 4.5 and 5.3 million words respectively from the Longman Corpus of Spoken and Written English).
- Elena Cotos is collaborating with Sowmya Vajjala and Carol Chapelle to identify discipline specific patterns in scientific writing by applying approaches from machine learning and natural language processing. Their current study explores different algorithms for both general move/step models and for discipline-specific ones, as well as patterns and variation within and across the Methods sections of thirty disciplines in the RWT corpus.
- Elena Cotos and Ágnes Sandor from the Xerox Research Centre Europe (XRCE) are using the RWT corpus to test an integrated method for the automated analysis of rhetorical intent. Motivated by the need to develop theoretically-grounded methods for automated analysis of academic discourse, this study integrates genre analysis and concept-matching frameworks to explore the realization of rhetorical intent through functional language from two perspectives: interpreted as moves/steps and represented as linguistically-instantiated patterns of concepts.
- Elena Cotos is collaborating with David Kaufer, Suguru Ishizaki, and Xizhen Cai from Carnegie Mellon University on a study that integrates EAP genre theory and the rhetorical theory of representational language effects. The purpose is to identify and describe patterns of language that prime interactive experiences in scientific writing. This work merges methodological paradigms by analyzing the RWT corpus annotated for moves and steps in the DocuScope text analysis environment, which allows for examining the semantic variation that distinguishes the functional meanings of rhetorical moves in disciplinary discourse.
- Elena Cotos and Sarah Huffman are analyzing functional language indicative of study design and mapping study design to rhetorical moves and steps to inform the development of customized rules for automated detection of study design elements. This work is part of the AFLEX project, a large-scale cross-disciplinary project. The AFLEX team aims to develop research synthesis technologies with the long-term goal of improving the translation of research findings from scientists to society. The project is funded through ISU’s Presidential Initiative for Interdisciplinary in Data Driven Science.
- Elena Cotos is leading several corpus-based move analysis studies aimed to comprehensively describe the conventions of different academic and professional genres. Her research teams are engaged in top-down analysis of corpora representative of the following genres: research article, grant proposal, cover letter, teaching philosophy, and research statement. The study of Broader Impacts sections in grant proposals was supported by the National Science Foundation under grant number HRD-0963584. Corpora of cover letters, teaching philosophies, and research statements are complied in collaboration with ISU’s Center for Excellence in Learning and Teaching, which is also funding the research.