Projects are presented within the three areas of emphasis of our programs in applied linguistics– language assessment, language learning and teaching and linguistic analysis. They were categorized on the basis of their primary current emphasis even though most projects are influenced by and will ultimately contribute to more than one area.
1. Language Assessment
Projects in language assessment explore issues in the development use and validation of assessments for a variety of purposes
- Carol Chapelle and Amy Hutchison are conducting foundational research informing potential future development of English language tests using mobile-technology. The goal of the research is to complete a domain analysis paper focusing on the new language practices afforded by tablet technologies in K-12 classrooms and to develop task framework, task specifications and prototype assessment tasks in collaboration with ETS staff. (funded by Educational Testing Service; 2013-2015)
- Carol Chapelle is working on the integration of concepts argument-based validity into the mainstream of language assessment through projects that make more transparent the links between past research and aspects of the validity argument. One project analyzes similarities and differences among authors’ use of argument-based validity concepts in applied linguistics and a second presents an analysis of the intersection of multiple issues to do with rating spoken and written language and validity argument.
- Elena Cotos‘ project, “Domain Description: Validating the Interpretation of TOEFL iBT® Speaking Scores for ITA Screening and Certification Purposes” investigates functional language use in the lecture, recitation, and lab curriculum genres of the academic ITA contexts to determine whether the language functions elicited by TOEFL iBT® Speaking tasks are representative of language functions used in the target domain. Implications of relevance to theory and research are also expected, as the results may inform task design, test use, and further investigations of the validity of TOEFL iBT® for instructional settings (funded by Educational Testing Service; 2015-2017).
- Elena Cotos is leading the continuous development and evaluation of the Research Writing Tutor (RWT) [https://vimeo.com/90669213], a unique automated writing evaluation tool for teaching and learning research writing in the disciplines. This project is interdisciplinary, stimulating a fruitful collaboration among faculty and graduate students in Applied Linguistics, Computational Linguistics, Human-Computer Interaction, Computer Science, and Education. The research strands focus on pedagogically-driven materials development, building computational models for discourse analysis and feedback, devising metacognitive scaffolding, and investigating the effects of RWT when integrated in learning environments.
- Bethany Gray and Volker Hegelheimer are building and analyzing a corpus of test responses to study test takers’ performance on the writing section of the English Placement Test (EPT). The goal is to examine the use of grammatical features associated with language complexity and language development in the EPT data, to investigate how these features are used differentially across placement levels, student level (graduate vs. undergraduate), and different prompts. The data analyzed in this project will be used to study language development, and inform the interpretation, development, and validation of the EPT.
- John Levis and Elena Cotos are working on a word stress intelligibility study using Indian and Korean graduate students who were rated at different levels on the Oral English Certification Test.
- Gary Ockey and Volker Hegelheimer are leading a team of graduate students in developing a performance-based oral communication assessment with the aim of using it as part of the English Proficiency Test, which is used for placement at Iowa State University. This project involves the development of test tasks, rating scales, rater training procedures, and setting cut scores. Imbedded in this project are a number of graduate student and faculty research studies, including investigating the construct of academic oral communication ability, investigating the use of generalizability theory and Multi-faceted Rasch measurement to aid in developing rating scales, and examining new task types for better assessing oral communication.
- Gary Ockey and Volker Hegelheimer are working with a group of graduate students to develop new writing tasks for the English Proficiency Test, which is used for placement at Iowa State University. These integrated skills tasks leverage technology and current understanding of the academic writing construct to better assess the academic writing skills that are needed by university students. Graduate student research embedded within this project includes investigating different techniques for developing cut-scores for a high stakes examination.
- Gary Ockey is conducting research on the issues involved in developing a rating scale for strength of accent. This project involves using human judges to rate the strength of a speaker’s accent based on how different it is perceived to be from the judge’s own accent. This study will be published in a book Co-Authored by Elvis Wagner and Ockey: Emerging Issues in the Assessment of Second Language Listening (John Benjamins, 2017).
- Jim Ranalli is investigating how feedback provided in real-time by a tool for automated writing evaluation (AWE) influences the duration and timing of cognitive writing processes during the act of writing, as well as the quality of the final written product. The research is operationalized in the CyWrite system currently under development at Iowa State University. The study uses thinkaloud, keystroke logging, screen-capture, and questionnaire data in addition to expert raters’ judgments.
- Jim Ranalli is investigating the accuracy, usefulness, and efficiency of feedback provided by Criterion, an AWE tool widely used in writing classrooms, from two perspectives: argument-based validity and instructional design. [See related presentation given at the 2014 Teacher’s College, Columbia University Roundtable in Second Language Studies, which addressed the theme Learning-Oriented Assessment.]
2. Language Learning and Teaching
Applied linguistics faculty are working on projects to investigate new approaches to language learning pedagogy as well as the construction and evaluation of learning materials.
- Gulbahar Beckett is working on several research projects including: 1) Investigation of English as a medium of instruction policies and practices in EFL contexts through document analyses, classroom observation, and interviews with students and faculty; 2) Quantitative analysis of teaching and learning language form through technology enhanced projects with Dr. Tammy Slater (Iowa State University); Dr. Sandra Zappa-Hollman (University of British Columbia), and Dr. Andrea Stiefvater (Southern Utah State University); and 3) Meta-synthesis study of published education research articles with Dr. Leigh Wang and Juanjuan Zhao (University of Cincinnati). Findings of these studies are expected to contribute to language policy and pedagogy; technology integrated second language acquisition and experiential learning; and academic and professional writing as well as research methodology.
- Carol Chapelle is investigating the selection of Québec cultural content in beginning-level French language textbooks with particular emphasis on political content. The sample includes textbooks used in the United States over the fifty year period of 1960 through 2010, whose image and textual content is analyzed from a social semiotic perspective. The chronological analysis examines choices made by textbook producers as well as the relationship of these choices to contemporary pedagogy in foreign language teaching and politics in Québec (forthcoming, Palgrave Macmillan).
- Elena Cotos‘ project, “Graduate Peer Mentor Program: Exploring forms of discipline-specific writing support,” evaluates a new model for academic writing support to graduate students and post-doctoral associates implemented by the Center for Communication Excellence. Writing support includes individual consultations on discipline-specific research writing by Graduate Peer Mentors, peer review groups, and a series of seminars and workshops. The objectives are to analyze the discipline-specific research communication needs of graduate students and post-doctoral associates; to investigate the impact of the writing support to meet these needs; and to evaluate the effectiveness and efficiency of the approach to training the Graduate Peer Mentors.
- Tammy Slater, John Levis and Greta Muller Levis have collaborated on studies of International Teaching Assistant discourse, paying attention to the intersection of prosody and Systemic Functional approaches to analysis. Two publications looked at first, the ways that Chinese, Indian and American TAs turned the same written material into spoken discourse, and second, the ways that Chinese, Indian and American TAs in STEM and non-STEM field employed spoken parentheticals in creating a communicative channel parallel to that of new and given information.
- Tammy Slater and Barbara Schwarte are conducting research into how ESL (English as a second language) teachers make sense of, value, and apply linguistic concepts addressed in their preparatory programs. While having a strong foundation in linguistics remains a core requirement in many ESL teacher programs, there has been a lack of research on how teaching linguistic concepts to pre-service ESL teachers impacts their teaching experience as full-time ESL teachers. The investigation of this topic will provide insight into (1) the beliefs that ESL teachers have about how to use and apply linguistic concepts in their instruction and (2) the variation in these beliefs among pre-service (both early and late program) and experienced ESL teachers. In other words, the research sets out to understand what beliefs and linguistic concepts have/have not transferred from the teacher education program to the actual teaching experience, and which of these appear to be the most important to those learning and using the concepts.
3. Linguistic Analysis
Projects in linguistic analysis produce knowledge about language use that contribute to the scientific goal of understanding human language in a manner that can contribute to language teaching and assessment.
- Elena Cotos conducts corpus-based genre analysis to comprehensively describe the conventions of discipline-specific discourse. Two genres are of primary interest: research articles and grant proposals. Projects centering on the research article use a corpus of 900 journal manuscripts and focus on devising and validating cross-disciplinary IMRD frameworks of communicative moves and functional steps. Similar in methodological approach, the study of the grant proposal genre focuses on the rhetorical composition of its part-genres and comparisons with the colony of promotional and academic genres. For example, a corpus of 80 NSF grant proposals from different disciplines is used to examine the internal structure of Broader Impact sections, with a secondary goal to identify differences between funded and not-funded proposals.
- Bethany Gray‘s forthcoming book (John Benjamins) examines the linguistic structure of published research articles in 6 diverse disciplines. A unique contribution of this book is the fact that it also considers the research method of the research reports as a possible variable influencing linguistic variation, something which has largely not been considered in previous research. As such, it contributes to a methodological implications for how to design corpora to represent academic writing across disciplines.
- Bethany Gray and Douglas Biber’s forthcoming book (Cambridge University Press) tracks the historical development of grammatical complexity in academic writing by tracing the development of complex phrases (rather than clauses) over the past 300 years, making comparisons of academic across time, between academic writing and other written and spoken registers (e.g., fiction, news, conversation), as well as between disciplines (humanities vs. science writing) and types of academic writing. The research challenges our beliefs about the nature of grammatical complexity, and of linguistic change in writing.
- Bethany Gray and her collaborators are investigating quantitative methods for characterizing discontinuous lexical frames, a type of formulaic language where words form a ‘frame’ around a variable slot (e.g., the * of the, in the * of). The project explores a range of measures of predictability and variability such as type-token ratios, proportions of most frequent fillers, mutual information, entropy, and ∆p (delta p) for c. 500 4-word frames) occurring at least 40 times per million words in large corpora of conversation and academic writing (c. 4.5 and 5.3 million words respectively from the Longman Corpus of Spoken and Written English).
- Bethany Gray is working with Jesse Egbert to investigate the influence of topic on internal representativeness in a specialized corpus of writing in a single discipline (applied linguistics) by evaluating the reliability of multiple linguistic features (passive voice, pronouns, grammatical complexity, and multi-dimensional analysis scores). The corpus is composed of a series of 30-text sub-corpora representing a range of topic areas within applied linguistics. Intraclass correlation coefficients are calculated for a series of test corpora created by randomly assigning texts from the full corpus into sub-corpora for which topic is not controlled (sampling with replacement), to determine whether the linguistic findings are stable across the corpus. This study is a small component of a larger project looking at the issue of corpus design and representativeness
- Bethany Gray is working on how to identify ‘text types’ within the research article register by looking at the interaction of discipline and research method. She will use dimension/factor scores from the a previous multi-dimensional analysis as the dependent variables for the cluster analysis, which groups together texts which behave similarly (i.e., which use these linguistic features in similar ways). The groups are then analyzed to identify why they use these features in similar ways (e.g., one possibility is that history writing, and qualitative political science writing, are similar in some ways, and that theoretical philosophy and theoretical physics articles are similar in other ways). The goal of the analysis is to uncover underlying factors which result in linguistically similar and dissimilar texts, without arbitrarily saying that texts are similar just because they belong to the same discipline or research type.