Ruslan Suvorov's Homepage

Research

This page contains a list of my publications and conference presentations, as well as my dissertation abstract.


Publications

Suvorov, R. (accepted). Test takers' use of visual information in an L2 video-mediated listening test: Evidence from cued retrospective reporting. In G. Ockey & E. Wagner (Eds.), Emerging issues in the assessment of second language listening. John Benjamins.

Suvorov, R. (accepted). Investigating test-taking strategies during the completion of computer-delivered items from Michigan English Test (MET): Evidence from eye tracking and cued retrospective reporting. CaMLA Working Papers.

Suvorov, R., & Carrillo Cabello, A. (2017). The development of language learner autonomy in adaptive learning systems. In M. Cappellini, T. Lewis, & A. Rivens Mompean (Eds.), Learner Autonomy and Web 2.0 (pp. 36-66). Bristol, CT: Equinox.

Gruba, P., Cardenas-Claros, M. S., Suvorov, R., & Rick, K. (2016). Blended language program evaluation. London, UK: Palgrave Macmillan.

Suvorov, R. (2016). [Review of the book Advancing quantitative methods in second language research]. CALICO Journal, 33(1), 142-145.

Suvorov, R. (2015). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 32(4), 463-483.

Suvorov, R. (2015). Interacting with visuals in L2 listening tests: An eye-tracking study. In V. Berry (Ed.), ARAGs research reports online (Report #AR-A/2015/1). Retrieved from British Council http://www.britishcouncil.org/sites/default/files/interacting_with_visuals_in_l2_listening_tests_suvorov.pdf

Suvorov, R., & Hegelheimer, V. (2013). Computer-assisted language testing. In A. J. Kunnan (Ed.), Companion to Language Assessment (pp. 593-613). Malden, MA: Wiley-Blackwell.

Levis, J., & Suvorov, R. (2012). Automatic Speech Recognition. In C. A. Chapelle (Ed.), The Encyclopedia of Applied Linguistics (Vol. 1, pp. 316-323). New York, NY: Wiley-Blackwell.

Gleason, J., & Suvorov, R. (2012). Learner perceptions of asynchronous oral CMC: Proficiency and L2 selves. Canadian Journal of Applied Linguistics, 15(1), 100-121.

Karabulut, A., LeVelle, K., Li, J., & Suvorov, R. (2012). Technology for French learning: A mismatch between expectations and reality. CALICO Journal, 29(2), 341-366.

Gleason, J., & Suvorov, R. (2011). Learner perceptions of asynchronous oral computer-mediated communication tasks using Wimba Voice for developing their L2 oral proficiency. In S. Huffman & V. Hegelheimer (Eds.), The role of CALL in hybrid and online language courses. Ames, IA: Iowa State University.

Suvorov, R. (2011). The effects of context visuals on L2 listening comprehension. University of Cambridge ESOL Examinations Research Notes, 45, 2-8.

Suvorov, R. (2010). Using Moodle in ESOL writing classes. TESL-EJ, 14(2).

Suvorov, R. (2009). Context visuals in L2 listening tests: The effects of photographs and video vs. audio-only format. In C. A. Chapelle, H. G. Jun, & I. Katz (Eds.), Developing and evaluating language learning materials (pp. 53-68). Ames, IA: Iowa State University.


Refereed Conference Presentations

Chuang, H.-Y., Suvorov, R., Tschudi, S., & Hart, E. (2017, November). Designing assessments for online language learning. Paper presented at the American Council on the Teaching of Foreign Languages (ACTFL) Conference, Nashville, TN, USA (forthcoming).

Suvorov, R. (2017, July 24). Benefits and challenges of eye-tracking triangulation in language assessment research: Evidence from two studies. Paper presented at the World Congress of Applied Linguistics (AILA) Conference, Rio de Janeiro, Brazil.

Cardenas-Claros, M., Suvorov, R., & Rick, K. (2017, July 21). A language program evaluation using an argument-based approach: A case study in Chile. Paper presented at the Language Testing Research Colloquium (LTRC), Bogota, Colombia.

Suvorov, R., Cardenas-Claros, M., & Rick, K. (2017, July 21). Using an argument-based framework for language program evaluation: A case study. Poster presented at the Language Testing Research Colloquium (LTRC), Bogota, Colombia.

Suvorov, R., & Camp, A. (2017, July 7). An exploratory study of test-takers' cognitive states using electroencephalography (EEG) and retrospective interviews in the context of L2 testing. Paper presented at the XVIIIth International CALL Research Conference, Berkeley, CA, USA.

Suvorov, R., Chuang, H.-Y., & Tschudi, S. (2017, May 19). Using digital badges in professional development of language instructors. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Flagstaff, AZ, USA.

Suvorov, R. (2017, March 20). Test-takers' use of visual information in a Video-based Academic Listening Test: Evidence from cued retrospective reporting. Paper presented as part of a colloquium titled "The impact of globalization, technology, and evolving construct conceptualizations on the assessment of L2 listening ability" at the American Association for Applied Linguistics (AAAL) Conference, Portland, OR, USA.

Gruba, P., Cardenas-Claros, M. S., Suvorov, R., & Rick, K. (2017, March 20). Language program evaluation: An argument-based approach. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Portland, OR, USA.

Suvorov, R. (2016, November 18). Test-taking strategies during the completion of multiple-choice items from the Michigan English Test: Evidence from eye tracking and verbal reports. Paper presented at the Association for Language Testing and Assessment of Australia and New Zealand (ALTAANZ) Conference, Auckland, New Zealand.

Chuang, H.-Y., Suvorov, R., Tschudi, S., & Beaver, B. (2016, November 18). Developing materials for online language learning. Paper presented at the American Council on the Teaching of Foreign Languages (ACTFL) Conference, Boston, MA, USA.

Suvorov, R., Rodriguez, J., Tschudi, S., & Chuang, H.-Y. (2016, May 14). Technology tools for language teachers and learners beyond the classroom. Workshop presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, East Lansing, MI, USA.

Suvorov, R., & Carrillo, A. (2016, May 13). Aligning and integrating adaptive learning systems (ALS) in online language courses. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, East Lansing, MI, USA.

Tschudi, S., Rodriguez, J., Suvorov, R., Chuang, H.-Y., & Beaver, B. (2016, May 13). Online training for online world language educators: Lessons learned from the design and delivery of a webinar series focusing on interaction. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, East Lansing, MI, USA.

Chuang, H.-Y., Rodriguez, J., Tschudi, S., Suvorov, R., & Potter, L. (2016, May 12). Building immersive learning experiences through simulations. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, East Lansing, MI, USA.

Suvorov, R., & Gruba, P. (2016, April 12). An argument-based evaluation of blended learning in a university-level English language program. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Orlando, FL, USA.

Carrillo, A., & Suvorov, R. (2016, April 10). Developing language learner autonomy in adaptive learning systems: Affordances and hindrances. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Orlando, FL, USA.

Rodriguez, J., Suvorov, R., Chuang, H.-Y., Tschudi, S., & Yoshioka, J. (2016, March 5). Challenge yourself: Discover project-based language learning. Paper presented at the Southwest Conference on Language Teaching (SWCOLT), Honolulu, HI, USA.

Suvorov, R., Chuang, H.-Y., Tschudi, S., & Rodriguez, J. (2015, November 22). Preparing teachers to design project-based language learning experiences. Paper presented at the American Council on the Teaching of Foreign Languages (ACTFL) Conference, San Diego, CA, USA.

Chuang, H.-Y., Jiang, S., Suvorov, R., & Hart, E. (2015, November 21). Designing online language pedagogy for online language teachers. Paper presented at the American Council on the Teaching of Foreign Languages (ACTFL) Conference, San Diego, CA, USA.

Rodriguez, J., Chuang, H.-Y., Suvorov, R., & Tschudi, S. (2015, May 29). Weaving technology with project-based language learning (PBLL). Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Boulder, CO, USA.

Carrillo, A., & Suvorov, R. (2015, May 28). Placing and spacing millennials in an adaptive learning system: Are they a good match? Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Boulder, CO, USA.

Tschudi, S., Rodriguez, J., Chuang, H.-Y., & Suvorov, R. (2015, May 28). Badge of skill: Faculty training for online course development. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Boulder, CO, USA.

Chuang, H.-Y., & Suvorov, R. (2015, April 11). Using Rich Internet Applications (RIAs) for creating language learning activities. Workshop presented at the Hawaii Association of Language Teachers (HALT) Conference, Honolulu, HI, USA.

Suvorov, R. (2015, March 21). The impact of note-taking on L2 learners' viewing behavior and test performance during video-based L2 listening assessment. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Toronto, ON, Canada.

Suvorov, R. (2015, March 18). Test-takers' use of visual information from context and content videos in the Video-based Academic Listening Test. Paper presented at the Language Testing Research Colloquium (LTRC), Toronto, ON, Canada.

Carrillo, A., Santos, V., & Suvorov, R. (2014, May 9). Assessment practices, issues, and implications in online language classes. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Athens, OH, USA.

Cotos, E., Huffman, S., Link, S., Saricaoglu, A., & Suvorov, R. (2014, May 8). Automated feedback on rhetorical quality: Evaluation and iterative refinement. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Athens, OH, USA.

Suvorov, R., & Carrillo, A. (2014, May 8). Bridging content knowledge and language knowledge in Spanish for Healthcare Professionals courses: How can CALL help? Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Athens, OH, USA.

Suvorov, R. (2014, March 23). The use of eye-tracking technology in research on video-enhanced L2 academic listening assessment. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Portland, OR, USA.

Malinowski, D., & Suvorov, R. (2013, October 25). Choices, choices, choices: Creating videos for the flipped language classroom. Paper presented at the New England Regional Association for Language Learning Technology (NERALLT) Conference, Manchester, NH, USA.

Suvorov, R. (2013, September 21). The effect of context and content videos on L2 learners' performance on a Video-based Academic Listening Test (VALT). Paper presented at the Midwest Association of Language Testers (MwALT) Conference, East Lansing, MI, USA.

Suvorov, R. (2013, May 25). Performance on video-enhanced L2 listening assessment tasks in Moodle: Evidence from eye tracking. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Honolulu, HI.

Suvorov, R. (2012, November 2). Validity argument for a Video-based Academic Listening Test (VALT). Paper presented at the East Coast Organization of Language Testers (ECOLT) Conference, Washington, D.C.

Cotos, E., Huffman, S., Link, S., Paben, T., & Suvorov, R. (2012, October 20). From genre analysis to the teaching of genre. Paper presented at the Mid-America Teachers of English to Speakers of Other Languages (MIDTESOL) Conference, Ames, IA.

Suvorov, R. (2012, October 6). Interpretive argument for a Video-based Academic Listening Test. Paper presented at the Midwest Association of Language Testers (MwALT) Conference, Urbana, IL.

Link, S., Paben, T., Saricaoglu, A., Suvorov, R., & Cotos, E. (2012, September 22). Evaluating the accuracy of machine-based feedback on research article Introductions. Paper presented at the Tenth Annual Conference on Technology for Second Language Learning (TSLL), Ames, IA.

Suvorov, R. (2012, June 15). Perceptions and preferences of technology use in a third-year French class: A case study. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, South Bend, IN.

Cotos, E., Huffman, S., Link, S., Paben, T., & Suvorov, R. (2012, June 10). Teaching research writing: A genre analysis approach to materials development. Paper presented at the Applied Linguistics and Materials Development (MATSDA) Conference, Limerick, Ireland.

Suvorov, R., & Cotos, E. (2012, June 10). Developing materials and tasks with annotated corpora. Poster presented at the Applied Linguistics and Materials Development (MATSDA) Conference, Limerick, Ireland.

Suvorov, R. (2012, April 4). Interacting with visuals in L2 listening tests: An eye-tracking study. Work in progress, presented at the Language Testing Research Colloquium (LTRC), Princeton, NJ.

Suvorov, R. (2011, September 17). Interacting with visuals in L2 listening tests: An eye-tracking study. Poster presented at the Midwest Association of Language Testers (MwALT) Conference, Ames, IA.

Xu, J., Cotos, E., & Suvorov, R. (2011, May 19). Developing CAPT tasks in WebSwami. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Victoria, BC, Canada.

Suvorov, R., Cotos, E., & Xu, J. (2011, March 26). Evaluation of WebSwami for L2 pronunciation instruction: A theory-based approach. Paper presented at the American Association for Applied Linguistics (AAAL) Conference, Chicago, IL.

Gleason, J., & Suvorov, R. (2010, September 10). Language learner perceptions of technology-based tasks using Wimba Voice: L2 oral proficiency, motivation, and L2 selves. Paper presented at the Eighth Annual Conference on Technology for Second Language Learning (TSLL), Ames, IA.

Gleason, J., & Suvorov, R. (2010, August 19). Language learner perceptions of Wimba Voice Board: L2 oral proficiency, motivation, and L2 selves. Paper presented at the Antwerp CALL (Computer Assisted Language Learning) 2010 Conference, Antwerp, Belgium.

Carrillo, A., & Suvorov, R. (2010, June 10). Process writing in a hybrid environment: Students' perceptions of technology. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Amherst, MA.

Chapelle, C., Dzekoe, R., Jun, G., Karabulut, A., LeVelle, K., Li, J., & Suvorov, R. (2009, September 19). Technology and speaking: What do the students say? Paper presented at the Seventh Annual Conference on Technology for Second Language Learning (TSLL), Ames, IA.

Suvorov, R. (2009, September 17). Effectiveness of ASR-based feedback on L2 pronunciation. Paper presented at the inaugural Conference on Pronunciation in Second Language Learning and Teaching (PSLLT), Ames, IA.

Karabulut, A., LeVelle, K., Li, J., & Suvorov, R. (2009, March 14). Technology for French learning: A mismatch between expectations and reality. Paper presented at the Computer Assisted Language Instruction Consortium (CALICO) Conference, Tempe, AZ.

Suvorov, R. (2008, September 27). Context visuals in L2 listening tests: The effectiveness of photographs and video vs. audio-only format. Paper presented at the Sixth Annual Conference on Technology for Second Language Learning (TSLL), Ames, IA.

Suvorov, R. (2008, September 20). The effect of visuals on test-takers' performance in L2 listening tests. Paper presented at the Midwest Association of Language Testers (MwALT) Conference, Iowa City, IA.


Dissertation Abstract

Interacting with visuals in L2 listening tests: An eye-tracking study

Visual information plays an important role in second language (L2) listening comprehension (Anderson & Lynch, 1988; Field, 2008; Rost, 2011), yet visuals have seen limited use in the assessment of L2 listening. The limited use of visuals in listening tests can be partially attributed to the lack of solid empirical evidence about how visuals are viewed during such tests and what impact they have on test performance. In particular, existing research has produced mixed results regarding the effect of visual information on media-enhanced L2 listening test performance of language learners, whereas their viewing behavior during such assessment has not been explored in detail (Ockey, 2007; Wagner, 2007, 2010a). To address this gap, the present study employed eye-tracking technology to investigate the extent to which L2 learners view context and content videos during a Video-based L2 Academic Listening Test (VALT), learners' self-reported perceptions and use of the two video types, and the effect of these visuals on their test performance.

This mixed-methods study was based on Creswell and Plano Clark's (2007) data transformation model of the triangulation design and addressed five research questions that investigated (a) the appropriateness of statistical properties of test scores for norm-referenced decisions, (b) differences between scores on the subtests associated with different video types and between scores on the video and audio-only versions of the test, (c) learners' viewing patterns with regard to context and content videos, (d) learners' use of visual information when watching the two video types, and (e) learners' use of visual information when answering individual test questions. Three sets of data were collected and analyzed in the study. Test performance data consisted of L2 learners' scores on the Video-based Academic Listening Test (n = 75) and its audio-only version called the AALT (n = 46), which were analyzed using paired-samples and independent samples t tests, as well as descriptive statistics, reliability analysis, item analysis, and distractor analysis. Eye-tracking data were collected using eye-tracking technology and included the recordings of the participants' eye movements (n = 33), which were analyzed using descriptive statistics, paired-samples t tests, and correlation analysis. Retrospective verbal data obtained via cued retrospective reporting were composed of 33 participants' verbalizations regarding their use of visual information when watching the two types of videos and their perceptions of the helpfulness of this information for answering individual test questions.

The results demonstrated that scores on the VALT and on the AALT, which were developed for this study, were appropriate for making norm-referenced decisions. While the analysis of test performance data found no effect of visuals on L2 learners' test scores, the use of eye-tracking technology was instrumental in detecting the different effects of the context and content visuals. Moreover, the results revealed differences between context and content videos in terms of their perceived use during the test-taking process and their perceived helpfulness for answering questions on the VALT. This study has valuable practical and theoretical implications with regard to the use of visuals for L2 listening instruction and assessment.

This research was funded by the TOEFL's Small Grant for Doctoral Research in Second Language Assessment and the British Council Assessment Research Award.