Assessing Information Literacy Instruction with TATIL
Gauging Usefulness for Theological Education Students
Abstract During the 2024-2025 academic year, BSK Theological Seminary examined the institutional assessment potential for the Threshold Achievement Test for Information Literacy (TATIL) in a master’s level seminary setting through pilot testing of students near the beginning of their academic career. The student test results themselves, while largely positive overall, were a bit mixed for some individual test takers. The pilot program was successful in helping form plans for institutional assessment and in exploring the administration of external testing and the interpretation of the results. This report addresses the basics and a few important aspects and limiting factors of TATIL, as well as including suggestions for potential use and possible implementation.
The Threshold Achievement Test for Information Literacy (TATIL) is a tool inspired by the Framework for Information Literacy in Higher Education. The tool was developed by Carrick Enterprises and is offered through the Association of College & Research Libraries (ACRL). It is divided into four modules which roughly correspond to the six frames of the Framework: modules one and three cover two frames each, while modules two and four each cover one frame per module (Threshold Achievement, n.d.-e). The modules were developed and tested over several years and considered to be both valid and reliable (Doyle et al. 2019). TATIL is geared toward undergraduates, but our school was interested in evaluating its usefulness for master’s level students in theological education, specifically with an eye toward course/institutional assessment (Threshold Achievement, n.d.-b).
BSK Theological Seminary pre- and post-tested students enrolled in INT 2501: Introduction to Theological Education during the Fall and Spring semesters of the 2024-2025 academic year with Module Three (Research and Scholarship) of TATIL. Preliminary exploration of LIS literature and other sources did not reveal a great deal of information about the test, so we were interested in piloting this module in an attempt to get a sense of TATIL’s functionality and how it might fit our assessment goals. A few articles mention TATIL, but we did not find much analysis (Drabinski 2017; Doyle et al. 2019; Davidson Squibb and Zanzucchi 2020; Ibrahim et al. 2024). Clark and Radcliff1 (2019) address TATIL in more detail in a conference presentation.
Our testing sample size was small: between the two semesters we have data for 16 students for pre-testing, but only 12 who completed both the pre- and post-tests. The difference being that a few students changed registration before post-testing and, significantly, one student in each semester did not have post-test scores recorded for what is assumed to be either technical or user error. We limited our analysis (so far) to only those students who completed both because our main goal was attempting to assess course performance by focusing on overall results for the group.
Our results tended toward improvement but were clearly mixed, with many testers having lower scores in some areas on the post-test than on the pre-test. Reporting received from testing results is extensive. Every student who completes the test can, depending on test setup, receive a summary feedback report. The test administrator receives a longer report (26 pages in our case) as well as individual scores for Knowledge Outcomes and Dispositions and a list of test questions asked. Of the four instances of testing, the list of test questions for Module 3 were all identical. What is not immediately apparent is if the questions were asked in the same order or if the listed answers were randomized. Knowing it is the same batch of questions is helpful as we evaluate our pilot and move forward. Three months is likely too short an interval for pre- and post-testing on the same question set; a few students mentioned that many of the questions seemed familiar.
There are a few practicalities to consider when using TATIL. The first is cost; there are two basic plans: pre-pay ($8 per student per module) and post-pay ($9 per student per module); there is also an unlimited plan for large scale testing (Threshold Achievement, n.d.-c). We opted for post-pay and it is worth noting that we experienced payment disruptions each of the four times we tested. These were all solvable problems and the representatives at ALA, which handles payment, were always helpful and pleasant. However, it was one more thing to have to manage every time we tested, and we could not see testing results until payment issues were eventually sorted out. Secondly, setup before testing requires many decisions to be made about things like authentication for those taking the test, informed consent for those being tested, institutions for comparing scores, and numerous other factors. Third is the time commitment for testing, as “each module is designed to be completed within 50 minutes” (Threshold Achievement, n.d.-d). Our users essentially bore that out, although we had some real outliers, both much slower and much quicker, in actual practice. With a distributed student body, the only feasible testing scenario for us was to set it up to be taken unmonitored at the time of the student’s choice.
We are still interested in using TATIL but would not suggest pre- and post-testing on a course-length timeline. We are currently considering whether pre- and post- on a degree-length timeline would give us meaningful institutional assessment data. TATIL could also be useful for individual student interventions. The detailed reporting could help this to be fairly targeted and this seems closer to its advertised purpose (Threshold Achievement, n.d.-a). We would also recommend being attentive to testing conditions. This pilot testing was very much exploratory in our case, but one hypothesis for our mixed results related to timing/testing conditions. Pre-testing occurred during the second week of class when students are typically fresh and eager but post-testing occurred during the week before finals which can find them busy, behind, and tired. It would be difficult for us at this point to justify expanding our testing to include additional modules covering other aspects of the Framework, not just in terms of increased institutional costs, but also taking into account student time spent testing at an estimated 50 minutes per module.
TATIL is an intriguing tool which was relatively easy for us to pilot test, but the costs could add up quickly with additional modules and/or a larger student population. If we were to start again, we would consider putting one or more of our staff members through all four modules first, both to get a better sense of testing experience and to gauge which modules track best with our institutional outcomes. There are few substitutes for experience and student testing related to institutional assessment could have been better informed by staff more fully engaging in the role of learner.
References
Clarke, Kathy, and Carolyn Radcliff. 2019. “Information Literacy Assessment for Instruction Improvement and Demonstration of Library Value: Comparing Locally-Grown and Commercially-Created Tests.” Proceedings of the 2018 Library Assessment Conference: Building Effective, Sustainable, Practical Assessment: December 5–7, 2018, Houston, TX, October, 796–805. https://doi.org/10.29242/lac.2018.74.
Davidson Squibb, Sara L., and Anne Zanzucchi. 2020. “Apprenticing Researchers: Exploring Upper-Division Students’ Information Literacy Competencies.” Portal: Libraries and the Academy 20 (1): 161–85. https://doi.org/10.1353/pla.2020.0008.
Doyle, Matthew, Britt Foster, and Mariya A. Yukhymenko-Lescroart. 2019. “Initial Development of the Perception of Information Literacy Scale (PILS).” Article 5. Communications in Information Literacy 13 (2): 205-227. https://doi.org/10.15760/comminfolit.2019.13.2.5.
Drabinski, Emily. 2017. “A Kairos of the Critical: Teaching Critically in a Time of Compliance.” Communications in Information Literacy 11 (1): 76–94.
Ibrahim, Sharil Effendi, Md Rosli Ismail, and Thirumeni T. Subramaniam. 2024. “Information Literacy Assessment.” International Journal of Media and Information Literacy 9 (2): 339–50. https://doi.org/10.13187/ijmil.2024.2.339.
Threshold Achievement. n.d.-a. “Home.” Accessed July 11, 2025. https://thresholdachievement.com/.
Threshold Achievement. n.d.-b. “How the Test Was Developed.” Accessed July 11, 2025. https://thresholdachievement.com/the-test/background.
Threshold Achievement. n.d.-c. “Pricing.” Accessed July 11, 2025. https://thresholdachievement.com/the-test/pricing.
Threshold Achievement. n.d.-d. “Test Administration.” Accessed July 11, 2025. https://thresholdachievement.com/the-test/test-administration.
Threshold Achievement. n.d.-e. “Test Modules.” Accessed July 11, 2025. https://thresholdachievement.com/the-test/test-modules.
Notes
1 Radcliff is a founder of Carrick Enterprises.