Lab courses in the age of generative AI
I want to change the way I approach evaluations in a lab course, and I would be curious to brainstorm/crowdsource with other profs and see how they are approaching science (particularly chemistry) lab reports/grades in the context of AI.
I teach a moderate size (~400 students) second year organic chemistry lab, with ~30 TAs in charge of supervising and marking reports. My old approach for reports were questionnaire-based, which prompted the various key points to be discussed, along with independent but related post-lab questions on various techniques/concepts of interest to the course. The idea was to insure students covered all of the key elements for a given experiment, while standardizing everything for the various TAs and homogenize grading. However, being question-based, the format is too easy to exploit using AI. Even many of the post-lab questions can now be addressed or solved with AI.
I dislike grading primarily on results (at least for this course), partially because students are still learning, because accidents can easily happen (even by no fault of their own), and since they are somewhat limited in terms of what actual results they can obtain (we have advanced instruments, but the sheer number of students makes it impossible to use at scale. Many of the results they obtain here are easily forged, like the final mass/yield, unless the TAs pay really close attention in the lab). Some of the experiments involve data production and analysis of trends, but some of them essentially boil down to "prepare molecule X".
I already have an in-lab practical exam and an end-of-term final written exam. A "passing grade" (usually ~40%) is required on the latter to pass the course, the rationale being that if they did the work during the term, then the questions on the final shouldn't be surprising. This said, the average on the final has been dropping a few points every year for the past 3 years, despite the fact that many of the questions are lifted from either pre-lab quizzes or reports. I've considered adding a midterm, but that would probably add too many hours to the TA contracts, and would require cutting elsewhere. In-lab quizzes are also an issue; on top of having to print hundreds of copies weekly, I would need to write multiple versions for the various sections, and we would lose precious in-lab time, which is already tight. It would also add hours to the TA contracts.
AI is (likely) there to stay. Right now, half of the semester's grades is based on the two exams. I could make it more, but it feels a bit much for a lab course. As I can't grade entirely on results, and that I believe that being able to put out a respectable document, AI or not, is a key learning objective, part of my plan is to return, maybe paradoxically, to more traditional documents and focus on the writing and formatting parts (including figures, when applicable). However, with ~30 TAs of various levels of competence and/or willingness, I can already imagine this will be a challenge to overcome to ensure somewhat of a fair grading across the sections, even with a rubric. As for the post-lab questions, I plan on building a library of questions that will be posted online, along with a marking key, for them to practice at their own leisure. (I know many won't do it, or wait until it is too late, but that's on them.)
Bottom-line/TLDR I'd be curious to see how others approach grading in full lab courses. Do you just put more emphasis on a written/practical-exam and call it a day. Do you have any tips/elements to include in a rubric to grade written documents in an attempt to circumvent AI?