Field testing
The activities were administered in two undergraduate Cell Biology courses at two different private universities, Boston University and Boston College, in Spring 2021 by two different instructors. In one class, paper assignments were incorporated into hybrid discussion sections. In the other class, paper assignments were distributed throughout online lectures. Students were given five surveys throughout the semester; a preactivity survey, surveys after each of the three paper assignments was completed, and a postactivity survey after the final poster presentation (Appendix S4). There were a total of 156 students that completed all surveys across both classes.
For the purposes of this study, we combined the data across both classes due to the similarity of students across classes. Both universities are private universities in Boston, MA. Both classes were large classes of approximately 100 students each. They were both intermediate-level classes consisting of mostly sophomores. Each class read the same three papers over the semester, and both classes were provided with similar materials for instruction and assessment.
One concern with spending class time on skills rather than content is that it may take away from learning the content (
13). Students were asked about this in the last survey. Students reported that they felt spending time on scientific communication skills did not interrupt their learning of the course material; in fact, they reported that it helped them learn course content and engage with the content. We do note, however, that we do not have previous data for comparison, so these data are only representative of student perceptions, and further examination would be required to draw conclusions. Students agreed to the following statements on a Likert scale of 6 points: (i) reading primary scientific papers that related to course content enhanced my understanding of the content covered in class (4.8/6); (ii) it is useful to read scientific papers that relate to class content (5.1/6). Below are quotes from student surveys about how it was useful to connect course content to scientific papers and scientific communication skills.
“When we read about topics that related to what we learned it showed me exactly how it can be applied and it added to my understanding of how that protein or pathway functions.”
“The content is closely related with class material and helps me to understand more how to read a scientific paper.”
“Reading about how the content covered in class was applied in the experiments and studies helped me better visualize and understand it.”
Evidence of student learning
Student learning and skill development were assessed using assignments designed for each of the three papers (Appendices S1 to S3). The paper assignments included questions for students to develop and demonstrate each of the learning outcomes. The average score across both classes for all three paper assignments was 96%, demonstrating successful progress toward the learning objectives at the formative assessment stage. Content knowledge from the papers was also assessed in exam questions as summative assessments. Example exam questions are included in the instructor documents for each paper (Appendices S1 to S3). On average, 76% (±19% [standard deviation]) of students in both classes answered questions related to the papers correctly. In comparison, the average exam score across both classes was 83% (±6%). In conclusion, averages for paper questions were similar to the average performance on general knowledge questions, showing sufficient achievement of the content learning objectives at the summative assessment stage.
Development of all scientific communication skills learning outcomes was also assessed by a summative assessment in the form of the poster presentation. Students worked in groups of four to five students to choose a primary scientific article on a topic related to class, and they presented the results from the paper in a poster presentation (the rubric is included in Appendix S6; also, see sample student posters in Appendix S9). The average score across both classes for the poster presentations was 91%, demonstrating excellent achievement of scientific communication learning objectives at the summative assessment stage. The poster presentation assessed achievement of all learning objectives: reading a primary scientific paper, analyzing data in a primary scientific paper, drawing conclusions from a primary scientific paper, writing a summary of results from a primary scientific paper, orally presenting data from a primary scientific paper, and creating visual representations of scientific concepts.
Students were asked about which components of the class were most beneficial to them in learning and gaining confidence in scientific communication skills. Iterative coding of their responses revealed six course components that students felt contributed the most to the improvement of their communication skills: the quizzes, the poster project, the paper assignments, participating in class, explaining concepts to peers, and discussions with peers and instructors (
Table 1). The most-often-identified course component was the poster project (44% of coded responses). Students cited the opportunity to practice or demonstrate their skills independently in a “real-world scenario” made the poster project valuable.
We also measured student self-efficacy of the learning objectives in five surveys distributed throughout the semester (
Fig. 2 and Appendix S4). Student self-efficacy increased over the semester for all learning objectives. The data were compared by a related-samples Friedman’s two-way analysis variance-by-ranks tests. If the null hypothesis was rejected, pairwise comparisons were made and significance values were adjusted by the Bonferroni correction for multiple tests (significance values are reported in Appendix S10). Interestingly, we found that for some of the learning objectives, students decreased in confidence or did not increase in confidence from survey 1 to survey 2. This may demonstrate the Dunning-Kruger effect, which suggests that novices often overestimate their own ability (
35). Since survey 1 was distributed before students had any exposure to the skills, they may have overestimated their confidence in those skills and readjusted their perspective for survey 2 after being exposed to assignments where they had to use those skills.
We asked students if their interest in research or a career in scientific research changed during the course of the semester and to explain why in a qualitative question in survey 5. Sixty-four percent of coded responses indicated no change in student interest in research, while 31% of responses indicated an increased interest in research or research careers (
Fig. 3). A small proportion of responses (3%) indicated a change in interest but were not clear about the direction, and 2% of responses showed a decrease in interest in a research career. Student quotes suggested that interest in research increased because of increased self-efficacy and understanding about the research process in terms of techniques, diversity of topics, and broader impact (
Table 2).
At first glance, the lack of change in interest level was surprising; however, of the responses that indicated no change, 33% cited that they were already interested in research, and 14% indicated that they were interested in a medical or clinical career. This likely reflects the population of an intermediate-level biology course at a research-heavy institution; many students take this course as part of a pre-medical school track. These assignments may result in a change of interest in research in a different population, such as introductory or nonmajors courses. These data also suggested that many students in the course believed that a medical career was exclusive of a research career, or that reading and interpreting papers may not be a part of a medical career. Additionally, student quotes showed a lack of understanding about what a research career entails.
“I have never been very interested in sitting down and doing lab work just because I’m a very hands on person. The class didn’t really change that mindset at all.”
“But I still think that the social aspect of a clinical job is a key component for me and research lacks it.”
These quotes, and the idea that a career in medicine is exclusive of a research career, could present an opportunity to talk more explicitly about the training for a career in research or the myriad of ways that medical careers interface with research.
We were also interested in whether students who reported increased confidence in multiple areas of scientific communication were also students that had an increased interest in research. We categorized students with an increase in confidence as any students that increased confidence by any amount in seven of the nine tested categories from survey 1 to survey 5 (other surveys were excluded, for simplicity). We categorized students with an increase in research interest as any student that increased agreement from survey 1 to survey 5 to the question “I am interested in research (for example, reading research papers, attending research seminars, conducting research).” We saw a significant correlation between students with increased self-efficacy in scientific communication skills and those that reported increased interest in research, based on chi-square analysis (two-tailed P value = 0.0115).