ASM events
This conference is managed by the American Society for Microbiology

Research Design

Table of contents
No headers

Two semesters, spring and fall of 2010

I focused my project to just study the effect on learning of dissecting the primary literature using an adaptation of the CREATE method. I am looking at the effect of using this method on student critical thinking skills and on their attitudes towards scientific research.

I am teaching the treatment section of the course (students using the adapted CREATE method) and a biology colleague, Dr. Nancy Dalman, is teaching the control section (students reading the literature without a structured method). To address the potential effect of having two different instructors, in the fall I will teach the control section and she will teach the treatment section. We will analyze our data across the two semesters.


1. Formative:

Performance on selected questions that target the analysis, evaluation and synthesis levels of Bloom's taxonomy  on the first and final exams in the lecture part of the course. These are designed using the Blooming Biology Tool (BBT; Crowe et al., 2008), and are graded using a standard rubric.

2. Pre-/post-:

a. We administered the California Critical Thinking Disposition Inventory (CCTDI; P. A. Facione & N.C. Facione, 1992) and the California Critical Thinking Skills Test (CCTST; Facione, 1990; P. A. Facione & N. C. Facione, 1994) to students at the beginning of the semester, and we will do it again at the end of the semester. These standardized tests have been validated and used to assess the particular disposition of students towards critical thinking (CCTDI) and students’ critical thinking skills (CCTST) by many groups (Kennison, 2006; Miri et al., 2007). We think that administering the critical thinking disposition inventory is important because we would like to assess independently the effect of our method for reading the primary literature on student disposition towards critical thinking as well as on actual critical thinking skills.

b. We hypothesize that reading primary literature using a method designed to get students thinking about experimental design, controls, and future experiments not only improves student critical thinking skills, but also changes their attitude from that of novices to that of "experts", in our case, scientists. To test this hypothesis, we administered the CLASS-Bio (Colorado Learning Attitudes about Science Survey-Biology) to students at the beginning the semester, and will have students complete the survey again at the end of the semester.

c.  Reflective paper. We asked students to write a one-page, reflective paper on their views on biological research at the beginning of the course. In particular, students were asked to answer the following questions: "What does the term "scientific research" mean to you? What do you think doing scientific research entails?". We will ask students to answer these questions again at the end of the semester in another reflective paper. We will then code these papers for the salient ideas students have about scientific research and biology, to determine if there was a change in student attitudes and perceptions about biology.

Data collection and analysis

We have collected all the pre- measures. We have not yet begun analyzing the data, mostly because our collaborators who have the skills to analyze it have been too busy during the semester to help us. We're hoping to do the bulk of the analysis this summer, after the end of the semester.

Tag page
Viewing 1 of 1 comments: view all
Hi Miriam

I am not sure what your question is -- are you asking which would be better, coding or a rubric? I think either could work. How many papers do you have to process? If there aren't too many, I think coding would be easier and faster. If you have a large number, or if you think you will continue this study in the future, you might want to create a rubric. However, you'll have to train at least one other rater to use your rubric, each person will have to score every student paper independently, and you'll have to report on inter-rater reliability (a measure of how consistently you both are using the rubric).

If you want to talk more about this, perhaps a quick phone call would be more efficient.

Hope that helps!
Posted 13:42, 12 Mar 2010
Viewing 1 of 1 comments: view all
You must login to post a comment.