Introduction
Advances in digital technology have created a pathway for reducing textbook costs by offering e-books, which are generally cheaper than traditional textbooks (Hanson 2022). Digital technology has also offered a means for widely distributing open-educational resources (OER), which can then be accessed by students online, often at no cost to them—presuming they have internet access. Digital OER can help students access course material—both from a reading and financial standpoint (e.g., Bliss et al. 2013; Delimont et al. 2016; Grissett and Huffman 2019; Hilton 2016; Watson et al. 2017). Research has also generally found that OER achieves comparable learning outcomes for students (e.g., Grimaldi et al 2019). Despite many benefits, admittedly, there are still barriers to faculty adopting OER, including concerns about the perceived workload needed to investigate and potentially modify available OER options (McGreal 2019). Considering the workload OER often requires begs the question of what could incentivize adopting OER. One such potential incentive might be the impact on students’ perception of their professors when they opt for lower-cost OER. The current study aims to replicate and extend previous work exploring this question.
Background
Student perceptions of instructors who use OER
Existing literature suggests that students’ appreciation for improved accessibility of digital OER may spill over into students’ perceptions of the faculty who use such materials. For example, students reported a higher satisfaction rating of their professor when the course used free- or reduced-cost materials; although those professors were not necessarily seen as more effective teachers (Fine and Read 2020, 167–68). Similarly, Nusbaum and Cuttler (2020) found that professors randomly assigned to use an OER textbook for their introductory psychology course, as compared to a traditional textbook, were rated more favorably by their students in an end-of-the-semester survey. Because such studies used ratings of real professors, it afforded their results higher external validity. That also means that the ratings of the professors could be due to several factors beyond their professor’s use of OER.
As an alternative approach, Vojtech and Grissett (2017) presented participants with descriptions of professors that manipulated whether the professor allegedly used a more expensive traditional textbook or lower-cost OER. Their preliminary evidence suggested that students might perceive a professor more positively—kinder, more encouraging of students, and more knowledgeable—when the described professor assigned the lower-cost open-source option. Further, by reducing potential confounding variables that occur within a more field-based study, their study more clearly pointed to the use of OER being the cause of the positivity participants felt about the professor. That is not to say that their results were without limitations.
Limitations of past work
As Vojtech and Grissett (2017) noted themselves, there were limitations with their early evidence, including its basis on a small sample of 23 participants enrolled in an upper-level psychology course (166–67). The small sample meant that while Vojtech and Grissett varied factors like the professor’s gender, length of teaching experience, and whether the professor opted to use the textbook “as-is” or to customize it for their class, it would be difficult to consider whether these variables interacted with the professor’s choice to use OER in shaping participants’ perceptions. Considering the sample consisted of students in an upper-level psychology course—who would predominantly represent psychology majors, or potentially other social/behavioral science students—and asked about a psychology professor, it was also unclear whether these results would generalize when students were looking at a professor in other fields of study. Finally, as Vojtech and Grissett noted, their design set up participants to make a direct comparison between the two professors, and the wording “only $30” for the OER professor may have portrayed that target more positively (167).
Addressing limitations: overview of the current study
To address some of these limitations, the current study made a few alterations. First, a larger sample was drawn from an introductory psychology course; this sample recruitment pool aligned with Nusbaum and Cuttler’s (2020) approach for Study 1. The advantage of recruiting from an introductory psychology course was two-fold: 1) the majority of these students were first-year students who would have had limited previous experience with open-source textbooks and purchasing textbooks for college-level classes; and 2) the students in the course tend to represent a wide range of majors beyond psychology, and in fact, beyond social and behavioral sciences (Altman et al., 2021). Given participants were completing the study through their introductory psychology course, and all sections of that course used the OpenStax Psychology textbook, the current study shifted the hypothetical professor to other fields; modeling Nusbaum and Cuttler’s second study, the professor’s field of study was either biology or history.
To avoid some of the issues with students comparing two professors who vary only in terms of their textbook cost, the current study employed a between-subjects design: all participants read about and rated a single professor, with reduced language bias in that single description. The current study also was able to consider some variables that might interact with the professor’s choice to use OER or a traditional textbook. Specifically, retaining the manipulation of Vojtech and Grissett (2017), the current study explored whether altering the textbook, leading to a more customized portrayal of the content for students, or using the book “as is” influenced students’ views of the professor, whether they opted to use OER or a traditional textbook. Because OER is often touted for its potential to be readily adapted by instructors, this manipulation seemed particularly valuable in understanding students’ perceptions of professors based on textbook choices. Of course, traditional publishers will also sometimes offer this flexibility, and in fact, assist in modifying such materials, reducing the level of effort an instructor needs to invest to adapt course materials.
Whether adapting OER or traditional materials, one argued value is that customizing the textbook can produce a more relevant textbook that students can better connect with. That potential outcome might be in part supported by previous studies that find students feel customized content was more relevant to their class (e.g., Bliss et al. 2013). It seems reasonable that if students find such materials relevant, they might be appreciative of professors who put in that effort, feeling that those professors care about their students. Another proposed benefit of customizing course materials is equity, in that professors can create materials that meet their diverse classroom of students where they are (e.g., de los Arcos et al. 2016; Van Allen and Katz 2020). However, research seems limited to support these ideas, including whether students perceive modified textbooks in this way.
Hypotheses
Given previous findings suggest that using OER might be associated with more positive evaluations of professors (e.g., Fine and Read 2020; Nusbaum and Cuttler 2020; Vojtech and Grissett 2017), the current study hypothesized that 1) participants would have more positive views of the professor who used a no-cost/low-cost open-source textbook, as compared to a high-cost textbook obtained from a traditional publisher. Additionally, as some research suggested, students might appreciate customized textbooks (e.g., Bliss et al. 2013), it was hypothesized that 2) participants would have more positive views of the professor who modified the textbook, as compared to using the textbook “as is.” If these two variables interacted, it seemed reasonable that they might have additive effects such that 3) participants would rate the professor who used a modified low-cost textbook as most positive, and the professor who used a high-cost “as-is” textbook as least positive. No specific predictions were made about the impact of the field of study; ideally, the field would have no impact, suggesting that any effects of using OER would generalize to different fields of study.
Methods
Design
The study used a 2 (textbook cost: high-cost/publisher’s textbook vs. low-cost/open-source textbook) x 2 (textbook adaptation: as is vs. modified) x 2 (subject area: biology vs. history) experimental between-subjects design. The choice to vary the field of study was driven by several factors. Participants were completing the study through their introductory psychology course, which uses an open-source textbook that is available to students digitally for free; thus, it seemed best to avoid using psychology as a field of study. Biology and history were selected for two reasons: 1) Nusbaum and Cuttler (2020) used biology and history in their second study, and thus there was methodological precedence for using these fields, and 2) both fields represented common courses taken by students for their general education curriculum at the current institution.
Procedures
To safeguard participants, the study was reviewed and approved through the IRB. Introductory psychology students were invited to participate in the study to earn partial credit toward their experiential learning grade. Students had several options besides the current study to earn credit toward this grade. To be eligible, participants had to be 18 years of age or older. When students signed up for the study, they were directed to the online survey which they could complete based on their availability.
Manipulations of independent variables
After completing the informed consent, participants read a brief description of a professor. These descriptions largely replicated those used by Vojtech and Grissett (2017), with a few alterations:
- The professor was not designated by name. Gender-based pronouns were changed to “they.” Participants were informed this was to protect the identity of the professor in the description. The alteration aimed to reduce the potential effect of the professor’s gender on participants’ perceptions, as well as interaction effects between the professor’s gender relative to the participant’s gender.
- Teaching experience was held constant at 10 years, rather than varying it between 5 and 25 years, as teaching experience was not a variable of interest in the current study.
- The description varied whether the professor taught in the field of biology or history, to demonstrate the generalizability of any findings across different areas of study.
Participants were informed that identifying information, including gender-based pronouns, had been removed. Beyond the independent variables, all descriptions were highly similar:
Professor A is a [biology/history] professor. They have been teaching in this field for 10 years and really enjoy their job. In class, they often use a mixture of lecturing, videos, and activities to help students learn. Professor A expects that students come to class prepared, having studied the required readings for the day ahead of time. The textbook they use is [a popular publisher’s textbook; students can get a digital copy for $175 or a printed copy for $250/an open-source book; students can get a digital copy for free ($0) or a printed copy for $40]. Professor A uses [the textbook “as-is” directly from the publisher, with no alterations to the book’s content. Although that means the content sometimes feels a little less personally relevant for the students in Professor A’s class, it does make it easier for students to sell their textbooks at the end of the term/modifies the textbook from the publisher and customizes the book’s content for their students. While this customization makes the course content more personally relevant for students in Professor A’s class, it does make it harder for students to sell their textbooks at the end of the term.] Overall, the textbook has received good student reviews and appears to help students better understand the material.
Participants saw a reminder that they would be quizzed on the description, to encourage them to thoroughly read the description before moving on. Three of the four quiz questions served as manipulation checks, to confirm that participants paid attention to the professor’s subject and textbook choices; a fourth question asked about how long the professor had been teaching, which was consistent across all descriptions. Participants who failed to answer all four questions of the quiz correctly were excluded from the data analysis.
Measurement of dependent variables
To measure participants’ perceptions of the professor, various close-ended statements were derived from the work of Vojtech and Grissett (2017) and Nusbaum and Cuttler (2020): “The professor…
- …cares about students.”
- …lacks knowledge of their field.”
- …is very enthusiastic about teaching in their field.”
- …is supportive of students.”
- …lacks creativity in their approach to teaching.”
- …is committed to student learning.”
- …is committed to creating an equitable learning environment.”
- …cares about making course materials accessible to students.”
- …works to represent diverse voices in the class.”
- …wants their students to relate to the course material.”
The display of the statements was randomized. For each item, participants indicated to what degree they felt the statement was true, ranging from 1, being not at all true, to 4, being entirely true. Considering that positive impressions may drive decision-making when it came to enrollment, participants were also asked to rate how likely they would be to take a course with this particular professor if they had to take a class in the professor’s field and if they did not have to take a class in the professor’s field; for both of these items, participants responded on a scale with 1, being extremely unlikely, to 5, being extremely likely.
Before completing the study, participants reported on demographics, including their number of earned credits to represent class standing, their field of study, and whether they were the first in their family to attend or earn a college degree. Participants described their gender identity and race/ethnicity using open-ended responses to allow participants to use language that was suitable to them. Finally, participants confirmed whether they wished for their data to be included in the analysis; this technique was employed to reduce any pressure on participants. Any participants who opted to exclude their data from analyses still earned credit toward their experiential learning component of the course.
Analysis of data
Before conducting any analysis, responses were removed for any participants who indicated they did not wish their data to be included in analyses. Additionally, responses from participants who failed to answer the four manipulation check questions correctly were removed; based upon their performance in these manipulation check items, it was unclear if they had thoughtfully processed the manipulation and thus would be unlikely to have been affected by the manipulation. Due to the multifactorial design and the multiple dependent variables, multiple analysis of variance (MANOVA) was used to avoid problems with repeated testing.
Results
Excluded data
Of the 190 participants who completed the study, 5 (3%) reported that they wished their data to be excluded from data analysis. Another 32 (17%) participants were excluded based on their failure to answer the manipulation checks correctly. This left 153 total participants included in further data analysis.
Demographics
Most of the participants were in their first year of college: 44% had earned 0 college credits and 30% had earned less than 29 credits before the current term. The remaining participants were in their second (11%), third (9%), or fourth year (3%). The sample represented various fields of study, with the largest portion of participants representing the health professions (33%) and/or social and behavioral sciences (20%). There were at least some students in all other areas: arts (7%), business (12%), education (13%), humanities (6%), life/natural sciences (4%), and physical sciences (12%). A small portion of participants (26%) reported that they were the first person in their immediate family to attend college, and another 6% reported that they would be the first in their immediate family to earn a college degree. Approximately 74% of the sample identified as White/Non-Hispanic, including those who identified themselves as European American or “Caucasian.” The majority of participants (67%) identified as a woman or female, while 29% identified as a man or male, and 2% identified as non-binary or genderfluid; the remaining participants (2%) did not provide anything in terms of gender identity.
Overall perceptions of the hypothetical professor
In an exploratory analysis, ratings of the hypothetical professor appeared to be correlated (see Table 1). Of the evaluative statements that seemingly stood by themselves was the degree to which the professor seemed to lack knowledge in their field. Despite the relationships between ratings, no efforts were made to create a singular measure by combining ratings, to better examine any nuances in terms of perceptions. All reported means represent the mean for that individual rating across all participants within the identified conditions.
| Judgment: The professor… | M (SD) | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1. cares about their students. | 3.27 (0.75) | -0.15 | 0.45c | 0.64c | -0.23b | 0.47c | 0.41c | 0.41c | 0.29c | 0.38c | 0.27b |
| 2. lacks knowledge in their field. | 1.12 (0.49) | -0.06 | -0.05 | 0.18a | -0.18a | -0.04 | -0.10 | -0.07 | -0.10 | -0.11 | |
| 3. is very enthusiastic about teaching in their field. | 3.28 (0.76) | 0.39c | -0.31c | 0.44c | 0.24b | 0.36c | 0.03 | 0.28b | 0.43c | ||
| 4. supportive of students. | 3.20 (0.69) | -0.17a | 0.48c | 0.47c | 0.41c | 0.35c | 0.32c | 0.25b | |||
| 5. lacks creativity in their approach to teaching. | 1.67 (0.92 | -0.21c | -0.12 | -0.26b | -0.12 | -0.44c | -0.49c | ||||
| 6. committed to student learning. | 3.44 (0.63) | 0.37c | 0.30c | 0.22b | 0.33c | 0.43c | |||||
| 7. committed to creating an equitable learning environment. | 3.10 (0.86) | 0.52c | 0.41c | 0.31c | 0.19a | ||||||
| 8.tries to create an inclusive learning environment. | 2.99 (0.80) | 0.37c | 0.50c | 0.34c | |||||||
| 9.cares about making course materials accessible to students. | 3.16 (0.98) | 0.14 | 0.07 | ||||||||
| 10.works to represent diverse voices in the class. | 2.25 (0.88) | 0.42c | |||||||||
| 11.wants their students to relate to the course material. | 2.77 (1.10) |
a: p < .05, b:, p < .01, c: p < .001
Hypothesis testing
Hypothesis 1 predicted that participants would view the professor who used a no-cost/low-cost open-source textbook more positively than the professor who used a high-cost traditional textbook. Overall, the results provided limited support for Hypothesis 1. Participants appeared to feel that the hypothetical professor cared more about making course materials accessible (M = 3.71, SD = 0.56), relative to the professor who opted for the high-cost textbook (M = 2.63, SD = 1.02), F (1, 145) = 68.66, p < .001, ηp2 = .32. No other statistically significant differences emerged that would suggest opting for a low-cost textbook created more favorable perceptions of the professor (ps >.06). As another means to examine positivity towards the professor, participants’ reported likelihood of taking a course with the hypothetical professor was examined; although textbook cost did not impact this measure for courses that participants would have to take, F (1, 145) = 1.97, p = .16, ηp2 = .01, it did have a statistically significant impact when students did not have to take a course in the professor’s field, F (1, 145) = 5.10, p = .03, ηp2 = .03. Participants reported being more willing to take a course with the professor who opted for the low-cost textbook (M = 2.97, SD = 1.16) as compared to a professor who opted for the high-cost textbook (M = 2.53, SD = 1.30); of note, neither of these means exceeded the mid-point of the scale. Factoring in the field of study did not alter these findings (interactions: ps > .06).
Hypothesis 2 predicted that participants would view the professor who modified the textbook more positively than the professor who retained the book “as is.” This prediction was supported across several ratings (see Table 2 for statistically significant results). Unlike what was noted with cost, modifying the textbook did impact participants’ reported likelihood of taking a course with the hypothetical professor if they had to take the course, F (1, 145) = 4.99, p = .03, ηp2 = .03, but not if they did not have to take a course in the professor’s field, F (1, 145) = 2.62, p = .11, ηp2 = .02. For the former main effect, participants reported being more willing to take a course with the professor who opted to modify the textbook (M = 4.01, SD = 1.00) relative to the professor who kept the textbook “as is” (M = 3.63, SD = 1.10), providing some additional support for Hypothesis 2. Factoring in the field of study did not alter any of these findings (interactions: ps > .35).
| Judgment: The professor… | MANOVA Results | “As Is” Condition (n = 75) M (SD) | Modified Condition (n = 78)M (SD) |
|---|---|---|---|
| …cares about their students. | F (1, 145) = 11.29, p < .01, ηp2 = .07 | 3.08 (0.78) | 3.46 (0.66) |
| …is very enthusiastic about teaching in their field. | F (1, 145) = 34.96, p < .001, ηp2 = .19 | 2.95 (0.80) | 3.60 (0.54) |
| …is supportive of students. | F (1, 145) = 6.72, p = .01, ηp2 = .04 | 3.05 (0.72) | 3.33 (0.64) |
| …lacks creativity in their approach to teaching. | F (1, 145) = 34.17, p < .001, ηp2 = .19 | 2.07 (0.96) | 1.28 (0.68) |
| …is committed to student learning. | F (1, 145) = 28.13, p < .001, ηp2 = .16 | 3.20 (0.68) | 3.68 (0.47) |
| …tried to create an inclusive learning environment. | F (1, 145) = 12.80, p < .001, ηp2 = .08 | 2.77 (0.78) | 3.21 (0.76) |
| …works to represent diverse voices in the class. | F (1, 145) = 25.24, p < .001, ηp2 = .15 | 1.92 (0.80) | 2.58 (0.83) |
| …wants their students to relate to the course material. | F (1, 145) = 104.28, p < .001, ηp2 = .42 | 2.05 (1.00) | 3.46 (0.66) |
Hypothesis 3 addressed the potential interaction between the cost of the textbook and whether or not it was modified. Specifically, it predicted that participants would rate the professor who used a modified low-cost textbook as most positive, and the professor who used a high-cost “as-is” textbook as least positive. This hypothesis was partially supported on a single rating, F (1, 145) = 4.66, p = .03, ηp2 = .03 (see Figure 1). For a low-cost textbook, participants rated the professor as more caring when using a modified textbook, instead of using the book “as is,” t (70) = -4.04, p < .001. For the high-cost textbook, however, there was no statistically significant difference based on modification, t (79) = -0.86, p = .39. Cost seemed to have a statistically significant effect when the book was modified, t (76) = -2.06, p = .04, but not if the book was kept “as is”, t (73) = 1.17, p = .25.
When factoring in the subject area, a three-way interaction emerged, affecting whether participants saw the professor as committed to creating an equitable learning environment, F (1, 145) = 4.87, p = .03, ηp2 = .03 (see Figure 2). This interaction seemed largely driven by perceptions of the history professor: no statistically significant differences emerged related to perceptions of the biology professor (ps > .12), but a two-way interaction emerged for the history professor, F (1, 74) = 5.04, p = .03, ηp2 = .06. In breaking down this interaction, modifying the low-cost textbook was seen more favorably than keeping it “as is,” t (36) = -2.71, p = .01, while there was no statistically significant difference between modified and “as is” for the high-cost textbook, t (38) = 0.54, p = .30. Likewise, cost only had a statistically significant effect when the textbook was modified, t (39) = -1.95, p =.03, but not when the textbook was adopted “as is,” t (35) = 1.27, p = .11.
Finally, when examining participants’ reported likelihood of taking a course with the hypothetical professor, there was no interaction between cost and modification for either a required or elective course (ps > .45). Factoring in the field of study did not alter any of these findings (interactions: ps > .20).
Discussion
Although the current study did not fully replicate the results of Vojtech and Grissett (2017), it did offer some limited confirmation of those previous findings. Specifically, participants saw the professor who used low-cost OER as more concerned about making course materials accessible to students. They were also more willing to take an elective course with this professor, as compared to the professor who used a high-cost traditional textbook. The discrepancy between the findings of Vojtech and Grissett (2017) and the current findings may speak more to methodology differences. First, the current study had a larger sample size drawing from students beyond the psychology major. Additionally, the current study did not create a forced comparison between the professor who opted for OER and the professor who opted for the traditional textbook; any positivity felt towards professors who opt for OER might be more relative in nature, in comparison to their colleagues who opt for higher cost traditional textbooks. Third, the current study permitted for comparison between another decision about textbooks: whether the professor modified the textbook or used it as is.
It was indeed that choice—whether the professor opted to modify the book or use it as is—that seemed to garner more positive evaluations from students. Specifically, modifying the textbook appeared to signal to students that the professor was enthusiastic about teaching, cared about their students, and was committed to student learning. This decision also seemed to suggest to students that the professor cared about creating an inclusive environment where diverse voices were represented. Within the description of the professor who opted to modify the textbook, it was noted that customizing the textbook can make the content feel more personally relevant to students—making the main effect of modifications on their judgment of whether “the professor wants their students to relate to the course material” less interesting. None of the other judgments were referenced in the description.
Finally, the current study considered that any positivity felt towards the professor based upon their textbook decision might be more complex than the main effects would reveal. For example, the professor who adopted the low-cost OER was seen as caring for their students, as long as they modified the textbook. It was not merely that students appreciated the professor’s extra effort to customize the book, given customizing a high-cost book was not seen as any more caring than leaving that expensive book as is. The field of study further complicated any effect of opting for OER, at least regarding whether participants felt the professor was committed to creating an equitable learning environment. For the history professor, opting for low-cost OER seemed to incur a less favorable rating if the professor kept the materials as is, with no modifications. On the other hand, for biology, no statistically significant effects were observed based on the choice to use low- or high-cost textbooks or to modify the book or not. This distinction between the two fields might be in part explained by the fact that students who take biology courses are used to spending more on textbooks, while those who take history classes have grown to expect the course materials in these classes to be more low-cost, at least relative to science courses.
Limitations and directions for future research
The current study relied on brief descriptions of hypothetical professors. As Nusbaum and Cuttler (2020) found, while students rated their real professors more positively when said professor used open-source resources, their findings did not generalize to a hypothetical professor/course. While the current study affords an element of control by creating an artificial description, it does create a disconnect between judgments of a hypothetical scenario and reality. Some participants even admitted that it was hard to make assumptions about a professor based only on a brief description. That said, students regularly make decisions about professors based upon reading a few brief evaluations on sites like ratemyprofessor.com.
Additionally, biology and history generally differ in terms of textbook costs, with the former being a costlier field of study. This confound could explain the observed three-way interaction in terms of whether the professor was seen as committed to creating an equitable classroom. Although the goal had been to provide a similar replication by using similar price points to Vojtech and Grissett (2017), these price points seemed higher than most textbooks that students would purchase in their first year at the present institution, especially if they were taking more humanities courses, such as history; the lower cost OER might not necessarily have been as attractive as the higher cost traditional textbook was unattractive.
Implications
In making choices about textbooks, faculty might be prone to focus on the content of the book as the primary criterion to be evaluated. Certainly having valid and credible content in a textbook is important. Beyond that decision, though, students seemingly make assumptions about their professor based on whether that textbook is affordable and whether the professor modifies the textbook. Although these assumptions likely will be only a small factor in terms of whether students take a professor’s class or not, they reasonably could affect the end-of-term evaluation surveys students complete, and when those evaluations are factored into decisions about maintaining and promoting faculty, they perhaps are not a matter to be entirely ignored.
Conclusions
With ongoing changes to digital technology, OER may become a choice more professors are considering. Students may seemingly give some credit to those professors, in that they perceive those professors as willing to make material accessible to students. However, that positivity may not be generalized to other evaluations, and in fact, modifying a textbook seems to bring broader positivity in terms of evaluations from students. For professors looking to impress their students, open-source resources tend to offer avenues for modifications to allow professors to tailor course resources to their course and their students.
