The website RateMyProfessors has allowed students to rate their university professors on a variety of different criterion for nearly twenty years (Glasner, 2018). These criteria include the professor’s overall quality, difficulty of the course, usefulness of the course, attitude of the professor, teaching style, and course workload amongst other qualities (“Ratemyprofessors.com”, n.d.). Previously, RateMyProfessors had an option for students to rate their professor’s attractiveness by adding a chili pepper to their review, however, due to accusations of sexism from academic faculty, the chili pepper was dropped from the site (Dalbey, 2018). Previous research also supported the criticism expressed by faculty members (Theyson, 2015).
While the chili pepper was removed from the website, the other criteria still remain (“Ratemyprofessors.com”, n.d.). These criteria may be looked at by students when determining which school (service business) or professor (service provider) to learn from. Students that wish to attend an academic institution look into their ranking and quality of education when deciding which institutions to apply to (Alter & Reback, 2014). Using quality of education has its merits, especially when teaching style has been shown to have a positive influence on learning (Biggs, 2014).
Consumer interactions with companies have recently increasingly changed from a one directional to a two-directional interaction between sender and receiver (McQuail & Windahl, 2015). With this in mind, it may follow that consumers of educational services might further consider other students’ ratings on RateMyProfessors when trying to decide between two universities.
Online ratings have been shown to have a demonstrable influence on the decision-making customers make when choosing service businesses and service providers in the service industry (Trehan & Daluiski, 2016). Customers perceptions of quality are influenced by online ratings of service businesses (Gao, Greenwood, Agarwal, & McCullough, 2015). Additionally, people tend to put more trust into service providers when their quality of service is rated higher compared to their competitors (Gavilan, Avella, & Martinez-Navarro, 2018). These studies demonstrate the value of online ratings for businesses and service providers.
Due to the importance of online ratings, it is important to understand the factors that go into a student’s review of their professors. Understanding these factors may help to provide some insights into which criterion a professor should focus on to better reflect their teaching skills. Specifically, the relationship between workload and rating would provide valuable information on the influence workload has on perceived difficulty. Furthermore, the effect personality has on perceived quality of instruction is another important aspect to study further. In addition, it should be determined whether or not difficulty and teaching style are correlated.
Just as important as determining how these factors influence student’s perceptions of their professors, it is also important to determine how all of these factors influence the professor’s score. Teaching style, difficulty, and attitude may all influence a student’s score differently. For this reason, it is important to determine which factor influences a student’s score the most. Finally, the interrelationship between all of these factors will provide insights that educators and educational institutions can use to market their school as well as shape their curriculum. This content analysis will seek to determine the criteria students use to review their professors on RateMyProfessor.
Method
Sample
The content analysis sample was composed of 290 comments in November 2018. The sample covers comments from University of California Los Angeles students regarding their professors on ratemyprofessors.com. The reviews coded ranged from August 17, 2001, to November 11, 2018.
Procedure
The reviews were rated by graduate students at the University of Southern California. Raters were trained in the operationalization of variables and in the coding procedures. A total of eight variables were coded. The first of these variables, Professor, was defined as the name of the professor. The second variable coded for was Score, which was measured on a five-point scale. The third variable Usefulness refers to whether or not students found the professor’s course to be useful or not. The fourth variable, Take Again, was measured on a yes or no scale. The fifth variable difficulty was defined by comments left by students on the website. The sixth variable Attitude was analyzed by examining comments provided by the students. Examples of this included but were not limited to “caring, genuine, nice, arrogant, brash, rude”. The seventh variable Teaching Style also analyzed comments given by the students. Examples of this included but were not limited to “entertaining professor, engaging class, hard, and boring”. Finally, the eighth variable, Workload, was defined by analyzing student’s comments. The full codebook can be seen in the Appendix.
Results
A series of five correlations were conducted. The rating score of professors was found to have a significant positive correlation with usefulness of the course (r = .77, p < .001), professor’s attitude towards students (r = .95, p < .001), and professor’s teaching style (r = .85, p < .001), but a significant negative correlation with difficulty of the course (r = -. 48, p = < .001), and a negative correlation with workload of class (r = -.13, p = .505).
A series of two correlations were conducted. The level of difficulty was found to have a significant positive correlation with workload (r = .72, p < .001), but a significant negative correlation with usefulness (r = -.42, p < .001).
A series of two correlations were conducted. The level of difficulty was found to have a significant negative correlation with attitude (r = -.56, p < .001), and teaching style (r = -.38, p < .001).
Analysis revealed that the teaching style was the most mentioned factor while evaluating the professor (89.0%). The least mentioned factor was workload (10.0%).
A series of independent-sample t tests were conducted to evaluate how the choice of “take again” affected scores and evaluation of other aspects. (see Table 1). Participants who chose “no” to “take again” scored significantly higher than participants who chose “yes” on difficulty (p = .001) but non-significantly higher on workload (p = .838).
A series of one-sample t tests were conducted to determine which factors were considered important by comparing responses to the midpoint neutral value of three. Six out of the six factors were considered important (score, usefulness, difficulty, attitude, tech style, workload, all p= .001)
Discussion
When considering which factors were most important to a professor’s score it was found that all six factors were considered to be important. However, in terms of frequency, teaching style was mentioned the most and workload the least when evaluating the professor.
The content analysis indicated that professors who received higher scores and were considered to teach more useful classes by their students also tended to teach less difficult classes, had better attitudes towards their students, a better teaching style, and taught less difficult and lower workload classes.
Professors that were considered more difficult were found to have higher workloads, and lower scores when considering teaching style, attitude towards students, and usefulness.
Difficulty of classes also seemed to be related to the amount of workload given by the professor, and teaching style. Some possible explanations for these results could be that students may undergo more stress as a result of more homework, these higher levels of stress may in turn cause students to think of their professors negatively, which in turn may impact the score of their professors.
This content analysis was limited by not determining if similar results could be found by comparing different subject areas with the same characteristics. For example, students may rate all professors of traditionally difficult courses such as math, lower regardless of the professor, which may indicate professor’s scores may be tied into the subject area just as much as teaching style and personality. Future content analysis research in this area could be improved upon by considering subject area as an additional criterion.
Analysis was also limited by only focusing on one university. University of California Los Angeles is consistently ranked amongst some of the best universities in the world. The elite ranking of their university may result in higher quality professors teaching at their university, which may have a positive effect on students and their average rating for professors. The quality of professors may be tied in more to students’ perceptions of their university and perceived teaching quality as a result of these rankings. Findings of this content analysis were limited by not comparing other universities to determine if professor rating was also influenced by the university. Future studies could compare schools at various levels of ranking and their average professor score to determine if school quality influences ratings of professors. Analysis can also look at professors who teach at multiple institutions to determine if their score fluctuates based on the university.
Given the popularity of RateMyProfessors, and the increasing necessity of social media presence, universities could potentially benefit by posting scores and profiles of their professors on their social media pages. Students may look at rankings when considering which university to apply to but may use professor quality as a criterion when deciding between which school to attend. Finally, universities can use these insights to determine not only which professors, but also which departments are rated lower in order to determine strategies to improve student satisfaction. Increases in student satisfaction may lead to improvements in perceived quality of professors, courses, and the university.
Easiness is most important