Sustainability Literacy Assessment

Sustainability Literacy and Culture Assessments are surveys being implemented at more and more universities. Cal Poly first administered a survey of this type to students, faculty, and staff in 2018. Now, in 2023, key faculty and staff have worked hard to update the survey so that it might be relaunched at a bigger and better capacity. These assessments serve several purposes.  

First, they test students’ knowledge on sustainability and climate topics with regional, state, national, and global implications. For example, questions might pertain to the United Nations’ Sustainable Development Goals (SDGs) as well as local San Luis Obispo and Cal Poly emission sources. Second, the survey presents an opportunity for students to self-report where they receive their information on sustainability from. This helps faculty and staff assess whether or not curriculum and research offerings from universities like Cal Poly play an important or effective role in educating students on these important subjects. Next, students are also able to share how much they value sustainability as a whole, how often they implement sustainable principles and practices in their own life, and where their values lie as a whole. This can help faculty and staff see if there are correlations between valuing sustainability, seeking out more educational opportunities, demonstrating higher rates of literacy, etc. Finally, students can give feedback on how they would like to see sustainability improved on campus, from infrastructure and planning to curriculum and events.   

The initial survey in 2018 was led by Dr. Yiwen Chiu from the Department of Natural Resources and Environmental Sciences (NRES) and Kylee Singh, the campus’ Sustainability Coordinator, among others. A paper was produced to summarize their data and takeaways; quick statistics can also be viewed here. In 2023, the initial survey process was refined and led by Dr. Chiu, Singh, and Hadley Willman and Erin Pearse, Assistant Director and Director of the Initiative for Climate Leadership and Resilience, respectively. For Cal Poly’s assessment, the above aspects are split into two sections in the survey: the literacy section and the culture section, the latter entailing all value and belief questions, since they pertain to the general campus perspective and culture of sustainability.  

The main changes made to update the survey in 2023 were as follows:  

• The literacy section had 10 questions in 2018 and 13 questions in 2023. More questions were added to ensure that this section effectively tested students’ knowledge through more lenses. For example, 2 new questions were added that pertained to Cal Poly’s sustainability infrastructure and performance, whereas none existed on the topic previously. Additional questions ensured a global perspective on climate change was assessed as well.  
• The culture section had 10 questions in 2018 and 14 questions in 2023. Similarly, more questions were added to provide the surveyors with a broader look at students’ beliefs, values, and opinions, and allow them to analyze nuanced differences in perspective in similar questions. For example, students were now asked to rank their agreement with the following statements on a scale of 1-5: 
  • “Global warming is happening.”  
  • “Global warming is mostly caused by human activities.”  
  • “I am worried about global warming.”  
  • “Global warming will harm people in the US.”  
• In 2018, the survey received responses by conducting individual interviews with students from selected classes in order to record answers to questions. In 2023, surveyors went to over 30 classes to introduce the purpose of the survey and make time to take the survey during class time, which students took on their phones using OneDrive’s survey software. This time around, students were also able to take the survey outside of selected classes, with it being advertised in email campaigns, on social media, via posters, and at Sustainability Month events. It was important to find classes with willing professors from all six colleges on campus because the main marker for a representative sample was determined to be college representation, e.g., receiving a significant number of responses from students in OCOB, CENG, CLA, and COSAM, as opposed to just the already sustainability-minded CAFES and CAED.  

Faculty and staff are beginning to conduct an in-depth analysis of the data received to produce another formal paper. However, initial results and conclusions have been released to the public.  

Demographic and Overall Data 
There were a total of 724 responses to the survey, representing a significant portion of Cal Poly’s student population. 

Figure 1. College breakdown of 2023 assessment participants.

 

Figure 2. Gender identity breakdown of 2023 assessment participants.

 

Figure 3. Degree level breakdown of 2023 assessment participants.

Literacy Section 
The first 13 questions of the assessment were literacy questions, and students were graded on their multiple-choice answers. The average score was 7.91 out of 13, or 60.8%. For 9 of the 13 questions, the average correct response rate was 60.40% to 91.57%, meaning these questions were those that students did better on by a significant margin. The 4 questions with the worst averages, in comparison, got a correct response rate average of 38.54% to 17.82%. The 4 questions associated with lower literacy were the two with a Cal Poly focus, and the two related to greenhouse gas emissions. To the surveyors, this demonstrates a need for the campus to increase educational efforts on these two subjects. 
 
In the values section, students were able to indicate how much they cared about sustainability on a scale of 1-5, 5 being “very important” and 1 being “not at all important”. Those who gave sustainability a 5 out of 5 importance got approximately 62.77% of literacy questions correct. Those who gave sustainability a 4 out of 5 importance got approximately 55.92% of questions correct, and those few students who rated it a 1-3 out of 5 collectively got 46.7% of literacy questions right on average. This data demonstrates that caring very strongly about sustainability correlates with a slightly increased knowledge of sustainability. 

 
Figure 4. Average literacy score vs. personal rating of importance of sustainability for 2023 assessment participants. 

Students were also able to indicate the number of classes they had taken that taught sustainability topics (participants were able to indicate having taken 0, 1-2, or 3+ courses of this kind). The score students received for the literacy section correlated positively with the number of sustainability courses they had taken. Those who took 3+ courses got approximately 69% of literacy questions correct, which is about 8% better than average. Those who took 1-2 courses got approximately 59% of literacy questions correct, and those who took 0 sustainability courses at the time of the survey got approximately 55% of literacy questions correct, both subset averages being lower than the overall average. This data demonstrates that taking several courses on sustainability correlates with a statistically significant increase in sustainability literacy.

 
Figure 5. Average literacy score vs. number of sustainability courses taken for 2023 assessment participants. 

Comparing these two datasets, it can be seen that students performed better than average if they took 3 or more sustainability courses, or rated sustainability a 5 out of 5 importance. However, taking college courses on sustainability correlated with a higher sustainability literacy, which indicates the importance of making sustainability courses available to students. It also implies that caring about sustainability is not necessarily enough to inform one’s increased knowledge of the issues, though it certainly is an important factor. 

More data will be released in the coming year. Please reach out to Hadley Willman (hwillman@calpoly.edu) with any questions. 

See the full list of questions below:
 
Literacy Section (multiple choice)

1. Define sustainability.
2. Which of the following best characterizes sustainability?
3. What aspects of the human population and its activities impact the Earth? 
4. Which of the following statements about water is true?
5. Of the following, which would be considered living in the most environmentally sustainable way?
6. Imagine that we had to pay for all the costs associated with the goods we use every day. What would go into calculating the true costs of a product?
7. Define economic sustainability:
8. Which of the following is an example of environmental justice?
9. Which of the following countries has produced the most carbon dioxide emissions over time?
10. The term circular economy refers to which of the following?
11. Which of the following statements about greenhouse gasses is true?
12. Which sector produces the most greenhouse gasses associated with the Cal Poly campus?
13. What is the source of drinking water at Cal Poly?

Culture Section (response type varies)

14. How important do you think sustainability is? (Scale 0 to 5)
15. How important do you think it is to include sustainability learning in the Cal Poly classroom? (Scale 0 to 5)
16. How well does Cal Poly teach sustainability? (Scale 0 to 5)
17. What are some ways to infuse sustainability education in the classroom at Cal Poly? (Choose up to 3)
18. What prevents you from receiving more sustainability instruction at Cal Poly? (Choose up to 3)
19. During your time at Cal Poly, how many courses have you taken that address the topics presented in this survey?
20. Which of the following has educated you the most about sustainability?
21. Rank your agreement with the following statement on a scale of 1-5:
    1. Global warming is happening.
    2. Global warming is caused mostly by human activities.
    3. I am worried about global warming.
22. Mark your familiarity for each of the following sustainability resources at Cal Poly: (Options: I have never heard of this before; I have heard of this, but never used or interacted with the resource; I have used or interacted with this resource)
23. Rank your current level of participation in the following individual sustainable practices. (Options: Yes, I know how; No, but I would like to learn; No, and I am not interested)
24. Rank your current level of participation in the following individual sustainable practices. (Scale of 1 to 5, 1 being never, 5 being always)
25. Prior to this survey, were you aware of Cal Poly’s sustainability website?
26. Prior to this survey, were you aware of Cal Poly’s sustainability newsletter?
27. OPTIONAL: In what way(s) might Cal Poly better engage students in discussion and making decisions about what sustainable practices might be promoted on campus?