Campus Surveys

I think one of the primary reasons that more schools do not conduct sustainability literacy surveys is because developing one that produces meaningful results is incredibly difficult.  If there were a group of people that would be willing to work to develop a sustainability literacy assessment tool that could be picked up and used by any school, I think we would see a dramatic increase in the number and value of sustainability literacy assessments.  (The model I'm imagining here is something like the Clean Air-Cool Planet GHG Inventory Calculator which was developed by students in partnership with a nonprofit organization and then released and refined over time as an open source tool.)

When I was a student and later when I was the Sustainability Coordinator at Duke University, we regularly talked about wanting to conduct a sustainability literacy assessment survey so that we could see if our programs were having the desired effect, as far as public education was concerned.  Despite our desire to do so, it never happened because we discovered almost immediately upon setting out to do it that the true difficulty in conducting such a survey is not finding the person power to conduct it or the money for participation incentives.  It is the knowledge power needed to craft questions that will produce meaningful results, design a distribution methodology that will ensure a good sample, and an analysis scheme free of common pitfalls. (The same thing was true before the CA-CP GHG Inventory tool. Getting students to collect the data was easy once you had a guidebook of how to do it, you had a place to put the data once you got it and you had analytical tools to help you interpret it.)  

I can tell you I would have *jumped for joy* if, while I was at Duke, somebody would have handed me a sustainability literacy survey, methodology and analytical tools devised with the input of a number of expert/wise people from a number of campuses!  And after I finished jumping for joy I would have conducted the survey according to the methdology, run the results through the analytical tools and published the findings so that all could learn from them.  Unfortunately, what I remember instead is that I searched and searched online for such a thing and came up empty-handed.  Maybe things are a little better 4 years later.

I think that the development of this tool needs to be an open source process similar to the iterative cycle of drafting, publishing, workshopping and revising that the CA-CP GHG Inventory tool and the Sustainability Tracking Assessment and Rating System (STARS) both went through.  I believe it needs to be developed this way because creating a survey tool that will produce meaningful results when deployed on a number of campuses will require the input of people from diverse perspectives.

I can imagine some people reading this and thinking "a one-size-fits-all solution is not the answer; the survey needs to be tailored to the location." I agree that some portion of the tool should be customizable to a particular setting (just as portions of the CA-CP GHG Inventory Tool is).  However, leaving each school with the task of creating a sustainability literacy assessment tool from scratch has a number of big liabilities:

1. Many schools will never benefit from the insights provided by a sustainability literacy assessment because they won't be able to get over the hurdle of developing a survey and methodology themselves.
2. Enormous hours of campus sustainability enthusiasts' time and energy will be wasted reinventing the wheel, campus by campus.
3. Let's face it, creating, conducting and analyzing a survey that produces results that are meaningful is very difficult and most people aren't well trained in the nuances and pitfalls of survey research.  (Worst yet, they aren't aware of the short-comings of their methodology so they publish their results as though they were meaningful.)
4. The results of sustainability literacy assessment surveys will continue to be completely incomparable and unaggregatable.

To re-cap, dear reader, I've been thinking about this and wishing for something like the tool I'm describing here for a very long time.  You can be the one that makes my dream come true!  Please be my s/hero!

Here at North Carolina State University we are in the process of conducting a student sustainability attitude survey, which will include elements of measuring student literacy. Our survey team includes a Ph.D. Marketing/Advertising Prof who specializes in consumer research, two Lean Six Sigma grad students who have training in voice-of-the-customer analytics, plus campus reps from sustainability, transportation, solid waste & recycling, and energy management. Like you, Mike, we started with a review of AASHE questionnaires, then began customizing our own survey tool. We have conducted 2 pre-tests to date, have administation buy-in, and hope to push the refined questionnaire out to student population later this month. The results will be used to develop a 2010 sustainability communication strategy (e.g., awareness --> understanding --> action). Once the communication strategy is defined, both media and creative strategies will be developed and implemented. The real benefit will be repeating the questionnaire next year to see if our new campaign moved the needle toward more sustainable awareness and behavior.  We have no idea if this effort will elevate the team to hero status, but we are making sure we apply survey research best practices that will help ensure we yield valid and reliable results.  We will share the results with AASHE, and others, in due course.

Paul McConocha, North Carolina State University