Commentary on SEI’s STARS Investment Analysis
This month, AASHE celebrates an important milestone in its history of inspiring and catalyzing higher education to lead the global sustainability transformation. For five years, the Sustainability Tracking, Assessment & Rating System (STARS) has enabled colleges and universities to measure their sustainability performance through a transparent, self-reporting framework. With over 650 institutions across the globe using STARS, the collective impact has been significant. We are encouraged with each new institution that joins the STARS community, and are inspired with every story we hear about how STARS has made a positive impact in advancing sustainability.
With collaboration and openness as cornerstones of STARS from the beginning, we’re also pleased to know that others are reviewing STARS data and sharing their analyses, insights, and questions in published works. Through a transparent and public platform, STARS data allows anyone to share sustainability trends, best practices, and areas for improvement. This promotes use of STARS data and helps to hold institutions accountable for the information they are publicly sharing. This is certainly the case with the most recent example, an analysis of STARS Investment data from the Sustainable Endowments Institute (SEI).
Investment Best Practices
SEI’s report identifies trends and best practices among STARS institutions in sustainable investment. (Note: While AASHE and SEI have collaborated on projects in the past, AASHE did not partner on this report and wasn’t involved in the research or analysis.)
The report highlights 12 STARS-rated institutions for sustainable investment practices that stand out from the crowd. Kudos to the following institutions for the great work they are doing in the area of sustainable investment! For more best practices, stay tuned for the STARS Annual Review that will be released later this month.
- Bard College, STARS Gold
- Columbia University, STARS Gold
- Green Mountain College, 2011 report
- Earlham College, STARS Bronze
- Okanagan College, STARS Silver
- The New School, 2011 report
- Unity College, 2011 report
- University of Iowa, STARS Gold
- University of Louisville, STARS Silver
- University of Michigan, STARS Silver
- University of Wisconsin System, 6 Rated institutions
- Wesleyan University, STARS Silver
Data Inconsistencies in the Investment Subcategory
Another key focus of SEI’s report is on data inconsistencies identified during analysis of STARS Investment data. An argument is made that these inconsistencies may hinder comparisons of sustainable investing trends, and the report calls for stricter verification of submissions.
It is important to note that the inconsistency claims in the report reflect SEI’s own interpretation of the credit criteria and are not always consistent with the intent or criteria of the STARS credits (particularly former Tier Two credits), In addition, the report largely focuses on STARS version 1 data rather than the most current version (STARS 2.0). Some of the issues identified in this report have been addressed with changes in the latest version. (Though a section on STARS 2.0 credits is included in the SEI report, there is limited information on methodology for identifying or defining the inconsistencies.)
While we may disagree with some of SEI’s specific interpretations of credit intent and criteria, AASHE appreciates and supports the report’s general stance on data accuracy and verification.
Since the inception of STARS, data verification has been a topic of discussion in numerous venues, and AASHE has also surveyed the campus sustainability community about the issue. While it is clear that some form of verification above and beyond current STARS data accuracy practices would be ideal, the best way to implement verification has been less than clear. Among the ideas that have been discussed are third party verification, verification by a peer institution, and enhanced AASHE staff audits. Each of these approaches has its costs in terms of time and funding, and there is concern that these costs could limit the accessibility of STARS and effectively price some institutions out of participation altogether. So the essential challenge of verification is to find the funding to minimize those impacts.
Current Data Accuracy Processes
The good news is that mechanisms for STARS data accuracy have always been in place and we continue to make progress in this area. Since the launch of STARS, data accuracy is ensured in three ways:
- Each credit is assigned a responsible party who provides a statement that the information submitted is accurate.
- Each submitted report must be accompanied by a letter from the institution’s president, chancellor, or highest ranking executive confirming that the report was checked for accuracy.
- The public nature of STARS data empowers anyone to draw attention to potential inconsistencies. Completing a Data Inquiry Form will bring any potential error to the attention of the STARS Liaison at that institution (see notes section on best practices for submitting a data inquiry).
Following the launch of STARS Version 1, AASHE staff also began conducting post-publication review for portions of all STARS reports after a rating has been earned, as well as periodic audits of data submitted by all institutions.
With the launch of STARS 2.0, additional measures were implemented. For every STARS Version 2 report submitted, AASHE staff now review at least 23 credits that have been identified as being more susceptible to data inaccuracies than other credits (including at least one of the revised 2.0 Investment credits). In addition, all institutions wishing to pursue a Platinum rating are subject to pre-publication review by AASHE staff, which entails a review of every STARS credit (to date, no institution has submitted a report under a Platinum rating). Our data accuracy policy details all of these mechanisms and others.
"AASHE values entities that hold sustainability to the highest standard. We will continue to work to improve the rigor of STARS and maintain transparency as the foundation of our efforts. We were excited to launch STARS 2.0 that included many recommendations from the community and changes geared toward improving data quality. We are looking forward to releasing our STARS 2014 Annual Review to share highlights of STARS submissions from the past year." - AASHE Executive Director, Stephanie A. Herrera.
One of the great outcomes of sharing STARS data and having it published is that this type of exposure can greatly promote data accuracy. Publications such as the latest SEI report help to hold institutions accountable for the information they are publicly sharing. These works prompt institutions to take another look at the data that has been submitted in existing reports, and can improve data collection practices for future reports. The credit analysis sections in SEI’s report and the section on Recommendations for Reporting Institutions can serve as a useful resource for institutions that wish to take a closer look at their investment data. Inconsistencies can be updated quickly and easily by submitting a STARS data revision request.
Looking Ahead - Data Accuracy & Verification
Members of the STARS team feel strongly that one of the best ways to promote data accuracy and high-quality reporting is to highlight best practices that meet and often exceed credit criteria. We are in the process of developing a comprehensive list of best practices in addressing credit criteria for numerous 2.0 credits in every STARS category, and look forward to sharing this resource publicly on our website in the coming weeks. We also look forward to reviewing the recommendations provided in the SEI report and other sources of feedback as we work toward developing STARS version 2.1 and all future versions of STARS.
In line with our core value of collaboration, we continuously look toward the campus sustainability community for feedback and assistance on identifying a process and funding mechanism for enhanced verification. We welcome feedback at any time (email@example.com) and also look forward to gathering thoughts and suggestions from STARS participants and AASHE members at the STARS sessions during next month’s AASHE Conference in Portland. Below are just some of the topics on which we hope to gather input:
- Opportunities to engage students, perhaps through internships, to help with data verification.
- Working with organizations that have a specific area of interest and would be willing to review certain areas of STARS.
- Business partnerships to fund enhanced verification.
- Partnerships and discussions with potential third party reviewers.
- Opportunities to involve STARS institutions, AASHE council members, STARS technical advisors, and other stakeholders.
With over 650 institutions across the globe using STARS, the collective sustainability impact of the system has been significant. The public nature of STARS means that others are reviewing the information and publishing their findings. These published works help hold institutions accountable for the information they are sharing within STARS, further improving data quality. SEI’s report raises valid points for improving the STARS framework and credits, though progress made in these areas over the last five years should not be overlooked. AASHE will continue to investigate enhanced data verification options in ways that will not limit accessibility through added financial burden. We look forward to some lively discussion on the topic at the AASHE Conference in Portland!
While SEI submitted numerous data inquiries upon conducting its analysis last spring, the nature of the inconsistencies was not fully explained. As a result, STARS Liaisons were not always equipped with all of the information to address the concerns raised. To help ensure that identified inconsistencies can be addressed promptly in the future, here are some recommended best practices for those looking to submit data inquiries:
- Provide a clear, concise, and complete explanation of the potential inconsistency.
- Keep in mind that STARS reports should be based on information available at the time the report was submitted.
- Connect with AASHE first if you wish to submit numerous inquiries (around 5 per month in total or more), so we can identify the most effective and efficient way in addressing the questions.
- Keep in mind that the content of the inquiry will be shared with AASHE staff and STARS Liaisons (and may be forwarded at discretion).
- Identify yourself and communicate openly. While inquirers have the option to remain anonymous, sharing contact information will enhance communication.
- Inquiries that are not valid in relation to credit criteria, are unprofessional, or request updates beyond the timeframe for which the report was submitted, will be dismissed.
Browse by Topic
- AASHE Biz
- Co-Curricular Education
- Community Engagement
- Connecting the Dots
- Dining Services
- Diversity, Access, and Affordability
- Faculty and Staff Development
- Government & Legislation
- Human Resources
- Presidents & Chancellors