The Role of Collaborative Review in Achieving STARS Platinum
Colorado State University (CSU) has become the first institution to earn a Platinum rating in the Sustainability Tracking, Assessment & Rating System (STARS). This achievement highlights the continuous improvement that comes as a result of participating in STARS. The Association for the Advancement of Sustainability in Higher Education (AASHE) would like to congratulate CSU and everyone who contributed to the report!
AASHE strives for optimal data quality in STARS, and staff follow a standard review process for every report submitted. The review process for any Platinum-level report is different from what is in place for other reports, however. AASHE staff began reviewing CSU’s report in December 2014, and a lot has taken place behind the scenes before the report was submitted. In the interest of transparency, this blog outlines the steps required to review, revise and publish a Platinum report, and explores how the Platinum review process may look in the future.
AASHE Staff Review
Regardless of whether a report has reached Platinum or not, the primary purpose of staff review is to ensure that content provided by institutions is consistent with the credit criteria provided in the STARS Technical Manual and meets the intent of the credit. The staff review identifies the following:
- Best practice credits - Highlight areas that provide clear and complete information, exceed minimum credit criteria, and/or supply superior documentation to support claims.
- Inconsistencies that require revision - Identify content that does not adequately address the credit criteria within required reporting fields or uploads. Also, point out any numeric data outliers or data inconsistencies between credits and reports that could indicate misinterpretation of credit criteria or data entry errors.
- Suggestions for improvement - Highlight optional reporting fields that appear to include inconsistent or incomplete information, areas where improved documentation would clarify content or better support the institution’s claims, and marginal data outliers that the institution may wish to check for accuracy.
STARS is a voluntary, self-reporting framework, and the staff review process is meant to enhance data quality rather than certify a STARS rating. This review process is not a verification or validation of a STARS report. Staff do not engage in auditing reports, requesting additional documentation, conducting interviews or making onsite visits. While a more detailed validation/verification process has been discussed, we’ve found that this level of review would significantly increase the time and expense required to earn a rating, thereby impacting the accessibility of STARS. (Learn more about the ongoing discussion related to STARS data quality here.)
The standard review process has evolved over time and was enhanced with the release of STARS 2.0. For reports submitted under Gold, Silver or Bronze ratings and Reporter designations, AASHE staff review one-third of all STARS content by checking a standard set of 22 credits plus three random credits. This review takes place in the days following report submission. Staff document any best practices, recommendations or inconsistencies, and share these results with the STARS liaison, along with requests to revise or clarify potential inconsistencies.
The main distinction for Platinum reports is that credits are reviewed prior to publication, as outlined in the STARS Data Accuracy Policy: “All institutions wishing to pursue a Platinum rating will be subject to pre-publication review by AASHE staff.” If an institution has achieved a provisional Platinum score upon completing an internal data accuracy check and finalizing all credit content, that institution should contact the STARS team to indicate that it is ready to pursue Platinum. AASHE staff can then begin reviewing credits before the institution’s report has been submitted. As with standard reports, AASHE staff document best practices, inconsistencies and recommendations and share the results with the STARS liaison for follow-up. Any inconsistencies that require revision must be fully addressed before a report can be submitted under Platinum.
Piloting Platinum Review
In December 2014, CSU’s STARS liaison, Tonie Miyamoto, contacted AASHE to inform us that the institution was ready to pursue a STARS Platinum rating. This was the first time that AASHE received a Platinum review request, so the process of reviewing CSU’s report became a pilot initiative that would inform the STARS Platinum review protocol moving forward. We realized early on that this would be a learning process for everyone involved, and could also be a great opportunity for the campus sustainability community as a whole. In addition, we recognized that the first STARS Platinum report would set a very high standard for quality once public. As a result, a strict review protocol was established that involved checking every field within each credit for adherence to criteria and intent. All potential inconsistencies identified required follow-up, no matter how minor. Once the initial review was completed, AASHE connected with Tonie and her colleague Carol Dollard, co-chairs of CSU’s President’s Sustainability Committee, as well as others at CSU to talk through the review results.
Upon completing the initial review of CSU’s report, AASHE staff realized that the rest of the review process would take longer than the 30-day timeframe that was originally established within our data accuracy policy. After the initial follow-up period, at least one additional round of staff review would be necessary to assess changes, and additional rounds of review and follow-up might be needed. In the case of CSU, two rounds of review and follow-up were allowed, and in the third and final staff review, all identified inconsistencies were addressed. The CSU team not only addressed each inconsistency, but also worked to improve the report even further by adopting nearly all identified suggestions for improvement. With an overall score of 85.3 after the review process concluded, we informed Tonie that CSU would be AASHE’s first STARS Platinum-rated institution (learn more about CSU's story here).
In our communication with Tonie, Carol and the rest of the CSU team, we learned how valuable the collaborative review process was in providing clarity to STARS that resulted in a higher quality report. We agreed that this learning experience could be beneficial for others as well, so we agreed to be transparent and publicly share the CSU Platinum review results. A detailed report of review results, including links to the draft version of the report that underwent initial review, can be accessed here.
One of the greatest takeaways from the CSU Platinum review pilot is that a higher than expected amount of time and resources were needed to conduct our first Platinum review. The thoroughness required to assess a provisional Platinum report, along with allowing additional rounds of review, resulted in approximately 40 hours of staff time dedicated to the endeavor. For comparative purposes, a standard review of one-third of all credits takes approximately three hours. While we hope that the time and resources dedicated to Platinum review will shrink as we make the process more efficient over time, we must be prepared for how this will impact our resources - especially when Platinum requests come in more frequently.
Lessons Learned and Looking Forward
Now that we’ve announced the first Platinum STARS report, we’re mindful that more institutions are likely to show interest in achieving the same goal. While this would signify great progress for the campus sustainability community, it could have a significant impact on AASHE in terms of staff capacity and the ability to review reports in a timely manner. As a result, the Platinum review process will continue to evolve with future submissions and may be approached differently with the next institution.
In the meantime, AASHE will continue to review each STARS report, and will look to enhance the standard review process for future reports. We’ve learned that reviewing reports is a win for everyone: STARS institutions benefit by gaining a better understanding of credit criteria, resulting in reports of higher quality; the STARS program benefits from the intel gained from review results, leading to improvements in future versions of STARS; and the entire higher education community benefits from more accurate, reliable campus sustainability data.
AASHE staff are exploring along with the STARS Steering Committee various approaches to enhance the report review process, including reviewing portions of all reports prior to publication. We’re also discussing ways to fund the review process, particularly to cover the added costs of conducting Platinum reviews. We hope to publish a follow-up blog about this soon and will encourage feedback from the campus sustainability community.
Announcing the first STARS Platinum report is a significant opportunity for advancing the sustainability movement within the entire campus community. AASHE is looking forward to working with CSU to publicly share their story, best practices and lessons learned in the coming months. We hope that this achievement will inspire other institutions to grow their own sustainability efforts and document their progress through STARS.
We encourage anyone with thoughts or questions to enter a comment below. We look forward to continuing the conversation about Platinum review soon, as well as working with many more institutions in reaching this important milestone and rating!
Browse by Topic
- AASHE Biz
- Co-Curricular Education
- Community Engagement
- Connecting the Dots
- Dining Services
- Diversity, Access, and Affordability
- Faculty and Staff Development
- Government & Legislation
- Human Resources
- Presidents & Chancellors