This mixed methods study evaluated the validity, and reliability of a scoring guide designed to assess a middle school student’s proficiency in systems thinking as described in the 2010 Oregon Environmental Literacy Plan. The commonalities between formal and non-formal educators revealed a high level of validity for the construct of proficiency with systems thinking, and a moderate level of reliability between the scores assigned by two groups of educators. In the words of the middle school students, formal, and non-formal educators, who volunteered to create the scoring guide, the ability to make responsible decisions with natural systems, community, and the future in mind involves: 1) creating solutions for systems that are not in balance; 2) presenting the complex inner workings of a system in a simple and succinct way; 3) collaborating; 4) exploring multiple solutions; and 5) sharing ideas in a way that people will understand you.