How the Report Card Was Created

The following section gives an overview of each of the nine measures on which we graded the states. It explains what data we used and how we calculated grades in every category. Each explanation is accompanied by a table comparing the performance of the states on that measure.

A technical explanation of the methodology can be found on the Methodology page.

Note: States earning a given letter grade are not listed alphabetically within the tables in every category. Where relevant, the states are ranked from highest to lowest depending on how well they performed on that measure.

1. Academic Achievement

To grade each state's overall achievement, we relied on the National Assessment of Educational Progress (NAEP). The federally sponsored NAEP, the only available metric for comparing performance across states, has four achievement levels: below basic, basic, proficient, and advanced. We compared the percentage of students scoring at or above the proficient level because this level indicates that the student has solid mastery of the knowledge and skills needed for work at grade level.

To grade each state, we first created a NAEP index by averaging the percentage of 4th and 8th grade students scoring at or above the proficient level on math and reading on NAEP in 2005. We then distributed grades based on a curve: The top 10 states received As, the next 10 states received Bs, and so forth.

2. Academic Achievement of Low-income and Minority Students

To produce a disadvantaged student achievement score for each state, we created several NAEP subgroup indices by averaging the percentage of 4th and 8th grade students scoring at or above the proficient level on math and reading on the 2005 NAEP for the African-American, Hispanic, and low-income subgroups². We then averaged these indices to create a ranking and, as with overall student achievement, graded the states on a curve. Every state reported sufficient data for its low-income students. States that reported enough data for either African-Americans or Hispanics to meet NAEP sampling requirements are included here; states that did not have adequate data for both subgroups did not receive a grade.

Under this methodology, we did not look at achievement gaps between subgroups. We believe that the most important question in judging the performance of minority and disadvantaged students in a state is what percentage are scoring at or above the proficient level, not how much distance there is between African-American, Hispanic, and low-income students and other subgroups.

3. Return On Investment

To determine the return that various states get for their education expenditures, we created a return on investment index by dividing state expenditures into student achievement, after first controlling for student poverty, the percentage of students with special needs, and cost of living. Specifically, we divided the percentage of students scoring at or above proficient level on the 4th and 8th grade NAEP reading and math tests in 2003 by 2004 state expenditures. The expenditures were adjusted for cost of living and students needs. We then graded the states on a curve.

If two states had the same expenditures and one state had better achievement than the other, the higher-achieving state received a higher index score.

4. Truth In Advertising About Student Proficiency

To grade the states in this area, we depended on a study by Paul E. Peterson and Frederick M. Hess titled Keeping an Eye on State Standards. The authors calculated a grade for each state based on the difference between the percentage of students deemed proficient by the state and the percentage identified as proficient on the NAEP in 2005. States that had large gaps did poorly; states that had small gaps received higher scores³. Minnesota, New Hampshire, and Vermont did not test their students in the 4th or 8th grades in 2005, so we gave them hash marks (-). We also removed the plus and minuses that had accompanied each state's grade in the original report.

5. Rigor of Standards

To grade the states in this category, we created a formula that takes into account the rigor of state academic standards, whether standards are aligned with college and workplace expectations, and whether the state has adopted rigorous high school exit exams. We then converted the numerical values produced by the formula into letter grades without using a curve.

State has high-quality math, English, and science standards
For almost a decade, the Thomas B. Fordham Foundation has evaluated the quality, rigor, and specificity of each state's academic standards. As part of the project, Fordham taps a panel of academic experts to review the written standards, which are graded against a set of strict criteria. Fordham also gives states credit if they take certain curriculum approaches like including evolution in their science standards.⁴ We relied on Fordham's 2006 evaluation of the quality and rigor of each state's science, math, and English standards and converted Fordham's grades into numerical scores.

State has aligned high school graduation requirements with college and workplace expectations
To understand what states have done in this regard, we used an indicator collected by Achieve's American Diploma Project, a partnership of several national organizations and 26 states that aims to raise academic standards. According to the group's 2006 state survey, only eight states have taken the necessary steps to align their academic standards with college and workplace expectations. States that had a policy in place received full credit, while states that did not have a policy received a failing score. A dozen other states are planning to implement these procedures; we did not give them credit because it is difficult to determine how close they are to actually implementing the policy.

Graduation is contingent on passing statewide exit or end-of-course exams at the 10th grade level
To receive a high school diploma, students should demonstrate that they have the necessary skills and knowledge to pass a rigorous exam based on the state's high school academic standards. Such graduation-linked exit or end-of-course exams show that high school graduates are job ready and ensure that students and schools are held accountable for their performance. To grade the states on this indicator, we used information collected by the newspaper Education Week in 2005. States that had an exit exam policy in place received full credit, while states that did not have a policy received a failing score.

6. Postsecondary and Workforce Readiness

To examine how well states are preparing graduates for college and the workplace, we examined the performance of the states in three areas that measure college readiness and also serve as an indirect proxy for workforce readiness: performance on Advanced Placement (AP) exams, high school graduation rates, and students' chances for college attendance by age 19. To grade the states, we averaged the indicators together and then distributed grades based on a curve.

AP quotient: students passing core AP tests divided by high school upperclassmen.
The AP program offers challenging college-level courses to high school students, measuring their success by using rigorous exams on which a score of 3 out of 5 is considered a passing grade. To examine what states are doing to ensure college readiness, we created an "AP quotient" by first reporting the number of students passing AP exams in core subject areas. Next we divided the number of public school 11th and 12th graders in 2005 who passed AP Biology, AP Calculus AB, AP English Language, and AP U.S. History by the total number of public school 11th and 12th graders in the state that year. This approach has the desirable effect of rewarding states that work harder to have significant numbers of students pass AP exams without penalizing states that push large numbers of students to take challenging AP courses.

Percentage of students graduating from high school
For this project, we declined to use notoriously unreliable official state graduation rate data. Instead, we included an estimated four-year cohort graduation rate measure created by Christopher Swanson, the research director of Education Week. He calculated this data in 2006. The estimate relies on grade-by-grade enrollment counts from the National Center for Education Statistics Common Core of Data to approximate how many 9th graders make it to graduation four years later.

Ninth graders' chances for college attendance by age 19
This information is compiled by Thomas Mortenson, a Senior Scholar at the Pell Institute for the Study of Opportunity in Higher Education, and serves as a measure of students' persistence from high school to college. To calculate the figure, Mortenson looks at the number of fall first-time freshmen enrolled anywhere in the United States in 2004 and then divides by the number of 9th graders four years earlier in each state. The data do not account for high school transfers out of state or students who drop out of high school and earn a GED (General Education Diploma).

7. 21st Century Teaching Force

We graded the states on their performance on the following four indicators using data from Education Week that were collected in 2005. If a state had all four policies, it received an A; if the state had three policies in place, it received a B; and so forth. States that had only pilot programs or future plans to implement such programs received no credit.

State requires teachers to pass basic tests
States should ensure the quality of their teaching pool by requiring a basic skills test of all incoming teachers. Numerous studies have shown that teachers with strong academic fundamentals help raise student achievement.⁵

State requires teachers to pass subject knowledge tests
There is also a growing consensus among education policymakers that teachers need strong content knowledge in the subjects they teach. To ensure that teachers enter the classroom with deep and relevant subject-matter expertise, states should test potential teachers and make sure they have the skills and knowledge necessary to teach students to reach high levels.

State has an alternative route program to recruit college graduates
Recruiting an effective teaching force, especially in areas like foreign languages, math, and science, is easier when states have sensibly designed programs for attracting nontraditional teachers. Alternative route programs can help attract second-career professionals and others with real-world experience who might not otherwise have pursued teaching.

State requires alternative route teachers to show subject matter expertise through test
Like conventionally prepared teachers, alternative route teachers should have to demonstrate their subject matter knowledge by passing a rigorous and appropriate assessment.

8. Flexibility in Management and Policy

To examine how well states manage their education system for schools, students, and parents, we examined three indicators and created a formula based on those indicators. We then converted the numerical values produced by the formula into letter grades without using a curve.

Strength of charter school law
Whatever the merits of any particular charter school model, the premise of charter schooling—heightened accountability for results coupled with enhanced flexibility—is clearly the future of American schooling. States that are pursuing this course aggressively—developing robust charter systems that include rigorous oversight and quality control—are equipping themselves for the new century. The data for this indicator come from a 2006 report from the Center for Education Reform (CER), an advocacy and research organization that evaluated each state's charter law based upon strict criteria including equitable funding, number of charters allowed, and operational autonomy. We converted the CER grades into numerical values; states that did not have a charter school law received a failing score.

Whether state has established a virtual school
Schools in which instruction takes place over the Internet, also known as virtual or cyber schools, provide students, parents, and schools with choice and flexibility. Virtual schools are highly adaptive and offer states a potentially cost-effective way to reach a wide range of students, including those in rural areas, with advanced or specialized course work that might not otherwise be accessible to them. The data for this indicator come from an Education Week report published in 2006. States that had a policy in place received full credit; states that did not have a policy received a failing score.

Percentage of principals who report a major degree of influence over how school budgets will be spent and on new teacher hiring
Like other executives, principals need the authority to allocate resources within their organizations. It makes no sense to hold managers responsible for performance if they lack the ability to assemble their own teams or control the dollars they are asked to steward. To examine this issue, we contracted with Richard Ingersoll, Professor of Education and Sociology at the University of Pennsylvania, to conduct an analysis of the 2004 Schools and Staffing Survey, a nationally representative survey of teachers administered every three years by the National Center for Education Statistics. Professor Ingersoll produced two indicators for our report, looking at the percentage of principals who reported a major degree of influence over school budgets and teacher hiring. This is the first time to our knowledge that these recent data have been published as part of a state-by-state report.

9. Data Quality

To examine state efforts to collect and report high-quality education data while providing student privacy, we graded the states using information that was collected in 2006 from the Data Quality Campaign. This campaign, managed by the National Center for Educational Accountability, is a national effort to encourage states to implement longitudinal data systems to improve student achievement. These commonsense metrics include the kind of benchmarks that any well-run organization—public or private—needs to monitor its effectiveness. The criteria include whether a state uses a unique statewide student identifier, whether it can match student test scores from year to year, and whether it can match data on teachers with students' academic results, as well as other measures that are critical to understanding whether students, teachers, and schools are succeeding.

To calculate grades, we first created a Data Quality index, which gives states credit for each of the 10 data quality policies that they have in place. Then we distributed grades based on a broad curve. If states had the same score on the index, we gave them the same grade.