The SPACE rubric for academic research assessment reform
SPACE to evolve academic assessment: A rubric for analyzing institutional conditions and progress indicators
Download a pdf of the rubric (English)
Descarga un pdf de la rúbrica (Spanish)
Descarrega un pdf de la rúbrica (Catalan—in development)
Improving research and scholarship assessment practices requires the ability to analyze the outcomes of efforts and interventions. However, when conducted only at the unit level of individual interventions, these evaluations and reflections miss opportunities to understand how institutional conditions themselves set the table for the success of new efforts, or how developing institutional capabilities might improve the effectiveness and impact of these new practices at greater scale. The SPACE rubric was informed by an iterative development process involving nearly 75 individuals in 26 countries and 6 continents, and was designed with the intention of helping institutions at any stage of academic assessment reform gauge their institutional ability to support interventions and set them up for success.
Organizations can use the SPACE rubric to support the implementation of fair and responsible academic career assessment practices in two ways: First, it can help establish a baseline for the current state of infrastructural conditions, to gauge an institution’s ability to support the development and implementation of new academic assessment practices and activities. Second, the rubric can be used to retroactively analyze how strengths or gaps in these institutional conditions may have impacted the outcomes of concrete interventions targeted to specific types of academic assessment activities—such as hiring, promotion, tenure, or even graduate student evaluation—either helping or hindering progress toward those goals.
Recommendations for use:
Identify who to involve. The SPACE rubric can be used by an individual or as an exercise for a group of people, such as a research assessment task force or a departmental hiring committee, etc.
Decide on the “level” of analysis. The rubric can be applied to different organizational levels, such as a department, school (i.e. Medical School or Law School), or an entire academic institution (university). Choose the largest unit level that you feel has direct and substantial influence on the kinds of assessment practices you are interested in learning about.
Decide how you would like to use the rubric: a) to establish a baseline for institutional “maturity” or b) to retroactively analyze how institutional conditions may have impacted the outcomes of a specific intervention, such as hiring.
To establish a baseline for institutional “maturity”:
Work through each of the five SPACE categories (Standards for scholarship; Process mechanics and policies; Accountability; Culture within institutions; Evaluation and iterative feedback) to capture the current state of organizational capability for the unit level you just identified. To help users collect and organize information, we provide a blank rubric that can be used as a worksheet (Download worksheet Word document). Consider things like:
Types of activity: What types of institutional activities to support the development of new research assessment practices are occurring, from building foundational capabilities, to expanding on them, to scaling them?
Coverage: Where is there a lot of activity, and where are there organizational “blind spots”?
Availability and use of resources: What resources (people, money, time) are used to support these institutional capabilities, and where are those resources being underutilized or underinvested?
Tensions: Where do you see conflicts between stated organizational goals and what capabilities are actually in place to support those goals?
To retroactively analyze how institutional conditions may have impacted the outcomes of a specific assessment intervention:
First, identify the specific academic assessment intervention or activity you are interested in exploring further. Then, work through each of the five SPACE categories (Standards for scholarship; Process mechanics and policies; Accountability; Culture within institutions; Evaluation and iterative feedback) to capture how institutional conditions supported or hindered the development, implementation, and success of the specific intervention you are exploring. To help users collect and organize information, we provide a blank rubric that can be used as a work sheet (Download worksheet Word document). Consider things like:
Strengths: What kind of capabilities helped this intervention succeed, or provided the right conditions for it to flourish?
Gaps: Where did a lack of organizational capability contribute to challenges in getting buy-in, support, or execution?
Investment: Where do you think additional investment or attention (of people, resources, time, or money) may have made a significant difference? This could either be developing new capacities where they are currently absent or bolstering ones that already exist by making them more robust or better integrated.
Identify next steps to improve academic career assessment based on the information captured above.
Important considerations:
Using the rubric is not an exercise in completeness, and you should not feel obligated to feel out every square! Blank spaces may indicate where future work is needed.
The “This might look like” content within the rubric merely as representative examples of what one might see, not as definitive criteria for how to define success. On the second page of the rubric, for example, we indicate some illustrative starting points and potential areas of focus for institutions to consider, depending on their current level of research assessment reform.