Change Risk Score Calculation
The Change Risk Dashboard assigns a risk score to every planned change, predicted by AI Incident Prevention.
Risk scores are between 0 and 100, where 0 is the safest and 100 is the riskiest. That single number is not a single measurement. AI Incident Prevention evaluates each change across five independent risk categories, scores each one on the same 0-100 scale, and then blends those five scores into the final risk score you see on the dashboard. The blended score maps to a five-level risk rating from Very Low to Critical.
The five categories draw on three different kinds of evidence, so that no single signal can dominate an assessment:
AI-assessed: Implementation Risk, Topological Impact Risk, and Historical Incident Risk are determined by AI through analysis of the change record and its history.
Statistically computed: Team and Individual Risk is derived from a statistical model of past change outcomes, with no AI judgment.
Configurable evaluation: Organization-Specific Risk is scored against the written rules and criteria you define.
A high score in any one category does not automatically make a change high risk overall. What matters is the blended score, which reflects the weight you place on each category.
The five risk categories
Each category answers a distinct question about the change, and all five are expressed on the same 0-100 scale, allowing them to be compared and combined.
Category | How it is assessed | What raises the score |
|---|---|---|
Implementation Risk | AI-assessed | Work that spans multiple teams or interconnected systems, unclear or incomplete steps, and a weak or missing test or backout plan. |
Topological Impact Risk | AI-assessed | A broad or business-critical blast radius, and affected configuration items (CIs) with a history of change-caused incidents. |
Historical Incident Risk | AI-assessed | Similar past changes that caused severe or recent incidents. |
Team and Individual Risk | Statistically computed | A sparse or troubled delivery record for the assigned team and individual, measured against the size of that record. |
Organization-Specific Risk | Configurable evaluation | The change matching risk conditions in your own written rules, such as maintenance windows, freeze periods, or change types. |
Documented plans are credited, not penalized
When implementation, test, or backout details live in an attached document rather than inline text, AI Incident Prevention analyzes the attachment and takes it into account. A well-documented change is not penalized simply for lacking inline text. Supported file types are listed on the Change Risk Dashboard page.
From category scores to a blended score
The final risk score is a weighted sum. Each category's 0-100 score is multiplied by its weight, and the results are added together. Because the weights sum to 100%, the blended score remains on the same 0-100 scale as the individual categories.
When no custom weighting is set, all five categories are weighted equally at 20% each. You can tune these weights in the Change Risk Configuration to reflect what matters most in your environment. If Organization-Specific Risk is not configured, it does not participate in the assessment, and the remaining categories carry the full weight of the blend.
Weight tuning
Set weights that reflect your organization's approach to risk. For example, an organization with a mature change-management culture might lean more heavily on Historical Incident Risk, while another might emphasize its own policy checks through Organization-Specific Risk.
Risk score and risk rating
The blended 0-100 score is translated into a five-level risk rating. The rating is what most stakeholders act on, while the underlying score is preserved for finer-grained sorting, trend analysis, and threshold-based automation.
Risk score | Risk rating | Rating number |
|---|---|---|
80-100 | Critical | 1 |
60-79 | High | 2 |
40-59 | Medium | 3 |
20-39 | Low | 4 |
0-19 | Very Low | 5 |
The rating number runs from 1 (Critical, highest risk) to 5 (Very Low, lowest risk), while the risk score runs the opposite direction, where a higher score means higher risk. Both describe the same assessment, so you can sort by the score or filter by the rating.
How the categories are scored
Each category arrives at its number differently. The following notes explain the principles behind each one.
Implementation Risk: AI reads the full change record, including the description, the change tasks, the schedule, and any attached implementation, test, and backout documentation. Clear, well-sequenced steps with a credible rollback plan and a narrow scope keep the score low.
Topological Impact Risk: The criticality of the affected systems is weighed together with two structured signals about the specific CIs involved: the per-CI incident history (how often each component has been involved in incidents and how often changes to it caused incidents) and a focused look at past changes that touched the exact same components. Anchoring the score on real incident history means two identically worded changes can carry very different risks depending on where they land.
Historical Incident Risk: AI finds genuinely similar past changes using meaning-based matching rather than keyword overlap, grades each on how closely it resembles the current change, and weighs the severity and recency of any incidents those changes caused. The assessment uses only retrieved historical records; where there is no relevant history, the category reflects that absence rather than inferring risk.
Team and Individual Risk: This category is purely statistical. It estimates how reliably the assigned team and individual have delivered past changes, using historical change outcomes and, where available, the frequency with which their changes directly caused incidents. The model treats small samples cautiously, so a clean record over a handful of changes cannot earn a near-perfect score, and it rewards proven, high-volume track records. The team score and the individual score are then combined into the single number this category contributes.
Organization-Specific Risk: You provide risk-evaluation instructions in plain language, such as adding points when a change occurs during a maintenance window or treating emergency changes as high risk. AI applies those exact rules to a comprehensive picture of the change and returns a score, a summary of how it was determined, the specific factors it identified, any ambiguities where the data was unclear, and a confidence level. The evaluator is instructed not to invent data. If the context needed to apply a rule is missing, it reports an ambiguity rather than guessing.
How scores stay current
A change record is rarely static, so AI Incident Prevention re-evaluates risk as a change moves toward implementation.
Materiality gate: Not every edit triggers a fresh assessment. When a change is updated, its new version is compared against the last assessed version. Cosmetic or bookkeeping edits are ignored to avoid needless churn, while meaningful edits to scope, schedule, affected systems, tasks, outages, conflicts, or attachments trigger a re-assessment. When it is unclear, the change is re-rated to stay safe.
Risk history: Each assessment is recorded, building a timeline of how a change's risk has moved from one evaluation to the next. This shows whether a change is trending safer or riskier as it is refined. You can view this on the Risk Prediction tab under Risk History.
Explanations of change: When a change is re-assessed and a prior assessment exists, a short plain-language explanation describes why the scores moved, grounded in the actual differences between the two versions.
On-demand re-rating: You can force a fresh assessment at any time. On the Risk Prediction tab, click Force Re-Rate.
Signals that shape every assessment
Several cross-cutting mechanisms keep assessments consistent, tunable, and explainable across all five categories.
Incident priority filter: You can define which incident severities count when AI Incident Prevention measures change-caused incidents. This filter is applied consistently everywhere incidents matter, so the whole assessment reflects the same definition of a significant incident.
Attachment analysis: Attached runbooks, test evidence, rollback procedures, and diagrams are analyzed and distilled into structured summaries that highlight risk indicators, affected systems, and whether rollback, test, and scheduling information is present. These summaries feed the Implementation Risk assessment.
Graded similarity: Wherever the assessment reasons from similar history, similarity is graded rather than treated as an exact match. More similar and more recent history carries more influence than distant or loosely related events.
Evidence-bound scoring: The AI-assessed categories use only the retrieved change and incident records and do not invent past changes or incidents. Where evidence is absent, the score reflects that absence.
Assessment rationale
Along with the scores, each assessment produces a written rationale describing the specific factors that drove the risk up or down, and a set of targeted mitigation suggestions aimed at the weaknesses it found. The scores tell you how much risk there is, and the rationale and mitigations tell you why and what to do about it. You can review both on the Risk Prediction tab.
Scores support your judgment
Change risk scores are estimates designed to inform, not replace, your change-management decisions. Specific weightings, close-code mappings, incident-priority filters, and organization-specific rules are configurable and may differ for your environment.