The Decisions No Dashboard Can Make

LEADERSHIP UNDER REAL CONDITIONS

ARTICLE ONE

Why Judgment Becomes More Important—not Less—in the Age of AI

LEADERSHIP QUESTION

What remains the leader’s responsibility when technology can produce an answer before the organization has decided what it values?

“Data can tell us where to look. It cannot tell us what we are willing to see.”

— Dr. Dwan Anthony Jordon

The meeting had all the markings of modern leadership: a dashboard projected on the wall, color-coded performance indicators, trend lines, subgroup comparisons, and enough data to make every person in the room feel informed. The numbers revealed a problem. They did not reveal what we should do about it.

One group of students was underperforming. Attendance was part of the story, but not all of it. Instructional consistency mattered, but so did family instability, teacher turnover, special-education needs, and the accumulated effect of adults lowering expectations without ever naming that they had done so. The dashboard could identify the gap. It could not decide whether we would respond by narrowing the curriculum, replacing people, redesigning support, confronting assumptions, or simply explaining the results away.

That decision required judgment.

Leadership has entered an era in which information is abundant, analysis is increasingly automated, and artificial intelligence can produce recommendations with astonishing speed. This is often described as a technological transformation. It is equally a moral and institutional test. The central question is not whether machines will become capable of more. They will. The question is whether leaders will become more disciplined about the responsibilities that cannot be delegated.

More Information Does Not Eliminate Judgment

Organizations have long treated data as protection against subjectivity. The impulse is understandable. Evidence can expose comfortable myths, interrupt favoritism, and reveal inequities that anecdotes conceal. In my own work, structured data reviews helped schools move from impressions to action. At Friendship, performance dashboards, observation cycles, and student-support systems created a shared language for identifying where implementation was breaking down. Used well, evidence concentrates attention and clarifies responsibility.

But data does not arrive without interpretation. Someone decides what to measure, which comparison matters, what threshold defines success, how quickly improvement should occur, and whose experience counts as evidence. A metric is not reality. It is a decision about which part of reality will be made visible.

AI does not remove these decisions. It accelerates them and can obscure them. An algorithm can rank candidates, predict risk, recommend interventions, draft policy, or summarize community feedback. Yet its output reflects its training data, design choices, institutional context, and the question it was asked. NIST’s AI Risk Management Framework treats AI as a sociotechnical system precisely because technical performance cannot be separated from the people, processes, purposes, and power structures around it. Governance, mapping, measurement, and management are human responsibilities before they are technical procedures.

The danger is not only that AI may be wrong. Human beings are wrong too. The deeper danger is that an automated answer can acquire an authority its evidence does not deserve. Speed can be mistaken for certainty. Precision can be mistaken for truth. A recommendation can become policy before anyone has asked who bears the risk if it fails.

The Difference Between Analysis and Judgment

Analysis identifies patterns among available facts. Judgment determines what those patterns mean in a particular human context and what responsibility requires next. Analysis asks, “What is happening?” Judgment adds, “What matters, to whom, under what conditions, and at what cost?”

That distinction becomes clearest when values collide. A school may need to raise academic performance while protecting student belonging. A company may need to become more efficient without treating experienced employees as disposable. A public agency may need to communicate transparently while preserving confidentiality. No model can dissolve these tensions because they are not computational defects. They are features of consequential leadership.

Judgment requires at least four capacities. The first is contextual intelligence: understanding history, relationships, constraints, and the lived conditions behind the data. The second is moral clarity: knowing which values cannot be sacrificed for convenience. The third is consequence awareness: anticipating who benefits, who absorbs risk, and what precedent the decision creates. The fourth is accountable courage: accepting ownership rather than hiding behind the recommendation of a system, consultant, board, or majority.

The strongest leaders use technology to improve each of these capacities. They do not use it to escape them.

What Evidence Cannot Decide

Evidence can show that a policy produces unequal outcomes. It cannot decide how much inequity an institution is willing to tolerate. It can show that a leader’s strategy is unpopular. It cannot determine whether resistance reflects poor leadership, threatened interests, inadequate communication, or the predictable discomfort of meaningful change. It can reveal that an employee is missing targets. It cannot fully explain whether the cause is capability, clarity, opportunity, bias, workload, or leadership failure.

This is why leadership by dashboard can become morally thin. When leaders manage only what is visible, people learn to improve appearances. They protect the metric, manipulate the denominator, delay bad news, or avoid serving those whose needs make performance harder to display. Goodhart’s law is often summarized as the idea that when a measure becomes a target, it can cease to be a good measure. The leadership implication is broader: every accountability system creates behavior, and leaders are responsible for the behavior it rewards.

At Sousa Middle School, the gains that attracted national attention were not created by staring harder at test scores. The data mattered, but improvement required coherent instruction, adult accountability, family engagement, student culture, and a belief that the school’s history did not have to become its destiny. Michelle Rhee’s decision to place me in that role gave me real responsibility. My responsibility was to translate that mandate into daily systems with teachers and families. The numbers later told part of the story. The work began with a judgment about human potential.

The Questions Leaders Must Keep

As AI becomes embedded in hiring, evaluation, communication, resource allocation, and learning, leaders need a set of questions that cannot be surrendered to the technology:

1. What purpose are we serving? Efficiency is not a purpose. Neither is innovation. Leaders must identify the human outcome the technology is meant to advance.

2. Whose reality is represented? Data can be extensive and still exclude the people most affected by a decision.

3. Who can challenge the output? A system without an appeal path converts technical confidence into institutional power.

4. What happens when the system is wrong? Leaders should examine reversibility, harm, and remedy before deployment—not after injury.

5. Who remains accountable? “The model recommended it” is not an ethical defense. Authority must remain attached to a person or governing body capable of explanation and correction.

These questions slow an organization down at the right moments. Responsible leadership is not resistance to speed. It is the ability to distinguish decisions that benefit from velocity from decisions that require deliberation.

When the Leader Is a Black Man

The age of algorithmic management creates particular risks for Black men in leadership because automated systems often inherit the same interpretive patterns that already shape organizations. Research has shown that Black men can be perceived as larger, more threatening, or more aggressive than similarly situated White men. Studies of leader legitimacy also demonstrate that race can alter how identical leadership behaviors are interpreted. Confidence may be read as arrogance. Directness may be labeled intimidation. Calm may be interpreted as distance. Passion may be converted into threat.

If those judgments appear in historical evaluations, disciplinary records, hiring decisions, or written feedback, an AI system trained on that record may reproduce the pattern while appearing neutral. Bias can be laundered through data.

This does not mean Black leaders should reject measurement or demand exemption from accountability. My own research with Black male educators reinforced the opposite: belonging without high expectations becomes patronizing, while accountability without support becomes extractive. The answer is not less evidence. It is better evidence, transparent criteria, multiple evaluators, meaningful appeal, and leaders willing to interrogate whether “fit,” “tone,” “style,” or “executive presence” are disguising unexamined assumptions.

Black men also face a second risk: being expected to supply the human judgment an organization has failed to develop, particularly in moments involving race, community conflict, discipline, or institutional legitimacy. They are invited into the room as symbols of credibility, then denied authority when their judgment challenges the institution’s comfort. Representation without decision rights is not inclusion. It is exposure.

A Discipline of Human Oversight

Human oversight is often discussed as a final checkpoint—someone reviews the machine’s answer before action. That is too narrow. Meaningful oversight begins before a tool is purchased. It includes defining the problem, examining whether automation is appropriate, selecting evidence, testing for disparate impact, establishing decision rights, training users, monitoring consequences, and creating a remedy when harm occurs.

The leader’s role is therefore not to stand outside technology and criticize it. It is to build the conditions under which technology serves human purpose. This requires technical literacy, but it also requires institutional literacy: an understanding of how incentives, fear, hierarchy, race, status, and informal power shape what any system will produce.

AI will make competent analysis cheaper. It may make sophisticated writing, forecasting, and pattern detection widely available. That will not diminish leadership. It will expose it. When answers become easy to generate, the quality of the questions, values, and accountability surrounding them becomes the differentiator.

The future will not belong to leaders who can out-compute a machine. It will belong to leaders who know what must never be reduced to computation.

Prediction Is Not Purpose

The modern organization is becoming increasingly skilled at prediction. It can estimate which employees may leave, which students may fall behind, which customers may default, which patients may return to a hospital, and which communities may require additional intervention. Prediction can direct scarce attention toward emerging need. It can also narrow the institution’s imagination.

A prediction describes a probability based on prior patterns. A purpose declares what the institution will attempt to make possible despite those patterns. Confusing the two is dangerous. If past attendance, income, discipline, health, or performance predicts a difficult outcome, a leader may use that information to mobilize support—or to reduce investment in the people deemed least likely to succeed. The same analysis can become an instrument of opportunity or abandonment.

That choice is not inside the model. It belongs to leadership.

Education makes the distinction vivid. A risk score might correctly identify a student as unlikely to graduate under existing conditions. But “under existing conditions” is the phrase leaders must not ignore. The score says nothing about what may happen if the schedule changes, the student gains a trusted adult, transportation stabilizes, instruction improves, or the school stops interpreting absence as indifference. Predictive accuracy can coexist with institutional failure if the organization becomes better at forecasting harm than preventing it.

Leaders must therefore ask whether a model is predicting people or predicting the consequences of the system surrounding them. That question changes the intervention. It also changes where accountability belongs.

The Rise of Algorithmic Authority

Technology acquires authority long before it acquires formal decision rights. Employees begin deferring to a recommendation because the system is described as objective, advanced, or data-driven. Managers hesitate to override it because they fear being unable to justify a human judgment if the outcome later fails. The tool becomes an invisible participant in governance.

This is the problem of automation bias: people can overvalue computer-generated recommendations, particularly when tasks are complex or time is limited. A nominal “human in the loop” may provide little protection if the human lacks time, expertise, information, or institutional permission to disagree. Clicking approval is not oversight.

The risk increases when organizations cut the professional capacity meant to review automated output. An institution may introduce AI to support human resources while reducing the number of experienced human-resources professionals. It may automate instructional planning while giving teachers less collaborative time. It may deploy decision support in health care while increasing caseloads. Under those conditions, the human becomes a ceremonial safeguard around a system whose speed has already reorganized the work.

Real oversight requires the ability to understand the recommendation, access relevant context, delay the decision, document disagreement, and reverse the outcome. It requires leaders to protect the person who says, “The system is technically functioning and institutionally wrong.”

Judgment Under Conditions of Uncertainty

Leaders rarely choose between a correct answer and an incorrect one. They choose among incomplete options with different forms of risk. Data may be missing, stakeholder accounts may conflict, time may be limited, and every available action may create loss.

Judgment in those conditions is not intuition detached from evidence. It is the disciplined integration of evidence, experience, values, context, and consequence. It also includes metacognition: knowing what the leader does not know and how that uncertainty should affect the decision.

Strong judgment asks three temporal questions. What happens immediately? What pattern will this decision reinforce? What capacity will the organization possess afterward? A staffing cut may balance the current budget while destroying future capability. A strict discipline response may restore order while weakening the relationships necessary for lasting safety. An AI tool may improve short-term throughput while preventing junior professionals from developing expertise.

Leaders should also distinguish uncertainty from ambiguity. Uncertainty means the probabilities are unclear. Ambiguity means people disagree about what the problem is or what success should mean. More data may reduce uncertainty. It cannot resolve a conflict of values. When the disagreement is about whether efficiency, equity, privacy, safety, or autonomy should take priority, leadership must make the tradeoff visible rather than hiding it inside analysis.

The Moral Weight of False Positives and False Negatives

Every classification system makes errors. A false positive identifies a risk that is not present. A false negative misses a risk that is present. Technical teams can calculate error rates; leaders must decide which errors are more tolerable in a particular context and who will experience them.

The answer varies. In a screening tool for a treatable health condition, accepting more false positives may be reasonable if follow-up is safe and accessible. In a disciplinary or fraud system, false positives can damage reputation, employment, freedom, or access to services. In child protection, both kinds of errors carry serious human consequences.

Aggregate accuracy can conceal unequal burden. A model that performs well overall may make more harmful errors for a smaller racial, linguistic, disability, or socioeconomic group. Leaders who review only the average effectively decide that some people’s errors do not count enough to change the system.

This is why responsible governance requires subgroup analysis, but even subgroup metrics are not sufficient. Leaders should study actual cases at the boundaries: the person incorrectly denied, flagged, removed, or overlooked. Quantitative fairness tells us whether patterns differ. Narrative review tells us what those differences mean in human life.

The institution should define remedies before deployment. If the system is wrong, can the person understand why, reach a responsible human, correct the data, receive timely reconsideration, and recover what was lost? An appeal process that takes longer than the opportunity lasts is not a remedy.

AI Will Change What Organizations Reward

Technology does not merely complete tasks; it changes what leaders notice and reward. Once AI makes written production easier, organizations may value volume over thought. Once analytics make employee behavior more visible, managers may reward what is measurable over what is meaningful. Once automated systems standardize decisions, professional discretion may begin to look like inconsistency.

This can create a profound shift in culture. People learn to optimize for the signals the system reads. Teachers may choose assignments that are easier to monitor. Employees may generate more communication to display activity. Leaders may favor short-cycle outcomes over institution building because the former appears quickly on dashboards.

The work least visible to technology often remains essential: mentoring, noticing, repairing trust, translating across communities, exercising restraint, and asking the question that prevents an efficient mistake. These forms of labor are frequently performed by people whose contribution organizations already undervalue.

Executives should conduct a reward audit whenever technology changes workflow. What becomes easier to count? What becomes harder to see? Which behavior will people rationally optimize? Which human capacity may atrophy because the system appears to supply it? The answers belong in strategy, evaluation, and workforce development.

From Human in the Loop to Human Accountable for the Loop

The language of “human in the loop” can imply that responsibility is satisfied by placing a person somewhere in the process. A stronger standard is a human accountable for the loop: someone with authority and obligation to understand the entire decision system.

That person must know the purpose of the tool, the data lineage, the populations affected, the performance limitations, the escalation route, and the conditions for suspension. Accountability should be named, not distributed so widely that no one possesses it.

Boards have a role as well. They should not ask only whether an AI policy exists. They should ask which decisions are automated or augmented, which use cases could create material harm, how incidents reach governance, what evidence demonstrates value, and whether management has preserved meaningful human capability. AI risk is not only cyber risk. It is operational, legal, ethical, reputational, and strategic.

Frontline workers and affected communities must participate in governance because leaders cannot anticipate every consequence from the executive level. Participation should occur before the problem is defined, not after the system has been purchased. A listening session at the end of deployment does not redistribute decision power.

The Leadership Advantage in an Age of Abundant Answers

As AI lowers the cost of producing competent content, leaders will be tempted to compete through greater output. The more valuable advantage will be coherence. Can the organization connect its technology decisions to a clear purpose, a trustworthy culture, and a workforce capable of judgment?

The World Economic Forum’s Future of Jobs Report 2025 reflects this dual reality. AI and big-data capabilities are growing rapidly in importance, but so are resilience, flexibility, leadership, social influence, creative thinking, curiosity, and lifelong learning. These are not consolation skills left over after technology does the serious work. They are the capacities that determine whether technological power becomes human progress.

The leader of the future will need to read systems, not merely reports. That leader must understand enough technology to question it, enough humanity to see who is missing, and enough courage to remain accountable when the output appears authoritative.

Machines can extend cognition. They cannot decide what an institution owes a person. They cannot determine which risk is morally acceptable, when efficiency has become dehumanization, or whether a technically successful decision has made the organization less worthy of trust.

Those are not gaps waiting for a better model. They are the work of leadership.

The Decision Record as Institutional Memory

Consequential judgments should leave behind more than an outcome. A decision record can capture the purpose, evidence considered, alternatives rejected, assumptions, affected groups, dissent, owner, review date, and conditions that would require reversal. This is not bureaucracy for every routine choice. It is a discipline for decisions whose consequences outlive the meeting.

The record improves accountability without pretending leaders possess certainty. When conditions change, the organization can distinguish a poor judgment from a reasonable judgment made with incomplete information. It can also detect recurring blind spots: the stakeholder never consulted, the risk repeatedly discounted, or the model whose recommendation receives unearned deference.

AI makes this practice more important. Leaders should document where a system influenced the decision, what human verification occurred, and who held authority to override it. Otherwise, institutional memory will contain the final answer but not the relationship between machine recommendation and human responsibility.

Good judgment becomes organizational capacity only when others can study it. The decision record turns leadership from a private performance into a body of learning.

Direction for Leaders

Before adopting an AI-supported decision, require a short leadership record: the purpose of the decision, the evidence considered, the groups affected, the plausible harms, the person accountable, and the conditions that would trigger reconsideration. Invite someone with enough independence to challenge the premise, not merely the output. Then return to the people who will live with the consequences and ask what the analysis could not see.

Dashboards should inform attention. AI should expand capacity. Neither should become a place for leaders to hide.

The most consequential decision in the room will often be the one the system cannot make: what kind of institution we are willing to become.

The Question That Remains

When the technology produces an answer that is efficient, popular, and profitable—but inconsistent with the institution’s stated values—who will have the judgment and courage to say no?

Selected References

Edmondson, A. C. (2019). The Fearless Organization. Wiley.

National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). U.S. Department of Commerce.

National Institute of Standards and Technology. (2024). Artificial Intelligence Risk Management Framework Generative Artificial Intelligence Profile. U.S. Department of Commerce.

Laux, J. (2024). Automation bias in the AI Act: On the legal implications of attempting to debias human oversight of AI. European Journal of Risk Regulation.

World Economic Forum. (2025). The Future of Jobs Report 2025.

Todd, A. R., Thiem, K. C., & Neel, R. (2016). Does seeing faces of young Black boys facilitate the identification of threatening stimuli? Psychological Science, 27(3), 384–393.

About the Author

Dr. Dwan Anthony Jordon is an education executive, scholar-practitioner, author, and organizational strategist whose career spans nearly three decades across public, charter, nonprofit, and specialized educational settings. He has served as a teacher, principal, head of schools, system leader, and executive, leading school transformation, leadership development, virtual learning, and organizational design. He earned his Doctor of Education in Education Policy and Leadership from American University. Through Building Opportunity, LLC, he integrates research, executive practice, writing, speaking, and advisory work to help leaders develop people, strengthen institutions, and expand opportunity.

About Leadership Under Real Conditions

Leadership Under Real Conditions is a 15-article series by Dr. Dwan Anthony Jordon examining power, judgment, change, identity, and the human consequences of leadership. Written for leaders across sectors, the series combines research, executive experience, and practical direction. Each article also considers the distinct implications of the issue for Black men in leadership—not to narrow the conversation, but to make visible dynamics that conventional leadership writing too often leaves unnamed.