Explainability as a Constitutional Requirement: Can “Black Box AI” ever be Judicially Legitimate?

Artificial Intelligence is no longer a speculative presence in judicial systems. Artificial Intelligence is used for case management software to algorithmic risk assessments used for bail and sentencing decisions across jurisdictions. Courts are increasingly engaging and investing in automated tools that promise efficiency, consistency, and speed. Yet many of these systems still operate as "black boxes"—producing outcomes without intelligible explanations. The inclusion of Artificial Intelligence in judicial administration challenges the foundational constitutional question: Can judicial legitimacy survive when reasons are replaced by results? Through an analysis of precedents set by the Supreme Court on reasoned orders and non-arbitrariness, the article demonstrates that opacity in decision-making technologies often undermines equality before laws and erodes procedural dignity. Comparative insights from the European Union’s regulatory approach and the experimentation of the United States with algorithmic risk assessment tools further reveal a global judicial reluctance in application of unexplainable systems in adjudication. Through this article, we cross-examine whether the use of non-explainable Artificial Intelligence in judicial decision making is compatible with the principles set by the Indian Constitution. This article argues that explainability is not a mere desirable feature of algorithmic systems but a constitutional requirement in accordance with the provisions of Article 14 and Article 21 of the Indian Constitution. The duty to give reasons, the guarantee of a fair and just procedure, and the prohibition of arbitrariness collectively demand that any tool influencing judicial outcomes must be capable of meaningful explanation.