Explainability

No jargon, no math

A large model is a black box: billions of tuned weights turn your input into an output, with no dials to read. Explainability is the toolkit for shining a light inside, or around, that box. There are two families. Interpretable-by-design systems are simple enough to read directly: a scorecard, a decision tree, a rules engine; you see the whole path. Post-hoc explanations work on black-box models: feature attribution (SHAP-style scores) ranks which inputs pushed the answer up or down; counterfactuals tell you what small change would have flipped it (“two fewer late deliveries and this supplier would have passed”). Neither is a confession straight from the model: post-hoc scores are faithful approximations, useful for auditing and debugging, not proof of the true reason. That is the boundary: an explanation can be plausible and still be a reconstruction. High-stakes use needs explanation plus audit plus human review, especially where the EU AI Act demands it.