Artificial Intelligence (AI) is increasingly deployed in high-stakes domains such as healthcare, where decisions carry significant consequences for individuals and society. This amplifies the need for safe and trustworthy AI systems. While traditional explainable AI (XAI) methods aim to increase transparency, they often omit the uncertainties that are inherent in XAI-augmented decision-making, where AI supports human decision-makers. To better understand the sources and effects of uncertainty
in XAI-augmented decision-making and structure the existing body of knowledge, we conduct a structured literature review focused on four main sources of uncertainty: data uncertainty, model uncertainty, XAI method uncertainty, and human uncertainty. Our review shows that these sources are largely examined in isolation, resulting in a fragmented understanding of how uncertainties interact and influence decision-making. Based on these findings, we outline how multiple sources of uncertainty
can be incorporated into uncertainty-aware explanations and propose an integrated research agenda for developing uncertainty-aware XAI.