Conversational agents struggle to answer questions during complex tasks such as do-it-yourself (DIY) projects and cooking due to difficulties in understanding task context and user information needs. This study examines the efficacy of integrating conversational and task context in query and document representations to enhance question answering (QA) performance in cooking tasks. We evaluated three document representations with increasing granularity on two task-based QA datasets with a total sample size of 6217 question–answer pairs: full recipe documents (document-based), segmented recipes by cooking steps (step-based), and detailed task structures (task-based). The results show step- and task-based representations outperform traditional document-based approaches by 10% on average (). Task-based representations provide superior performance for fact-based needs (e.g., ingredients, time, equipment) in most cases, while step-based representations better address competence needs (e.g., preparation, cooking techniques). Simple conversational history prepending of two to three turns yielded the best performance, improving results by up to 24% over no context. These results emphasise the importance of selecting a representation that matches the structure of the surrounding task in order to enhance QA performance.