Predictive inference is an important cognitive function and there are many tasks which measure it, and the error driven learning that underpins it. Context is a key contribution to this learning, with different contexts requiring different learning strategies. A factor not often considered however, is the conditions and time-frame over which a model of that context is developed. This study required participants to learn under two changing, unsignalled contexts with opposing optimal responses to large errors - change-points and oddballs. The changes in context occurred under two task structures: 1) a fixed task structure, with consecutive, short blocks of each context, and 2) a random task structure, with the context randomly selected for each new block. Through this design we examined the conditions under which learning contexts can be differentiated from each other, and the time-frame over which that learning occurs. We found that participants responded in accordance with the optimal strategy for each contexts, and did so within a short period of time, over very few meaningful errors. We further found that the responses became more optimal throughout the experiment, but only for periods of context consistency (the fixed task structure), and if the first experienced context involved meaningful errors. These results show that people will continue to refine their model of the environment across multiple trials and blocks, leading to more context-appropriate responding - but only in certain conditions. This highlights the importance of considering the task structure, and the time-frames of model development those patterns may encourage. This has implications for interpreting differences in learning across different contexts