Across development, people seek to control their environments, often demonstrating a preference for contexts in which they have the opportunity to make choices. However, it is not clear how children, adolescents, and adults learn to calibrate this preference based on the costs and benefits of exerting control. Here, 92 participants between the ages of 10 and 25 completed a probabilistic reinforcement-learning task across contexts in which the instrumental value of control varied. On every trial, participants selected between two slot machines to try to gain the most reward. Critically, however, machine decisions were preceded by first-stage agency decisions in which participants decided whether to choose between the machines themselves or forgo agency and let a computer randomly select between them. On each trial, we manipulated the instrumental value of control by varying the reward probabilities of the machines, as well as an offer amount that participants would earn by forgoing agency. We found that across age, participants overvalued agency, but they also demonstrated a preference for control when it had greater instrumental value. Moreover, we found that sensitivity to the value of control increased with age, indicating developmental improvements in weighing the costs and benefits of agentic choice.