Sense of agency (SoA) is a feeling of controlling one’s own action. Recognition memory can improve for stimuli that involve SoA perhaps because of the self-reference effect. Recognition memory consists of recollection (i.e., detailed memory of stimuli) and familiarity (i.e., a feeling that stimuli are in memory). The self-reference effect is often observed in the recollection. Here, we investigated whether SoA particularly improves the recollection process. Participants pressed a key to produce an outcome (i.e., box movement followed by word presentation in Experiment 1 or word presentation in Experiment 2) and rated their SoA over the outcome. The outcome was spatially congruent or incongruent with the action. The participants learned the words intentionally (Experiment 1) or incidentally (Experiment 2). Performances of recollection and familiarity were assessed using the remember/know procedure. Our results suggest that the participants’ SoA was successfully manipulated. However, contrary to our hypothesis and previous findings, we found no effects of voluntary action and action-outcome congruence on recollection and familiarity processes of recognition memory, regardless of the latency of word presentation and learning strategies. Further studies are needed to replicate and elucidate the relationship between the SoA and recognition memory.