Empathy is an important component of human relationships, yet the neural mechanisms that facilitate empathy are unclear. The broad construct of empathy incorporates both cognitive and affective components. Cognitive empathy includes mentalizing skills such as perspective-taking. Affective empathy consists of the affect produced in response to someone else’s emotional state, a process which is facilitated by simulation or "mirroring." Prior evidence shows that mentalizing tasks engage a neural network which includes the temporoparietal junction, superior temporal sulcus, and medial prefrontal cortex. On the other hand, simulation tasks engage the fronto-parietal mirror neuron system (MNS) which includes the inferior frontal gyrus (IFG) and the somotosensory related cortex (SRC). Here, we tested whether neural activity in these two neural networks was related to self-reports of cognitive and affective empathy in daily life. Participants viewed social scenes in which the shift of direction of attention of a character did or did not change the character’s mental and emotional state. As expected, the task robustly activated both mentalizing and MNS networks. We found that when detecting the character’s change in mental and emotional state, neural activity in both networks is strongly related to cognitive empathy. Specifically, neural activity in the IFG, SRC, and STS were related to cognitive empathy. Activity in the precentral gyrus was related to affective empathy. The findings suggest that both simulation and mentalizing networks contribute to multiple components of empathy.