In a recent seminar given us, Justine Kingsbury of Waikato University suggested that evolutionary psychology - and particularly the positing of informationally encapsulated 'mental modules' - could shed some light on our seemingly odd behaviour here. The core of her explanation is that although we know the fiction isn't real, this information might not be accessible to the emotion 'modules' of the brain. This view suggests that emotions are rather like automatic reflexes (and so a-rational, rather than irrational). If you see a monster on the movie screen, this stimulus may activate your 'fear module', which does not have access to the information that the threat is not real. (A similar explanation may be given of irrational phobias, which must be overcome through training - mere cognition is insufficient.)
I'm not entirely convinced, however. Our emotional response to fiction seems deeper than that. We need to be engrossed in the fiction; mere perception of the superficial stimuli is insufficient to elicit most emotional responses (there are some exceptions, e.g. disgust). I won't feel sad at a character's loss unless I reflect on their situation, and empathise. This suggests that our responses are more cognitive and less automatic than the modularity account would seem to grant. (Unless I'm missing something here?)
I think a better explanation would focus on the role of fiction as a sort of simulation. In reading a novel, we create an internal simulation of the events it portrays. This provides us with surrogate experiences, as of engaging unfamiliar environments or situations, but without any real-life risk. I think it makes sense for us to react emotionally to the simulation, for it would otherwise be incomplete. How we feel about a situation is a crucial aspect of it; to leave this out of the simulation would be to omit something of great importance to us.
So I think it appropriate for us to react emotionally to fictions. Whether it is, strictly speaking, rational cannot be meaningfully answered until we clarify what is meant by that word in this context. Some suggestions from a TLS article:
An emotion is “cognitively rational” if it is based on a well-supported belief (I clearly saw that it was you and not Susan who knocked over my glass of red wine), and “strategically rational” if it leads to actions that will achieve a desirable goal (the urgency of my anxiety encourages me to rush across the room and immediately throw salt on the stain).
I guess what I described above suggests an indirect form of strategic rationality: emotions about fictions are useful in that they help us attain our goal (loosely understood) of learning more about life. Arguing that they are cognitively rational is more difficult, given that these emotions are based on imagined (or simulated) beliefs, rather than real ones.
A neat solution would be to say that the emotions too are merely simulated rather than real. Jonathan Ichikawa has previously argued for something along these lines (at least for some cases). There is some independent evidence to support this claim: Compared to real life, we seem to have an unusual degree of control over our emotional responses to fiction. Also, the resulting emotions are often more transitory, and less likely to influence our behaviour than other emotions. Counter-evidence would include the phenomenological similarities (it seems that we really do feel happy/sad, not that we're merely simulating it), and physiological ones too.