We see that the mystery of time's arrow ("why does time only go forward?") appears to be related to the illusion of free will. Namely, heightened awareness of the future appears to eliminate free will.
In Christian theology, the prototypes for humankind, Adam and Eve, eat the fruit of the tree of knowledge. Lore says that it gave them the ability to tell good from evil, and that this caused the Fall, in which they became separated from the grace of God.
I have found it interesting to think of the Fall as if it were a transition from sempiternity to "ordinary" existence. A gift [1] is given. Knowledge of the future is revoked, and in exchange free will is granted. The ability to tell good from evil implies a choice between good and evil. In this perspective, the reason why time appears to have a direction is a product of this gift.
[1]The use of the term 'gift' here is not a great match. It seems to suggest that there's some discrete entity giving the gift, which is not the case here.
But why does it go forwards, and not backwards? I think this derives from entropy. As the physical universe advances through time, it goes from a more-ordered to a less-ordered state. The number of possible configurations increases as time moves forward. In a context of choice, this means that the number of choices increases as time goes forward: an expanding tree of possibilities. In the context of time going backwards, the opposite happens; the universe moves towards a more-ordered state, and the number of configurations decreases. Moving forward in time allows free will in the selection between options; moving backwards in time does not, since results diminish.
This does not mean that traveling backwards in time is not possible, as a conscious experience. What it does mean is that traveling backwards in time is an experience like sempiternity: it lacks free will. In this light, it appears that free will is not just a product of the veil of randomness, not just a product of uncertainty over the future, but a product too of which direction through time that a conscious entity is traveling. Time appears to go forward because of the gift of choice.