One of my favorite subjects, and one I've had the good fortune to discuss with some very brilliant people in my lifetime. I don't have answers, but I can tell that a particularly brilliant friend pointed out the fallacy in my thinking that free will and destiny were mutually exclusive. He explained that they can co-exist. I'm still not sure I entirely get that now, but back when he talked me through it, it made sense.
One of my favorite things to contemplate is called Determinism.
I came up with a concept much like this in my own mind before I realized that such a thing had been discussed and given a name a long, long time ago. I'm not saying I subscribe to it, but it's fun to contemplate.
I think the bottom line is that a background in science, where a study of what is truly a random, unpredictable event at a sub-atomic level, is essential to really "getting" this stuff.
But it's fun for hacks like me to ponder, regardless.