Part 1: Pain is Painful
I want to begin by saying that it isn't possible for me to give you an answer that will suddenly make whatever suffering you are going through bearable. Words and understanding won't change the fact that pain is painful. So check yourself; you may be appearing to ask for an explanation, but maybe you are actually demanding a solution. If so, you will be disappointed by any explanation given, and you were asking the wrong question.
If you are reading this because you want to find an answer for someone who has asked you about God allowing suffering, pay attention to whether they are asking because of their head or their heart. Are they asking because they are searching for truth, and are interested discussing philosophy? Or, (more likely,) are they asking because some suffering in their past or present is making them emotionally angry at your belief in a perfect, loving God. One requires facts and references, the other needs emotional help, and the biggest mistake you can make is to get them the wrong way around.
Part 2: The Big Picture, or People and Nature
The cause of a lot of suffering in this world is free will. God wanted to create beings that could freely choose to love Him, so He gave humans free will. Unfortunately that same ability allows others to make choices that can cause you pain, either on purpose or by accident. That risk has to be a part of having free will, because if God intervened in all of our bad choices then it wouldn't be free will at all.
But some suffering, such as earthquakes and floods, is not caused by human free will. In these cases, the cause is that the current world is "fallen", broken compared to what God originally intended. This again was technically caused by (Adam's) human free will, but such a long time ago that it doesn't really compare.
So in the grand scheme of things, why would God allow suffering to exist in the world? Because his highest priority is not our comfort. A great summary of the book of Job I heard recently was "We live in an amazing, beautiful, complicated world that is designed to show us how awesome God is, but it is not designed to prevent suffering."
Part 3: But what about Me and my Darkness?
Everything so far has been about the general, all-over-the-world scale of suffering that happens. I have given some answers to that. But if you want to ask me why your particular problem is happening, then I'm afraid I don't know. Even if you tell me all the details, and I think of all the possibilities, I still can't ever really know why an individual thing has happened. God knows, and my daily faith includes deciding to trust His plans. But it's not all just plans, God has something important He does in these situations...
I heard a story once of a man who wanted to show God's love to his community by helping them in basic, practical ways. A poor woman said to him "Give me money, I don't have any to pay my electricity bills," but being quite poor himself at the time, he said "I'm sorry, but I can't give you any money." He saw that this response angered her and she was about to respond, but he felt God's spirit prompt him to quickly add "...but I will come and sit in the dark with you after they've turned your lights off." It wasn't the answer she was expecting, but it also achieved something she hadn't hoped for.
God isn't detached from our pain. He experienced pain Himself when Jesus came down to earth in human flesh, and today He is with us through the Holy Spirit. In this period of time when the world is still broken and people are selfish, God won't wave a magic wand to make you comfortable, but He will be with you, He wants to be beside you, even in the darkness.
