When Sir Arthur Conan Doyle, author of the Sherlock Holmes stories, wanted to play a practical joke on twelve of his friends, he sent a note to each of them that simply read, "Flee at once. All is discovered." Within twenty-four hours, all twelve friends had left the country. That's what you call a guilty conscience.
If you ask me, I think we could use a little more guilt in our society. Guilt does serve a purpose. What good can possibly come from guilt? The same good that comes from that warning system in our bodies called pain. If you step on a piece of glass, your body sends a warning signal: "Stop! Don't go any further!" In the same way, God has installed a warning system called guilt into our souls, and we experience it when we do something wrong. Just as pain tells us there is a physical problem that must be dealt with or the body will suffer, guilt tells us something is wrong spiritually and needs to be confronted and cleansed.
So you see, guilt isn't necessarily a bad thing. The guilt feeling we experience is the symptom of the real problem, which is sin. All of the psychological counseling in the world can't relieve a person of his or her guilt. We can pretend it is not there or try to find someone else to blame for our problems. But the only real and effective way to remove our guilt is to get to the root of the problem, which is sin.