The Kaaba was a house of worship originally built by the Prophey Abraham and his son in Mecca. After many generations passed, the Arabs fell into ignorance and began idol worship, placing their idols witin the Kaaba. Many tribes from far and wide came to the Kaaba for worship, and so Mecca began to thrive as a trade city for many societies. When Prophet Muhammad established the religion of Islam, he broke all the idols in the Kaaba. At this time, Muslims were praying towards Jerusalem, but then revelation came from God, and Muslims were told to pray towards the Kaaba. This shows that the designation of direction of prayer doesn't mean that we are worshipping that place or the Black Stone--God is everywhere! We just need one unifying factor, to show discipline.
The Black stone is said to have been used by Prophet Abraham to build the original Kaaba, which is why when it was rebuilt, the Balck stone was given such significance and a place of honor. But we do not pray towars the Black Stone!
"We decreed your former qiblah (direction of prayer) only in order that We might know the Prophet's true adherents and those who were to disown him. It was indeed a hard test but not for those whom Allah guided." (Surah al-Baqarah 2:143 )
This verse shows another reason for praying towards the Kaaba--to know the true followers of Islam from the hypocrites.
We also perform Hajj (pilgrimage) to the Kaaba because this is what Prophet Abraham had done. He was commanded by God to call all of the people to gather for worship. To this day, millions of muslims answer his call, and come to the Kaaba to worship together.