When Allah first ordered Prophet Mohammad (S.A.W.) and all Muslims to offer the daily Pray, they were required to do it facing Jerusalem. This was the practice in Mecca and continued in Medina until the seventeenth month after Hijrat.
In Medina, the Jews also said their prayers facing Jerusalem. They did not like the fact that the Muslims had the same Qiblah as they did, and tried to use this fact to discredit Islam and Prophet Mohammad (S.A.W.). They said to the Muslims, "Mohammad claims to have a religion whose laws supersede all other previous laws, yet he does not have an independent Qiblah, and offers his prayers facing the Qiblah of the Jews."
After Prophet Mohammad (S.A.W.) received this news he used to come out at night and look into the sky awaiting the revelation from Allah about this matter. The following verse was revealed at this time:
"Many a time We have seen you turn your face towards heaven. We will make you turn towards a Qiblah that will please you..." Surah al-Baqarah, 2:144
One day, while Prophet Mohammad (S.A.W.) and the Muslims were praying together, the command came from Allah to change the Qiblah from Jerusalem to the Holy Ka'ba in Mecca. After Prophet Mohammad (S.A.W.) had already completed half of the noon prayer, the Angel Gabriel (A.S.) communicated to him the command of Allah.
He held the hand of Prophet Mohammad (S.A.W.) and turned him towards the Holy Ka'ba in Mecca. Prophet Mohammad (S.A.W.) at once changed his direction in the middle of pray. Hi cousin, Imam Ali (A.S.), followed this change immediately. The other Muslims were confused by this action and only a few followed the example of Imam Ali (A.S.).
The mosque where this happened is known as "Masjid-e-Dhul Qiblatain" which means "The Mosque with the Two Qiblahs". This mosque still exists in Medina today.
With modern instruments and science we can pinpoint the exact location of Medina to be at latitude 24 degrees and longitude 39 degrees. This makes the Qiblah 45 degrees south of Medina.
Prophet Mohammad (S.A.W.) turned towards the new Qiblah without hesitation. The old and new Qiblahs can still be seen today in Masjid-e-Dhul Qiblatain. It was one of the Holy Prophet's (S.A.W.) miracles that he turned exactly to face the Holy Ka'ba without the use of any scientific instrument or computation.*
(*) Source: http://www.ezsoftech.com/islamic/qiblatain.asp