God's House Tower stands at the south-east corner of the town walls that once encircled medieval Southampton. The whole structure is really an amalgamation of two buildings: a simple gatehouse, built in the late 13th century; and a massive spur work, an early 15th century addition consisting of a two-storey gallery and a three-storey tower. The building takes its name from the nearby hospital of God's House, founded in 1168 by Gervase le Riche as a refuge for poor travellers.

The original gatehouse was built to provide access to the newly constructed quay and remained more or less intact until the French raid of 1338, after which the defences were reinforced. God's House Tower is a culmination of that work and is one of the earliest forts built specifically to carry cannon.

The spur work provided artillery cover and protected the sluices that controlled the flow of water into a tidal moat used to power the water mill under the tower. The town gunner was made the highest paid town official and was responsible for making the gunpowder and gunshot that was stored, together with the guns, in the gallery of the tower.

By the 17th century the need for strong defences declined and by 1707 part of the building was being used as a house of correction. In 1786 it was adopted as the Town Gaol, a role it retained until 1855.

For 20 years the building stood empty until the Southampton Harbour Board requisitioned it for storage. In 1957 approval was sought to convert the building into a museum. Four years later in 1961 the Museum of Archaeology opened to the public and then closed in September 2011.