Although the Peak Reservation Ordinance prohibited the Chinese from living on the Victoria Peak, there were exceptions. The Ho family was the first Chinese family to be allowed to live on the Mid-levels. The Ho family, whose members included the prominent businessman and philanthropist Sir Robert Hotung, possessed much political and financial power that put them and their influences above their peers, and earned them a place in the social hierarchy.

In 1914, Hotung's younger brother Ho Kom Tong commissioned a project to build a four-storey mansion on Castle Road in Central. This majestic building, known as the Kom Tong Hall, became Ho Kom Tong's primary residence. Ho had another house at No. 29 Conduit Road to house his twelve wives and concubines.

Ho's social and financial status were clearly demonstrated in the architecture of the Kom Tong Hall. The size and location of the building, which faced the Victoria Harbour and could be prominently seen from Kowloon, the granite pillars and gold-gilded wrought-iron balconies, the lavish interior decor, and the use of exquisite materials all conveyed Ho Kom Tong's nobility.

The building also retained the class-consciousness of traditional Chinese society. For instance, Ho accessed the building via the entrance on the first floor at the side of the building (which was placed there for feng shui reasons, so that Ho's fortunes would not slip away). The building also has two staircases, one for the Ho family, and the other solely for servants.

Kom Tong Hall was owned by the Ho family until 1959, when it was sold first to a Cheng family and later to the Church of Jesus Christ of Latter-day Saints.

The architecture of the building remained largely the same through the years. It was declared Grade Two historic building by the Antiquities Advisory Board in 1990, and was upgraded to Grade One historic building in 2009.

Since moving into Kom Tong Hall in 1960, the Church of Jesus Christ of Latter-day Saints (the "Church") did make some significant changes to the building: including the addition of a baptismal font on the lowest floor, windows to the openings along the verandah on the second and third floors, to turn the area into office spaces, and removing partition walls between bedrooms.

The Church also took down many of the original doors, including the louver doors (百葉門) along the verandah. Fortunately, the doors were stored away in good conditions and were returned in 2003, when the Government bought the building.

In the forty years that the Church occupied the building, Kom Tong Hall housed a chapel, an apartment for full-time missionaries, and other Church-affiliated offices.

In 2002, the Church announced its decision to demolish the Kom Tong Hall and redevelop the site into a religious cum education centre, to make space for its growing membership.

Both the Government and the public objected the proposal. Thousands of signatures against the plan were collected from residents of the Central and Western District, Legislative Councilors protested, and community groups joined in the plea for conservation. The Church agreed to negotiate its decision with the Government later that year.

Progress was made a year later when a deal was struck between the Church and the Government, in which the Goverment would acquire Kom Tong Hall from the Church for HKD 53 million, and convert it into the Dr Sun Yat-sen Museum. This was an unprecedented attempt of the Government to purchase a historic building for conservation.

In fact, the Government had been looking for a place to house the Dr Sun Yat-sen Museum. The decision to place the Museum in Kom Tong Hall was based on the building's availability, its location in the neighbourhood of the Dr. Sun Yat-sen Historical Trail, and that Ho Kom Tong and Dr. Sun Yat-sen were contemporaries, both being influential figures at the same period of time*.

Restoration works to convert the building into the Museum went underway in 2004, after the Government completed the purchase of the building. The aluminum windows that were put up for the verandah office space were removed (the markings of the window frames are still visible today), the original doors that were taken down by the Church were put back, and other modern amenities and facilities needed for a museum were installed, such as those for fire safety, humidity control, security, and barrier-free access.

The building was restored to its state before the occupation by the Church, though some additions by the Church were kept, such as the baptismal font. Alterations were kept to a minimum, and original materials were used or kept as far as possible. This marked a new attitude towards heritage conservation, particularly the respect given to built heritage.

The location to install the lift, for example, was greatly debated, in order not to disturb the original architecture. It had been proposed that the servant staircase would be replaced by the lift. However, after much deliberation, the servant staircase was deemed a fine example of class hierarchy in traditional Chinese society and was not to be removed. The add-on lift thus took the current location to avoid demolishing any existing part of the building.

The Museum was open to the public in 2006.

* Both Ho Kom Tong and and Dr. Sun Yat-sen were born in the same year (1866), they also studied at the Government Central School.