The state Government has given the building to Sri Ramakrishna Mission on long lease. It is an imposing building on the Marina along the beach road constructed in 1842 to store ice bars imported from the USA by the Tudor ice company and the business was on until 1874, when the ice was produced locally.

This is why it is called ICE House even to this day. Later this building was purchased by Bilagiri lyyengar in 1885 who called it "Castle Kernon". It was in 1897 Swami Vivkananda visited this city, after his much acclaimed Chicago Speech,on his return journey to Calcutta. Swami Vivekananda stayed in this historic building from Febrauary 6 to 15 in 1897.

The Government took over the building in 1930 and in 1963 it was named after Vivekananda. On the eve of his Centenary Celebrations, the building was renovated during December 1999 and the swami's statue was unveiled by the Chief Minister of Tamil Nadu on the 20th December 1999. The photographs depicting the historical happenings associated with his life are displayed here in the Museum.