The main building at Castle Hill Inn was built in 1875 for a ... actress would take a dip while filming “High Society” in Newport in 1956 with Frank Sinatra and Bing Crosby.