Observatory Hill used to house the Academy's observatory. With the expansion of New London and Groton, the lights of the cities created a glow, which rendered the telescope useless. A new observatory was built in 2001 and is located in Stonington, CT.

Erwin Hall, one of many other names for the old observatory, used to house Cadet Objee, the Academy's live bear mascot. Cadet Objee, named because of his "objectionable presence", became a member of the Corps of Cadets in 1926 and was apart of the Corps for over fifty years. The 1st custodian of Objee was Cadet Stephen H. Evans who later would become the Superintendent of the Academy. Objee was and is the mascot of the Academy. He was often brought into the cadet barracks where he was set free to roam about. He was washed in the cadet showers and even was allowed to eat in the Cadet Wardroom. Unfortunately, the Corps was stripped of its right to keep a live bear in 1984 due to the "objectionable presence" posed by the bear on the surrounding community and new animal quarantine laws.

Across the street from Observatory Hill is the beginning of the Obstacle Course or the fitness course. It was designed to develop cardiovascular conditioning, flexibility, muscle-tone and balance conditioning. It is used throughout the year.