COEP's precursor, The Poona Engineering Class and Mechanical School was opened in July 1854, with an aim to provide suitable learning to the subordinate officers in the Public Works Department. The School was situated in Bhawani Peth, Poona , and the accommodation consisted of three small detached houses for teaching purposes and a separate house for the Principal.

In June 1865 Mr. Theodore Cooke, M.A , who held that appointment for 28 years was appointed Principal. The foundation stone of the new college was laid by His Excellency the Governor, Sir Bartle Frere, on the 5th August 1865.

College was affiliated to University of Bombay in 1866. In 1868 the College moved to the New Buildings. The college was divided into three departments for matriculated and unmatriculated students.

In 1879 two new classes, an Agricultural class and a Forest class , were added to the college , and the name of the college was changed from " The Poona Civil Engineering College " to " The College of Science".

Thanks to the vision in setting up such an establishment at Pune, the credit for most of the civil infrastructure in India goes to the alumni of this college, starting from none other than the most towering and illustrious engineers, Bharat Ratna Sir M. Visvesvarayya, in whose honor "Engineers Day" is celebrated. The school became "Poona Civil Engineering College", and subsequently in the year 1911, the nomenclature was changed to the "College of Engineering, Poona". It was initially affiliated to the University of Bombay for a degree of Licentiate in Civil Engineering known as LCE and later to the University of Pune. The degree programs in civil engineering, mechanical engineering and electrical engineering were started in 1908, 1912 and 1932, respectively. From then on, the college has gone on expanding adding new departments and new wings by the year. 

In 2003, the college got autonomous status, thus giving it the freedom to set its own curricula and manage its own finances. This has been the biggest change as far as pedagogy at COEP is concerned.

Today, COEP offers nine UG and twenty-three PG programmes, and has more than 3200 students enrolled in its various courses.