In 1739 a wooden palace for Count Reinhold Gustaw von Loewenwolde, a favourite of the Empress Anna Ioannovna, was constructed on this place according to F.B. Rastrelli's project. Nearby the homestead of his lover N.F. Lopukhina was located, so Loewenwolde chose this site. After the Empress Elizabeth Petrovna came into power, Loewenwolde was exiled and the palace was confiscated. In 1743 Elizabeth presented the palace to her former favourite A. Y. Shubin. However, Shubin, being a former favorite, considered it unsafe to stay in Saint Petersburg and left the capital.

In 1749 the site became the possession of Kirill Razumovsky, the Ukrainian hetman and the president of the Academy of Sciences. In 1760 the wooden construction was demolished due to dilapidation. The newspaper «Saint Petersburg Vedomosti» published the following announcement: «Count K. G. Razumovsky's wooden house on a stone foundation for those wishing to dismantle, transport and put it on the Krestovsky Island ... can come to the house office». In 1762, the architect A.F. Kokorinov started constructing a new stone building. By 1766 J. B. Vallin de la Mothe finished the construction. A part of the garden has been preserved in the homestead till the modern days. From the side of the Kazan street it is fenced by the lattice of the Kazan Cathedral. Razumovsky Palace was famous for its pompous balls and masquerades. 260 servants were working in the palace. On holidays the count organized alms for beggars here and this event attracted more than 2000 beggars.

In 1781 Razumovsky sold the palace to the Polish count K.P. Branitsky and left Saint Petersburg. Count Branitsky lived here until 1798 and then left Russia. The Razumovsky Palace was sold to the state for 450.000 rubles. Shortly after that the palace was granted to the Orphanage created by the order of Paul I on 2 May 1797. The Emperor's wife Maria Fyodorovna patronized it and I.I. Betskoy became its first director. The Orphanage was intended primarily for poor children: foundlings and children with disabilities. A pelican sculpture can be seen on the facades of the buildings and on the entrance arch. This symbol reflects the mission of the institution located here. In 1797 the neighbouring building was granted to the Orphanage – the house number 50 on the Moika Embankment. In 1798 the premises of the Razumovsky Palace were redesigned . In 1829-1834 a house church and a two-story hospital were built here according to the project of D. Quadri and P.S. Plavov, a third floor was attached to the hospital in 1842-1844. Side wings were reconstructed in 1831-1834. In 1837 the Orphanage was transformed into the Women's Orphan Institute (in 1885 its name was changed for the Nikolaev Orphan Institute). Its students were taught the pedagogical profession. The Imperial Women's Pedagogical Institute, the first pedagogical institute in Russia, was established here in 1903.

In 1839-1842 a new building was attached to the institute. In 1868 a monument to I. I. Betskoy created by A. Laveretsky, a copy of the work by the Danish sculptor J. Zamelgak, was erected in front of the palace. In 1918 the name of the institute was changed for the First Pedagogical Institute. Later the Second and the Third Institutes and other independent educational institutions were opened here. In 1922-1925 the Leningrad State Pedagogical Institute named after A.I. Herzen was created on their basis. In 1961 a monument to the outstanding Russian teacher K. D. Ushinsky (sculptor - V.V. Lishev) was erected in front of the main entrance. In 1991 the institute officially obtained the university status.