In the area of today's Kazimierz there already existed a settlement called Windy Hill in the early middle ages, which alongside the neighbouring hamlets was bestowed at the end of the 12th century (1181) by Kazimierz Sprawiedliwy (Kazimierz the Fair) to the Norbertan nuns from Cracow's Zwierzyniec. It is thought that the nuns changed the name of the settlement to Kazimierz in order to honour their benefactor. The oldest document in which appeared this name comes from 1249.

The tower was built in the middle Ages and was most likely a part of the former castle. In the mid 14th century King Kazimierz Wielki (the Great) erected a castle made of stone just below the tower. The legend about the king’s relationship with a beautiful Jewess, Esterka, has survived till the present. The king built a castle for Esterka in nearby Bochotnica; it was supposedly connected with the Kazimierz fortress by an underground tunnel. The king also gave the settlement municipal rights.

The 16th and 17th centuries were the town’s golden age. Being a crossroads of trade routes, it became very rich dealing in goods, especially corn. The beautiful granaries along the river bank commemorate this period together with the renaissance outlay of the town and the magnificently decorated tenement houses that belonged to rich merchant families (Przybyło, Celej, Górski). At that time Kazimierz already had 3 churches, out of which the most beautiful is the parish church (fara) that was renewed in the style of lubelski renaissance and in 1620, received the largest organ in Poland at that time. The Swedish invasion brought the decline of Kazimierz’s magnificence; the town was never to regain it again. After the fall of the January Uprising, Kazimierz lost its municipal rights and became a country settlement.

After World War I, Kazimierz became a haven for artists, mainly painters. In 1923 Tadeusz Pruszkowski (professor at the Academy of Fine Arts in Warsaw) initiated open-air summer programmes for his students. A group of painters founded St Luke’s Fellowship. Antoni Michalak, a member of the group and student of Pruszkowski, settled in Kazimierz for good. In 1927 Kazimierz regained its municipal rights.

During WWII, the thriving Jewish community of Kazimierz all but disappeared. The only reminders left of the Jewish people are the XVIII century synagogue, the building of the kosher butcher’s shop and the Jewish cemetery - kirkut. After the war, the town became an artistic and tourist attraction. Painters and landscapists started coming to Kazimierz. Maria and Jerzy Kuncewiczowie settled here for good. Artistic events such as the Polish Folklore Festival and the Film Festival “The Film Summer" are held here together with summer concerts of organ music at the church. There are also museums: the Nadwiślańskie Museum, the Natural Science Museum, Kamienica Celejowska, the Goldsmith Museum, the Kuncewiczówka House, and numerous art galleries.