The building of the castle was started in 1749 by the Vukovar landowner, Count Anselmo Kasimir Eltz. At first, he erected only the central part, and new wings were later added on several occasions. Since 1968, the Eltz Castle has hosted the collection of the Vukovar Town Museum. The building suffered great damage in 1991. Today, its interior has been partly reconstructed, and reconstruction should be completed in the near future.