I've often wondered this myself. It's definitely changed over time, for me.
As a teenage boy, I loved Rome for all the stereotypical teenage boy reasons -- the images of power, the cool military uniforms, the way Rome embodied all the so-called masculine virtues (military prowess, effectiveness in engineering and using tools, obsession with building huge things...). And for some reason, I was obsessed with maps, and loved looking at all the phases of Rome's expansion and contraction.
Now, I'm fascinated by the how the culture transformed, especially during the later years of the empire. It's like a big puzzle, looking at how Romanitas was transmitted and transformed as it passed to the German peoples, how its structures and influences were adopted by successor states, even by the Arabs to some degree.
But, I have to admit, I still love looking at those cool uniforms...