There is a difference between "worn" and "looks bad".
If you have a worsted cloth on your table you will get the inevitable white marks. This is not wear but burn marks from friction between the cloth and the cue ball leaving residue from the ball on the cloth. Worsted cloth is extremely smooth with no nap. Better quality ball sets with a superior finish will reduce the severity of these burn marks. They look pretty bad but it's not "wear" of the cloth.
On more common "napped" cloth the burn marks will be barely noticeable due to the coarseness or "nap" of the cloth however this type of cloth will develop grooves and valleys in the nap which affects ball roll accuracy.
Once you've been playing on your cloth for some time you will become accustomed to how it plays. When it starts deviating from the ball rolls you expect then that's a sign that the cloth is starting to "wear" out and due for replacement.
When it's time for cloth replacement we feel the best cloth in terms of accuracy, speed and durability is Simonis 860