It used to be a lot worse. The developers know about this issue and some time ago added a system that reduces the weight of your clothes by 25%, which can cut a few kilograms. This makes sense, and after all, would you not be feeling slightly more weighted down if you were wearing for instance a big bearskin coat?