You should want to purchase something because it makes you look and feel beautiful and confident. If something does this for you but maybe the pants are a little long or something needs to be taken in, take the time to get it fixed. You'll be happier investing in a piece that uplifts you every time you wear it, instead of substituting it for something else (unless altering it causes you to lose what made you feel and look confident and beautiful). This is often the value of investing in luxury fashion.
Although fit is mostly about finding the right size, it's also about finding pieces that fit your personal style. A designer may create something to fit baggy but if you're a feminine chic woman, you'll likely want that item to be form-fitting, so you're likely to go down in size to get that item to fit your style.
An item that fits perfectly fits your personality, your body style, your mood and your size. Since your mood often changes and because most things aren't custom made, you can't truly find the perfect fit. But when something fits your personality and body style, it's the right fit and is almost always what you're attracted to. You'll know it fits because you look and feel beautiful and confident while wearing it.
The next time you go shopping, find something that is the right fit for you and tell me how it made you feel when you tried it on, I can't wait to hear about your experience in shopping this way!