Resale shops (Goodwill, Salvation Army, independent shops) are a great source of affordable costume pieces and material. Basic shirts, pants, skirts, and sometimes jackets are all good things for which to search in resale shops.
Unless you're looking for something basic (dark pair of slacks, white shirt, etc.), you can't expect to find something specific. However, you might very well find something you weren't specifically looking for. I love perusing the women's shirts, particularly in the white, black, and brown sections (Goodwill organizes by color) looking for inspiration. If I know I want an outfit to incorporate a certain color, I'll start there. I tend to be looking for lacy and frilly items, of which they always have a surprising selection.
Potentially, I might buy an item because it's made out of a particularly interesting fabric that I eventually use in something else entirely. I've cut off the bottom of a shirt and turned the upper portion into a shrug, for example.
Read more: From Shirt to Shrug
Thrift shop skirts are an amazing resource. While a full skirt still has to be big enough for you to wear it, being too big is a minor problem at worst, easily solved by the addition of elastic or a drawstring at the waist.. An overly long skirt can be shortened and hemmed. A ruffle could even be added to an overly short skirt to give it more length.
Overly big skirts also have the benefit of providing extra fullness, as well as being more easily transformed into a decorative overskirt.
If you're willing to do at least a bit of sewing, thrift store curtains are gold. Why? Because you can often find rich fabrics and patterns for $3-$5 per curtain panel. Buying that sort of fabric in a fabric store could easily run $20-$40.
Round tablecloths might also make fine and simple circle skirts.
Leather belts with metal accents (rivets, eyelets, rings, etc.) are fairly plentiful at thrift stores and only cost a couple bucks. Use them as is, or incorporate them into your own creation needing straps. Canvas belts are a good alternate if you're looking for a more grungy look.
Thank goodness some women are fashion-conscious about purses, throwing them out every few years so people like us can re-use them! Thrift stores are full of them, generally ranging $2-$10. While most are not particularly steampunk on their own, they make great pouches. Particularly small ones can simply be slit in the back and attached to belts as-is. Alternatively, buy a larger purse with multiple outside pouches, snip off the pouches, slit the backs and fasten them to belts. The remainder of the purse can sometimes be used for other products as well.
Let's face it: there's a lot of ugly items in thrift stores. After all, their owners don't want them. But if the only problem with a garment is its color, that can frequently be corrected with fabric dye, of which the most available brand is Rit Dye, which can be found at any fabric store.
Read more: Dyeing for a Better Tomorrow
Don't limit yourself to the section dedicated to your gender. Sometimes it's actually easier to find particular pieces in the other gender's section.
Read more: The Relativity of Gender