Some cheap materials you don't need to get from a fly shop and probably have already can be purchased for a few bucks:
Sharpies - Get brown, red, and black. You can color whatever the heck you want with them.
Paintbrush - use the bristles for nymph legs, mayfly tails, etc. They make a good replacement for microfibbets.
Ziplock bag - You don't need scud back. Use it for mayfly nymph backs, scuds, czechs, whatever. Use sharpies on them. Just dont pull quite as tight as you would with scud back.
Dryer lint and pet fur - Mentioned above. It's ghetto, but it's dubbing.
Sally Hansen's Hard as Nails - Raid your wife's makeup and use this for head cement, wingcases, etc.
Featherduster - I heard this one last night for the first time and I checked ours out when I got home and it will work. The synthetic ones you can get at the dollar store make great body material for small streamers.
Speaker wire - Strip off the coating, and untwist the strands, no need to buy UTC ultra wire. Yeah, it might be a little thick, but it can work in most cases. You can also rip apart small battery operated toys and the little engines in them might have some wire to be stripped. Want it in black? See Sharpie above.
Dental Floss - Don't get the flavored kind. Great for building underbodies on larger nymphs.
Condoms and Latex rubber gloves - cut em' in strips and make lots of worm patterns.
Pet stores - get to know someone who works at one that sells birds. Ask them to grab some feathers as they fall out.
Hunting - If you don't hunt. Find someone that does. I'm always getting offered duck, rabbit, pheasant, deer, etc.
xmas decorations - as mentioned above. All the flash you could ever want can be find here.
Craft foam - $1 a sheet at any craft store. Print out a 40% coupon before you go. AC Moore and Michaels always have them online. a brown sheet for $.60 will last you about 200 flies and you can tie everything from hoppers to beetles to mayfly bodies with it. Or buy white and use sharpies
(have I mentioned yet how essential markers are?)