You can look in your local ads, craigslist, pawn shops, etc. to find used (but not abused) gear for cheap. On my local craigslist, there are several complete PA systems for sale (complete with stands/cables/everything needed). But, it sounds as if you want to understand the basics, first.
The basics:
1. Some way to pick up the sound = mics
2. Some where to send the signals and mix it down to 2 speakers = mixing board
3. Some way to amplify the signal = power amp
4. Some way to turn the signal to sound = speakers
5. Some way to carry the signal between all of the various components listed above = cables
The main fixtures that you have to construct in relation to each other are the mixing board, power amp, and speakers. You can buy a powered mixing board (so you just have one piece of gear rather than two). You want to make sure that the power amp and speakers are rated as compatible, otherwise you'll end up with speakers that either will receive too much or not enough signal, and either situation will damage them (or blow your power amp). Once you have those things squared away, everything else is preference...kinds of mics, kinds of stands, kinds of cables, etc... Basically, just make sure that if you need a stand/clamp and a cable for it, you get them. It's best to figure out what your need/want is (what you want miked--kick drum? snare? how many toms? overheads? hi hat?), and then ask somebody at a music store to show you what you would need for your setup. Thank them, and then piece together what would work for better prices (whether used or online deals).
Hopefully this helps...