Andrew,
I think there are no real tricks or short cuts to create great stuff. It's musical taste/maturity, experience, mastery of the instruments to some degree, and quite importantly, inspiration or the desire to create something new and not just copy what is already out there.
I learned it helps me to listen back to my stuff/ideas from time to time without having a guitar at hand (as for drums - I'm not good enough to create drum lines "in real time" by playing but arrange them on GuitarPro which I'm using to notate all my stuff). When I listen to my orig. stuff with a guitar on my lap or even playing along to the material I have a different perception and lack the distance necessary to decide whether some parts need to be worked over. This could be the tempo, instrumentation, time signature.
This listening back approach is quite effective for me. Usually I would rework ideas until I can't detect anything "sticking out" of what I'm hearing. In case something strikes me I would write it down (in a TXT file), note the song part (e.g. 3:05) and what I want to change or where the problem is. Then I would open the GuitarPro file, address that problem and listen back another time. I think I came up with good drum parts just by doing this - changing stuff until I was satisfied. Designing guitar parts is way easier for me as I'm coming from the el. guitar.
Record any ideas/fragments you have, organize them in a meaningful way, listen back from time to time and you'll see which parts fit together well.
I'm also applying a technique I've learned from a friend of mine. Sometimes I would change the tempo of some parts by 2-4 bpm up/down to make the specific part more effective. This is really a fine-tuning measure but it works quite well. Some songs simply don't need this but on others this trick works fine.
I think patience is a must when writing songs. I'm taking all the time it takes until I'm satisfied with something or until I can't get it any better. In rare occasions I would write a song in 2 days (happened a few times), in most cases the process will last months. I'm in no hurry, just collect ideas, work on them, try to combine them etc. I do try to proceed with some ideas every now and then, hoping for creativity. If there is no on that specific day I would simply fall back on practicing the guitar or drums and wait for new inspirations.
As for the arrangement/orchestration: Within my gothic metal project I have some classical parts here and there, and some harp and harpsichord on a few spots. Those classical parts create some nice variation.
This all refers to a songwriting context where I'm the sole songwriter. On some material I'm working in a team of two (gothic metal recording project), but I'm the main songwriter there, too. Now if you're working with a bunch of musos and every one of them can contribute inspirations/ideas/parts etc. the songwriting process will be different - probably faster and having more variation.
One other thing I'm noticing in my work is that I tend to write new parts instead of repeating existing ones. This makes the songwriting process even longer (esp. as the sole songwriter) but gives me more satisfaction in the end. This is not an approach I'm forcing myself into, I'm merely observing it. I have several songs which basically have no repeating parts, they keep on evolving all the time. The amount of work is suicidal, so what ;-)
As for vocals... I've learned an interesting lesson for myself. Within that gothic recording project the lyrics were provided by my musical partner. He also had a few melodic ideas I was able to work out, but in some cases I started with a sheet with the vocals on it, nothing more. I just kept looking at the vocals and trying to come up with good melodies which I could really relate to. Those vocal melodies were THE FIRST thing I did, all the rest fell into place accordingly (although over months usually). That's just a completely different approach than writing some riffs and add the vocals afterwards. Now if I create new ideas I would always try to come up with a riff but also try to find a matching melody at the same time. I think having vocal melodies & riffs which go together completely organically/naturally is so important. I'm not sure I'd ever go back to adding the vocal melody later.
