The SPD SX only has two inputs.
The SPD 30 has 4.
I am in the middle of doing the same thing. I am going to use a TD6 and 6 pads to trigger pads on an SPD SX or 30 (haven't taken the plunge yet) via MIDI.
Top 5 pads are for triggering recorded phrases. Far bottom left will be a kill switch pad because ....stuff happens.
Right now they are mounted on the same rack but I think I am going to have to build a separate one to avoid crosstalk. I won't know until I actually field test the sampler. I've been experimenting with using the TD6 module as a sampler and there are too many issues to deal with.