There's good information about thermocouples here:
OMEGA Engineering : Technical Reference. The Introduction To Practical Temperature Measurements seems to cover your question. I didn't read it closely, but it looks like you need to get a thermocouple with wires long enough to reach the DMM.