Yoma 37 What was Written on the Lots?

The Gemora states that the lots could be made from any material. One might have thought that it would be required to be from gold. Just like the ציץ (headband) of the kohen gadol said la'Hashem on it and it was made from gold, so too the lots, which one of them said la'Hashem on it, should be made from gold. The Gemora brings a passuk to show why we don't make this comparison.
The Sfas Emes asks how did the Gemora know that the word Hashem was actually on one of the lots? Would it not have been sufficient to write some sort of hint on one of the lots that this was for Hashem and not the azazel?