Take your dirty spice grinder and fill it with dry white rice.
Grind the white rice until it turns into a fine powder.
Dump out the rice powder, knocking it out with your finger or a paper towel.
If you want to take it a step further you can use an air compressor or canned air to blow out any remaining residue.
Wipe the remaining residue out with a damp paper towel.
Dry with a paper towel and your grinder is squeaky clean.