Option (Mac) / Alt (Win) -drag with the Smudge too, to smudge with the foreground color (instead of the colors in the image).