Start by carefully pitting the umeboshi, taking care to pierce only one side; I suppose an olive or cherry pitter might work on some types of umeboshi. Medium-firm umeboshi probably work best; mine were already incredibly soft and I ruined a few while stuffing them.
I did the pitting by hand; I could feel the sharp side of the pit through the skin of the umeboshi, and squeezed the pit out through that pointy side.