We're afraid you’ll need to be persistent with this! Dig up as much of the roots as you can (dig deeply rather than just pulling them up from the surface). This won’t completely eliminate the horse radish but the resulting new shoots that emerge may be easier to kill with a weed killer. Make sure you use a systemic weed killer that travels through the plant to the roots (such as Round-Up). With a combination of systemic weed killer and pulling up any new shoots you should eventually eradicate the horse radish (although it will take a few seasons).