Rune Factory 3 - Cactus Flower?

Okay, I got the bones and now I am trying to get the flower. I am at the place & I defeated all the monsters but whenever I walk up to the cactus with the yellow flower on it nothing happens? Am I searching in the wrong place? Please help!