You can get rid of the Cactus by matching adjacent blocks or using Bomb Items.
When matching adjacent blocks, you need to match the same color blocks as the Cactus's Flower. If there are more than one Flower on the Cactus, you need to match the adjacent blocks several times to get rid of the Cactus.