Every time, you use gems of a certain color, it will give spirit to your own Dragon of that color. You can witness that by the yellow spirit bar filling up below your green health bar.
Try out different strategies to use the gems as well as your own special abilities for a victorious battle!