Some Dragons do damage over time, so it might appear as if your attack has no effect at the moment. However, your opponent will feel the burn on its next turn.
Also keep in mind that if a Dragon of a specific color is missing on your team, very little damage is done by those gems.