The World of Avlis

I realized today that gems don't have any abilities that can be added to them. Is there any way to add the "Cast spell" ability to them?

No. Maybe you can assign a special use power to it and tie a spell to that, but you can't assign a spell directly AFAIK.

EDIT: There are several things in the misc small category that could be possibly used as gems. You can assign spells to misc objects.