The cats are well aware that they are too good for the orcs, such a contest would inevitably serve to only hurt the orcs, both physically and mentally. Therefor, the cats choose not to show up to this competition, which is obviously rigged in their favor simply due to the fact that they are cats.
The cats suggest that the orcs choose to pit themselves against someone closer to their own size, such as the mice.