Deformation around the snap ring grooves would be bad. Discoloration of anodizing would be bad. Changes in case dimensions would be bad. Serious nicks or major scratches would be bad as they weaken local areas of the case. I'd consider excessive corrosion bad as well as that also weakens the case.
Essentially, if a case doesn't look really good, look at it really closely. If in doubt, get someone else with experience to look at it as well. Smaller cases are inexpensive enough that it is not worth risking a rocket for that one more flight.
The bigger the case, the higher the pressure, and the longer the burn, the more all of this matters. Commercial cases are designed with a considerable safety margin, but that is based on some assumptions. Get a case hot enough to destroy the T6 temper rendering it T1, and flying it immediately, would be begging for a failure. Wait a week and it ages to T2, and it is still asking for it. Anyway that's all the recovery you'll naturally get from aluminum. Chances are a small case would be difficult to get hot enough to be an issue. It can be done though with large ones!
The smallest cases probably have the biggest safety margins.
Gerald