My thinking is
-woodpeckers that make big holes (to live in) are pecking into decayed stuff anyways...so no worries about the woodpeckers there. Just check out the decay.
-If they are making little holes in an organized grid up and down the trunk or branches are sapsuckers and the damage is superficial.
-There are birds around here that peck quarter sized holes than put acorns in the holes (to save them for later) I'm not worried about that either.
My guess is that they don't damage the tree to a point where we should be concerned about it from a safety standpoint.
Just to play devil's advocate here, Nick...you say a woodpecker that digs out a quarter sized hole deep enough to hold an acorn doesn't worry you, that it is doing no harm. Yet you would certainly say that a climbing gaff wound, likely smaller and no deeper, is something that can be unacceptably damaging to a tree.