It is probably not a violation of law for a board president to talk with all directors except the one person on the board who is a person of color.  Assuming that the directors serve as volunteers, their service would not likely be subject to anti-discrimination in employment laws.  It would most likely be a violation of the state nonprofit corporation law and/or the organization’s bylaws if the president tried to hold a meeting of the board without giving notice to the person of color.  Action taken at such a meeting would not be valid if the director did not attend and could be overturned in litigation.

The real question is what is this all about?  Is this an intentional snub?  Is it based on a personal dislike of the individual?  Is it based on conscious or unconscious bias?  But more importantly, is it condoned by the other directors?  Whatever the reason for the conduct, a majority of the board could put a stop to it or remove the president from office if it isn’t changed. 

An officer can normally be removed by a simple majority vote whenever the board deems it appropriate to do so, without the requirement of a lengthy hearing process or a finding of legal “cause.”  If the president doesn’t want to be removed, the others on the board hold a pretty big deterrent if they are willing to exercise it.