There is always a risk of the loss of exempt status if a charitable organization is used for the private benefit of insiders or others.  (See Ready Reference Page:  “Charities May Not Confer Private Benefits”)  The IRS has denied exempt status to artists’ groups where it felt that the primary purpose was to promote the small group of artists who were members and not to promote the public’s general interest.

At a minimum, you want to be sure that the compensation is reasonable for the talks they give.  You want to be sure that the speakers bureau is integral to your charitable purpose and not simply a way to provide extra income to certain writers.  You want to be sure that any benefit for the speakers is qualitatively and quantitatively incidental to the public benefit of your activities.

You also raise some administrative questions about the separateness of the speakers bureau.  If it is the organization’s bureau, the organization is responsible for it.  For the speakers to be truly separate, they would have to establish their own entity.  Assuming it is your bureau, do you want to subsidize the operation by paying the costs of website, promotion and marketing without allocating a portion of the fees received to cover those expenses?  There are no absolute answers to these questions, but you are correct in questioning the appropriateness of the arrangement, both legally and in its public appearance.