It really comes down to the kind of relationship you have with whoever is doing the exit interview or the company. If you think they would be positively receptive to your comments and that it might help them improve themselves and maybe even be reflected positively in their future dealings with you, then tell them. If you think they'll feel like you're just blaming them to get a dig in before you leave or don't think they're at all likely to even try to make improvements, then don't tell them.
It comes down to whether you are reasonably confident they'll view you telling them the truth as an attempt to help them. If not, don't take the risk, and tell them something safe about new challenges, new opportunities, job growth, or whatever.
