It doesn't happen often, but beneficiaries who object to how an executor or administrator is handling an estate can ask the probate court to remove the personal representative and appoint someone else. Personal representatives who violate their fiduciary duty to deal honestly and fairly with estate assets can also be required to repay any losses they caused the beneficiaries as a result.
A court can always remove an executor who is dishonest or seriously incompetent. Generally, it's up to the beneficiaries (or estate creditors) to go to probate court and prove that the executor needs to be replaced. Each state has its own rules about what constitutes reason for removal, but courts will remove an executor who:
For example, an Illinois court removed an executor who had failed to account for the loss of more than $33,000 of estate assets and had neglected estate business. Even though there was no evidence that the executor was personally dishonest, he had failed in his duty to protect the estate assets. (Matter of Abbott's Estate, 347 N.E.2d 215, Ill. App. 1976) Similarly, a Texas court decided that an executor had breached his duty by renting out estate property for just half as much rent as had been charged by the late owner and taking a separate fee for selling estate property. (In re Roy, 249 S.W.3d 592, Tex. App. 2008)
Executors who make good-faith efforts to manage estate property probably won't lose their jobs, even if the results of their efforts leave much to be desired. For example, an executor who makes what appears to be a reasonable investment decision or sells real estate for what looks to be a good price won't be removed, even if the investment doesn't turn out well or beneficiaries think the real estate could have fetched a better price if it had been sold earlier or later.
A court may also remove an executor if the court concludes that the executor has a conflict of interest that would interfere with administration of the estate. Being the executor and also a beneficiary is not a conflict—it's very common, and it's not considered a conflict because someone who inherits from an estate has a strong incentive to take good care of estate assets.
The conflict must make it nearly impossible for the executor to serve as a fiduciary. For example, a New York court removed an executor who had personally guaranteed a loan to the estate. Because the primary borrower was unlikely to repay the loan, the executor would be in the position of collecting from herself. (Matter of Palma, 40 A.D.3d 1157, NY App. Div. 2007)
A court that removes an executor must appoint someone else to take over the job. If the will names an alternate, generally the court would name that person to serve, unless there's some legal reason the person can't fill the post.
If the will doesn't name an alternate executor, then the court will turn to state law, which will provide a priority list of those who are entitled to serve as executor. In most states, the surviving spouse is first on the list, followed by adult children and then more distant family members.