They were hassling the man and his children who came to pay their respects to his ex wife. He didn't bring his current wife and they kept moving the man here and there. I would have gotten irritated too. But from the accounts they asked him to leave after inviting the man. Clearly they invited him only to dog him. They could have made room for that man.
