Officials say the trustees were given notices, but failed to prove the legality of the structure