A key feature of arbitration is the possibility for conflicting parties to participate in the selection of the arbitrator, the individual who will rule the case. We analyze this problem of the selection of arbitrators from the perspective of implementation theory. In particular, theoretical analyses document problems with veto-rank—a simultaneous procedure commonly used in practice—and develop a new sequential procedure—shortlisting—with better properties. Experimental results are consistent with the theoretical predictions, highlighting both the disadvantages associated with the veto-rank procedure and the advantages associated with the shortlisting procedure.