The authors consider a k-player sequential bargaining model in which the size of the cake and the order in which players move follow a general Markov process. For games in which one agent makes an offer in each period and agreement must be unanimous, the authors provide characterizations of the sets of subgame perfect and stationary subgame perfect payoffs. With these characterizations, they investigate the uniqueness and efficiency of the equilibrium outcomes, the conditions under which agreement is delayed, and the advantage to proposing.