In a Federal system of government, each unit of government decides independently how much of each type of public good to provide, and what types of taxes, and which tax rates, to use in funding the public goods. In this paper we explore what types of problems can arise from this decentralized form of decision-making. In particular, we describe systematically the types of externalities that one unit of government can create for nonresidents, through both its public goods decisions and its taxation decisions. The paper also explores briefly what the central government might do to lessen the costs of decentralized decision-making.