A model is developed in which two financial firms, a "bank" and a "non-bank", compete duopsonistically for deposit balances. It is shown first that the imposition of a deposit interest rate ceiling on the bank can increase its profit. It is then shown that an increase in the degree of substitutability between the two types of deposits can reverse the conclusion regarding the profitability of an interest rate ceiling. The analysis is used to support the conclusion that a bank may initially seek the imposition of a deposit interest rate ceiling and subsequently seek its removal, thus providing a "private interest" explanation of recent bank deregulation.