I study product diversity in presence of search costs and groups of consumers. I find that groups with heterogeneous tastes create a leverage effect on competition, so that a large majority of firms may end up offering a product that corresponds to the taste of the minority. I illustrate this idea with the case of smoking bans in bars and restaurants. When the first nonsmoking restaurants opened, there were few of them with little competition and high market power on nonsmokers. By extracting a large surplus from nonsmokers, nonsmoking restaurants became unattractive to other groups, while smoking restaurants were plenty and competitive, attracting both smokers and mixed groups.