Chicago Magazine - The secrets of shopping 5-15min

What economists and marketers are learning from newly accessible consumer data … Around the world, billions of sales transactions every month, down to a can of Coca-Cola from a local store, are recorded in some way by Nielsen, the measurement and information firm that has been gathering data from retailers and consumers for 90 years. For most of its history, Nielsen shared those data primarily with its retail customers and manufacturing customers under strict agreements that protected customer confidentiality. Academic researchers gained access to some data by negotiating directly and often at length with Nielsen, or by partnering with a corporation and promising the data and results would be for internal use only. … Now Nielsen is sharing three datasets through Booth, with a staggering amount of information. One dataset covers purchases by 40,000–60,000 households in the United States. Another contains sales results from 35,000 stores—grocery stores, drugstores, discount chains, and similar outlets—for the years 2006 through 2011. Those records span up to 3 million bar codes, and the data represent about 33% of the volume at mass merchandisers and about 55% of US retail volume from grocery stores and drugstores. … The information now available is a gold mine for researchers, marketers primarily, but also economists who see the potential to explore longstanding questions about consumer behavior.