The White Box Data Revolution

For the past 50 years, we have lived in a world dominated by “Black Box data” – data that is opaque, not well-defined, uses proprietary identifiers, and has poor data-quality feedback loops. This is rapidly changing – and nowhere more so than in the field of company data – as it is replaced by data fit for use in the modern world: so-called White Box data. This white paper focuses on this seismic shift – what White Box data means, the myriad benefits of it, how it is being used by some large companies, and how its importance will increase in the future, especially in company data. A dozen data experts were interviewed for this white paper and a clear theme emerged: White Box data can transform the way companies, regulators, investigators, and journalists work, while those that stick to Black Box risk becoming extinct.

But given that White Box data has so many benefits, the question might reasonably be asked: why do we have Black Box data in the first place? Why has, for example, the Dun & Bradstreet “business data” identified by their proprietary DUNS number been successful in the US, despite its many shortcomings? Why do firms use the DUNS number so heavily, even though it cannot tell them whether it refers to a business, a corporate address, or an individual, nor where or how the associated data came from? Black Box data has been the default because, in the past, it was sufficient to meet the basic data needs of businesses, governments, and banks. Data was typically looked at on a pay-per-view, record-by-record basis. The uses of data were very limited, by a small number of clients, and for those specific uses, Black Box was just about good enough.

The White Box Data Revolution