Dane Rook, head of research at Addepar and research engineer at the Stanford Research Initiative on Long-Term Investing, answered some questions from CIO about his paper, Governing Your Way to Better Long-Term Returns, written with Ashby Monk, strategic advisor at Addepar and executive and research director, Stanford Research Initiative on Long-Term Investing.
His answers below have been edited for brevity and clarity. A story about the research findings is available here.
Q: What would be the best ways for a public pension fund (which may have complicated governance structures and with competing interests) to implement strong data governance?
Dane Rook: Start with cultural introspection – that is, get a deep feeling for what people’s attitudes and senses of responsibility toward data are across the organization.
For example, if an error is spotted in a dataset, how would most folks go about addressing it: would they simply flag that error for someone else, or try to track down its genesis, or just ignore it? Likewise, what are most people’s typical knee-jerk reactions to someone in the organization passing them a new dataset or piece of analysis: do they accept its quality without questioning it (at least until they spot something flagrantly wrong), or is their first instinct to ask how that data was sourced and handled?
This kind of introspection is the best first step to strengthening data governance, regardless of whether the organization sees itself as already having formal governance in place. Introspecting in this way – before doing some organization-wide data audit – can reveal areas for improvement that would be far harder to find if you were instead trying to quality-check datasets en masse (although such audits are definitely merited at some point!) In introspecting, most organizations should start by focusing on the people who use data most intensively – usually analysts or junior folks.
Observe what datasets they touch most frequently, and what they do with them and why, and what their beliefs are regarding how that data will get used by others and their obligations to those other users (with respect to data quality). Once you have a solid picture of how these intensive users treat data, you can move on to trying to understand the nexus of decisions and data – i.e., figuring out what data gets used in the organization’s key decisions, and what the quality controls are for that data. That will typically yield good insights on what additional structures and mechanisms are needed to deliver the right level of data quality. After that, most organizations should work to catalog their datasets, audit them, and then decide if the processes by which each of them are produced and consumed have the correct form and level of governance – and not forget to actually write down \ all of the foregoing: governance needs to be explicit and visible to work well as intended.
Q: What are the best ways for investment teams with fewer resources to implement strong data governance?
In many ways, lean teams are probably better-positioned to implement strong data governance, because such teams will often (relative to their more-resourced peers) have less headcount, do more outsourcing of investments, or both.
Rook: Outsourcing can help because it shifts much of the DG burden to external asset managers, but that shift is successful only when those managers can be trusted to have their own good governance in place (which is something that must be verified when doing diligence on them – good track records aren’t sufficient proof of good data governance).
Small team sizes can help because the best DG is underpinned by clear, fluid communication, which becomes more challenging for larger teams. And it’s not just verbal communication that matters: documentation is also crucial, and both producers and users of datasets should be logging the details of how they source, modify, and use data. That documentation then becomes the connective tissue for communication, but it’s almost always valuable for data users – i.e., decision-makers – to talk directly with the folks who produce the data they’re using, to better grasp its limitations and any caveats.
Don’t under-value the benefit of such interactions taking place in person: it can be surprisingly enlightening for a data user to observe the steps taken in sourcing and transforming data, and be able to ask questions ‘live’ (this can be especially useful in identifying ways to enhance data quality).
So, overall, leaner teams can get away with relying more on people than processes (and technology), as long as there’s a strong willingness to hold people accountable, and for people to fully embrace their data responsibilities.
Q: Is there any variation in the implementation of DG relative to fund size, fund type or geography?
Rook: Yes, the degree of heterogeneity is enormous, but probably less so in geographies where best practices and standards are more visible, due to more extensive communication among industry peers (like what occurs among public pension funds in Canada or the superannuation schemes in Australia).
However, the variability that occurs across fund geography, size, and type (if you take ‘type’ as being things like public pension fund, endowment, sovereign fund, etc.) is less than the variability across fund strategies. By and large, asset owners that engage in a high level of in-sourced active management tend to have better DG practices and structures simply because the need for those is more apparent. Likewise, funds that have a higher proportional allocation to private/alternative assets also tend to have more evolved DG, because the need for it is more salient (and not because there’s more data on those assets – since there isn’t).
The tricky thing is that improving governance isn’t merely about having ‘more’ of it, but rather having it be more efficient, i.e., delivering more quality per unit of resource, whether that’s denominated in dollars, hours, or exasperation. From what we’ve seen, there tends to be more governance with higher portfolio complexity, but not necessarily more efficiency, or even sufficiency for that matter: a complex portfolio with extensive governance (in terms of data committees, stewardship mappings, documentation, etc.) may have poorer governance fitness than a very simple portfolio with light governance.
The most significant variable we’ve noticed in driving fitness and efficiency of governance is ultimately the buy-in of executive leadership on the importance of data governance: having it smooths the path to success; but not having it pretty much guarantees failure.
Tags: Addepar, Ashby Monk, Dane Rook, Data Governance, Stanford, Stanford Research Initiative on Long-Term Investing