Managing the Modern Data Warehouse: Overcoming the Tragedy of the Commons

Kevin Petrie's picture

This is the second in a series of blogs that examine how Fortune 500 companies use Attunity Visibility to manage their data more effectively.  Part Two focuses on data warehouse performance.

The English economist William Forster Lloyd highlighted a small problem with big implications in 1833 when he pointed out that individual cattle owners tend to overuse public land.  By acting in their own self-interest they hurt everyone.

This “tragedy of the commons” applies to many shared resources, including the environment, public infrastructure – and the modern data warehouse.

Today IT struggles to meet demands from the lines of business to access and analyze data that is increasing sharply on every dimension – volume, variety and velocity – while staff and budgets often grow in the low single digits.  Data volumes double about every two years and new unstructured streams like social media need to be assessed – much of it real-time, for example to identify sales opportunities.  As a result, shared data warehouse CPUs and storage space are collectively being overused by lines of business.

This is the tragedy of the commons for the modern data warehouse.

Time is often the primary victim.  Whether you work in financial services, pharmaceuticals, tech manufacturing or other industries, your business leaders need to extend customer offers, tune supply chains and make other decisions based on well-grounded insights that often require rapid queries.  But central data warehouses cannot return the results fast enough, and IT cannot identify the runaway users and bottlenecks that cause the problems.

But with visibility into how common resources are being used, you can.  Here are three examples of how customers use Attunity Visibility to measure data warehouse usage and improve performance.

  • Union Bank decides where to upgrade or correct systems based on usage trending of query activity by hour, day and week. This improves end user response times, ensures system availability and helps meet service level agreements.
  • Pfizer gains correlated, granular insight into user activity, application data and database information over time, without any disruption of production warehouse systems.  They share the resulting insights with BI stakeholders to make better decisions, for example reducing user IDs by 90% and cutting ETL load times by 75%.  They have saved $80K/year and now better meet their SLAs.
  • Application DBAs at one of the world’s leading tech manufacturers uses the Visibility dashboard of historical analytics for application testing and tuning to easily identify specific application users, their usage trends and patterns over any hours, days or weeks.  This enables them to identify root causes of bottlenecks, and rapidly tune and optimize applications to improve response times.

Check out this video to learn more about data warehouse performance analytics with Attunity Visibility.