Data Automation Is on the Rise, Paving the Way to More Scalable Enterprises | 7wData

Data Automation Is on the Rise, Paving the Way to More Scalable Enterprises | 7wData

Thanks to an emerging generation of Automation tools, there’s finally an opportunity to clear away the onerous and administrative tasks that have bogged down IT and data professionals for decades. The technologies that are freeing them up from onerous data management tasks are guided by robotic process Automation (RPA), robotic data automation (RDA), low- and no-code platforms, and other forms of data automation that are making their way into the enterprise. It’s a matter of identifying productivity opportunities and applying the right tools at the right times.

The case for increased data automation is clear. “Data teams are spending significant amounts of time on service requests like infrastructure, user provisioning, and incident coordination and communication,” said Tina Huang, CTO and founder of Transposit. “Teams today are often manually creating tickets, Slack channels, and Zoom meetings, plus communicating with stakeholders. Data teams must ensure internal customers using data have access to the data they need and real-time updates about interferences with that data.” Other tasks ripe for automation include log parsing, correlation, permissions and access, and more.

However, while database automation and practices have been surging, this has not provided relief for data teams tending to day-to-day tasks. In a recent survey by Unisphere Research, a division of Information Today, Inc., 43% of data managers said the amount of resources spent on ongoing database management is severely limiting their competitiveness—an increase of 65% over a previous survey in 2020. It is notable as well that a total of 86% agreed that, to some degree, their administrative tasks are inhibiting corporate growth (“2022 Quest IOUG Database Priorities Survey”).

“Time is wasted on a day-to-day basis as teams work to improve application performance or respond to issues with applications,” said Cass Bishop, director at ISG Automation. In addition, there are other time sinks, such as “pulling search-based queries for their teams, and specifically frontline workers,” said Cindi Howson, chief data strategy officer at ThoughtSpot. “Frontline workers need the ability to answer data questions in the same way they’d search for information in their daily lives—which isn’t possible with cumbersome, legacy analytics tools that require them to constantly go back to their data teams.”

IT and data teams are also bogged down with “support requests for one-off API integrations between SaaS apps,” said Rich Waldron, co-founder and CEO at “IT is understaffed and overwhelmed due to greater support demands as pandemic-era distributed teams rely on an ever-increasing array of cloud apps. As more line-of-business teams use more tools that don’t talk to each other, they make increasing requests for integrations across those tools. The average IT team has a project backlog of 3-to-12 months. In the meantime, IT continues to face increased demands for strategic projects such as digital transformation and improved information security.”

With the rise of more comprehensive data analytics across the edge, the need to tend to tasks such as time-series aggregation is only growing as well. “The trend is clearly emphasizing the importance of understanding how and why the data changes over time, simplifying the mechanics of correlation and causation, and surfacing easy to spot trends,” said Francesco Crippa, vice president of platform engineering for Uniphore. “Who doesn’t like a chart where you see a single data point changing over time? But time-series data usually comes with the heavy need of aggregation. It’s painful to see the amount of small tasks in cleaning, aligning, normalizing, and adjusting all the differences in time-series aggregations when dealing with non-centralized data.

Images Powered by Shutterstock