Historian Arthur Schlesinger once said, “Science and technology revolutionize our lives, but memory, tradition and myth frame our response.”
While science and tech have certainly revolutionized our life and times, I’d venture that big data (in addition to memory, tradition and myth) is close to being able both to frame and predict our responses.
It sounds scary, but big data is helping businesses and consumers alike. Its benefits for businesses — including financial institutions — include greater insight into consumer behaviors, fraud detection and prevention, and enhanced operations. On a more personal scale, it also helps consumers keep informed of the things they want and need to be informed about (e.g., their health via wearable tech like Nike+ and FitBit).
But achieving these data benefits is difficult because of five big challenges. Known as the five “V’s” of big data: volume, variety, velocity, veracity and value. These challenges are, ironically, the very things that make data insights so valuable and so difficult to harness.
Volume
How big is your data?
Having large samplings of data is a good thing. After all, the more you have to work with, the more you can do with it. More data leads to more sound analysis and better decisions. This is especially true when predicting future outcomes. Marketers, for example, have increased clarity into the future to gauge the likely effectiveness of a particular strategy or campaign — versus relying entirely on past performance. (A good analogy is deciding whether to cross the street at a busy intersection. Do you make the decision based on what happened yesterday, or even five minutes ago? Or do you make the decision based on what’s happening right now?)
Having a wealth of data is important. It’s just difficult to corral.
We’ve seen the mind-numbing statistics around big data, especially social media:
- Facebook has an average of 8 billion daily video views, totaling 100 million hours per day of video watch time
- Facebook accounts for 1 in every 6 minutes spent online (1 in every 5 minutes spent on mobile)
- 6,000 Tweets are sent every second
- Instagram users share more than 80 million photos per day
- Instagram users like 3.5 billion photos per day
- 1 in every 3 professionals in the world is on LinkedIn
- LinkedIn users conduct 1 billion searches per day
- YouTube has 4 billion views each day
- The average YouTube viewing session via mobile is 40 minutes
Combined with GPS tracking, Bluetooth and Near Field Communications, data is also available through our mobile devices and everyday objects such as our refrigerators, television sets and automobiles.
It’s literally everywhere. And it’s growing. Each day, more than 25 billion gigabytes are added to the big data ecosystem. And by 2020, it’s predicted that 40 zettabytes (43 trillion gigabytes) of data will have been created. In fact, if we took all the data generated in the world between the beginning of time and 2008, the same amount of data will soon be generated every minute.
The Impact on Financial Institutions
Volume is a particular challenge for financial institutions because not only do banks and credit unions need a way to store the data, but they also need qualified personnel to mine it for appropriate conclusions and (most importantly) a way to secure it for compliance, regulatory and customer peace-of-mind reasons. Existing legacy platforms can’t keep up with the volume of data coming in, so most financial institutions must seek new solutions that can reach into their legacy data stores to complete the picture for regulatory compliance.
These challenges are not insurmountable. But the complexities of addressing them — primarily by creating processes for data acquisition and management — prevent many financial institutions from getting started at all. Keeping an eye on the prize helps to overcome the initial hurdles. Remember: by leveraging big data, financial institutions are reaping unlimited insight into their customer’s way of thinking and priorities.
Variety
“Variety is the spice of life” — or so goes the old adage. When it comes to online media, the phrase definitely holds true. Think of all the different types of data you consume online in your day-to-day life: photos, videos, articles and shareable content on the web; health data via your FitBit or other wearable device; not to mention the data from marketing companies, IT, and other businesses.
Looking at social media alone (the most common source of big data), in one month:
- 30 billion pieces of content are shared on Facebook
- 400 million Tweets are sent
- 4 billion hours of video are watched
Add the data consumed via email, Bluetooth, GPS, Near Field Communications, online purchases, offline purchases, etc., and you see that it’s not just the sheer volume that is a challenge to financial institutions and other businesses, but the variety, as well. In fact, volume is the least of the challenges.
Historically, most data, including financial information, has been structured, fitting neatly into spreadsheets and relational databases, organized by product, region or salesperson. Today, more and more of the world’s data — 80 percent, in fact — is unstructured. Big data allows the combining of these different types, but the variation in formats and sources makes storage and analysis difficult.
Fortunately, the challenge sounds worse than it is (how could you possibly store a video in a way that also takes into account text and photo?). Thanks to the advent of new technologies, such as “data-as-a-service” cloud solutions and packaged solutions like the “smart data lake,” the bridge between gathering data and using it to make business decisions is smaller than ever.
The Impact on Financial Institutions
For small to mid-size financial institutions, these new technologies are especially compelling. Data-as-a-service provides the flexibility to select the tools a bank or credit union needs to solve specific business problems while still providing a single source of all enterprise data with documented quality and provenance. The smart data lake also provides a single source for all enterprise data, but allows the ability to collect vast amounts in its native, untransformed format — at a low cost.
Velocity
I’m looking at velocity, in terms of both how fast data comes in and how fast it’s now expected to come out in usable forms of information (i.e., in real-time).
Did you know that the New York Stock Exchange receives 1 terabyte of data each day? Or that by the end of 2016, was an estimated 18.9 billion network connections — nearly 2.5 for every person on the planet?
With so much volume — and so many internet connections to capture it — the transmission of data is faster than ever. Data flows between machines, networks and humans, through mobile devices, social media channels and business processes, faster than the speed of light, literally.
Think of how quickly we’re able to stream videos and music, or complete credit card transactions at retailers, or examine analytics in real-time for corporate websites. This flow of data is ongoing and massive, and (combined with the other four “V’s”) can be overwhelming if the necessary infrastructure to manage it isn’t in place.
This is especially important for financial institutions. Consider the common uses of big data for banks and credit unions:
- Better manage risk and compliance
- Forecast sales
- Increase account holder engagement and retention
- Improve the customer experience
- Create more strategic marketing campaigns
The above uses are all time sensitive and need data in real-time to deliver optimal results.
Factor in increased pressure from Fintechs, P2Ps and other tech-driven competitors and the stakes are even higher to get it right.
The good news is that today’s technologies allow for data to be analyzed in real-time, as it’s being generated, without having to be put into databases first. These technologies create a “feedback loop” that allows businesses to deliver data quickly where it’s needed while also generating usage insight from customers and other end-users.
The Impact on Financial Institutions
Many financial institutions are looking to the cloud in the form of software-as-a-service (SaaS) applications to help them overcome velocity (as well as the other four “V” challenges). SaaS allows financial institutions to collect data through a remote service and avoid overloading their existing infrastructure.
Open source software is another way that banks and credit unions can address the velocity challenge of big data. All they have to do is plug their algorithms and policies into the system and let it handle the increasingly demanding tasks of processing and data analysis.
Veracity
While volume, variety and velocity are considered the “Big Three” of the five V’s, it’s veracity that keeps people up at night.
New technologies like cloud computing and open source software can process great volumes and varieties of data at great speeds. Except you still have concerns. You’re getting all this great data, but how do you know if it’s accurate?
Four Ways to Ensure the Accuracy of Big Data
One in three business leaders don’t trust the data they’re given. And for good reason: flawed data costs the US economy approximately $3.1 billion annually. Keeping data clean, unbiased and free of noise and abnormalities can be a full-time undertaking.
Fortunately, there are a handful of ways to ensure your data is valid:
- Create separate processes for gathering data and analyzing it
- Find a technology with built-in operational assistance: algorithms for data validity, checks and balances, and the ability
to quickly scour multiple sources for correct information - Appoint someone to familiarize themselves with the data so they can spot anomalies and outliers (technology is incredibly useful, but it isn’t flawless)
- In extreme cases, it may be worth investing in a secondary data processing source so you can check by hand if something seems “off”
The Impact on Financial Institutions
With the scrutiny that financial institutions are under (from account holders and regulators), accuracy is a paramount concern. Their data needs to be verified for validity and context. There’s no point paying for big data architecture if problems with veracity lead to poor business decisions or compliance failures.
Value
For all the tactical considerations that come with these challenges, deriving value from big data is often the most difficult of the five V’s. After all, having thousands of lines of data is pointless unless you use them for something purposeful, such as measuring marketing efficacy or improving compliance programs.
However you choose to process big data, each application requires its own analysis and personnel to implement, manage and mine it for actionable items. With analysis of big data still being a new field, it can be hard to find talent — and harder still to develop an analytics program that truly hits at the heart of what your organization is trying to do.
The Impact on Financial Institutions
Some typical uses that financial institutions have found for big data include:
Leveraging big data to create profiles of account holder spending habits on the internet. With this knowledge, banks and credit unions can create targeted credit card marketing campaigns to reach this customer segment.
Banks and credit unions have used big data to assess the risk of consumers applying for credit — rendering a credit decision in minutes versus days.
Big data has helped financial institutions increase revenue through targeted cross-selling and promotions. Instead of hoping a mortgage promotion reaches someone who happens to be in the market for a home, banks and credit unions can know based on web and financial activity. In the old days, vast amounts of data dissipated throughout the enterprise, siloed into separate systems, servers, apps, etc., preventing it from being used across the enterprise. It was impossible, for example, for the mortgage department to use data that was hidden inside the savings and loan department. Using big data architecture to combine this data into one repository, where algorithms can be used to identify potential sales, has unearthed many cross-sell and other revenue-increasing opportunities.
Big data gives visibility on spending habits and online inquiries, which financial institutions are using to improve the customer experience and deepen account holder relationships.
Let’s also not underestimate the impact big data has on preventing fraud, ensuring regulatory compliance, and internal operations as well. If you’ve ever used your credit card in another city or state and instantly received an email, text or call from your financial institution, then you’ve experienced big data in action.
Fraud protection, as we all know, is a very big deal. It’s a $9 billion industry that affects more than 30 million Americans each year. Luckily, today’s technology can analyze hundreds of transactions, emails and purchases, and grade them for potential risk. This allows auditors and fraud detection departments to do their jobs faster and more accurately. As technology continues to evolve, we’re reaching the point of predicting fraud before it can occur.
It’s no secret that regulatory compliance is a big burden on financial institutions. Unfortunately, the smaller the bank or credit union, the bigger this burden can be. The power of big data, if correctly applied, can help financial institutions reduce regulatory compliance risks and avoid potential problems in real-time. The key ways it can do this are through building new compliance reports and performing regulatory stress tests. By aggregating data that is typically scattered across systems, servers, apps, lines of business, etc., it can be updated and sourced more quickly and easily. Just in regulatory compliance alone, a business case can be made for updating IT infrastructure to facilitate big data aggregation, analysis and use.
