If you’re in the C-Suite, the chances are good that you’ve absorbed at least a little information about business intelligence. You probably work with Big Data to some extent — but do you really understand how it works and what it’s capable of?
As a leader, you don’t need to be able to build a database yourself. You do need to understand the core concepts behind big data and business intelligence. Making strategic decisions based on an incomplete understanding of the basics leads to weak returns on your investments.
Though short enough to read over lunch, this guide will give you corporate-level insight into business intelligence. We’ll touch on the technical essentials you need to know, then pull back to explore what good business intelligence looks like, how to make it work for you, and how to troubleshoot if things go wrong.
The Big Data Details You Missed The First Time Around
Here's an eye-popping statistic: 2.5 quintillion bytes of data are produced every 24 hours.
In fact, the vast majority of all the information in the world was created in the last several years. Each of us makes a mountain of data just by going through a normal day — purchasing goods at the store, uploading photos to social media sites, looking up directions on our phones, using search engines, and tracking our fitness on athletic wristband apps.
This massive sea of information is called “big data". It’s the numbers behind everything we do, all day, every day.
Big Data Characteristics
Modern big data is unique because of its ever-increasing volume, velocity, variety and veracity.
Volume: The amount of stored and generated data.
Velocity: The speed at which data is created and processed.
Variety: The nature and type of data.
Veracity: The quality and accuracy of data, and the need for the same.
In other words, it's a flood of highly varied data that needs to be validated, stored in an accessible manner, and mined for insights in as close to real-time as possible — because there's always more being created.
Structured vs. Unstructured Data
Maintaining Big Data is a constant challenge. The datasets are so huge that traditional database programs can’t handle the volume.
More critically, most of it is unstructured. This is another term you’ve heard but might not be fully clear on because it’s by definition an “everything else” category.
What do we mean by "everything else"? Well, structured data is already in a specified format when it goes into a database. Everything is arranged into predefined fields that are funneled right into an organized database, stored in tables in a logical fashion. Banking information is a form of highly structured data.
The good thing about structured data is, it’s easy to use. All the data is neatly organized with a reliable structure. The bad thing is, it requires that the data has already been processed to some degree.
In a structured database (for example, SQL), you need some information before you can move data from Database A to Database B. You have to know the structure, and the data types must be consistent with what the receiving database expects to see. Otherwise, it creates errors. This is called "schema on write".
You can see the problem here. Big data has that volume and velocity we mentioned earlier. It has to go somewhere until it can be processed, and there just isn’t time to do that before more data pours in.
The data isn't able to be arranged into fields, either. Videos, music, Internet of Things (IoT) data, invoices, machine learning datasets, blog posts, and comments are examples of unstructured data stored in its original format.
Unstructured data frameworks like Hadoop use a method called "schema on read." You bring in the data, then use code to read the data without knowing the structure ahead of time.
Looking for ways to harness you big data? Request a free 1-hour consultation. Our experts can help you!
Checking Into The Data Warehouse
We’ve talked about the structure of data and how it’s stored. There’s another important concept when it comes to business intelligence. This one is as much management philosophy as it is technical innovation: the data warehouse.
every day, a company has thousands of workers entering data on a constant basis.
Whether they are salespeople, production workers, or management, each is entering records relevant to their department and filing them away against future use.
But what if the vice president in charge of the plant or division wants to pull reports using data from every department? Do they have to collect each department’s reports separately? That’s an inefficient, labor-intensive process that can create bottlenecks in the flow of information, limiting access for everyone.
And yet, it’s not unusual for executives (or their senior managers) to spend much of their time combining subordinate units’ data manually using Excel spreadsheets. 😅
In any company, data tends to collect in "silos." It gets collected by the department that creates it and stored in a way that other departments have a hard time accessing it. The passing of information between departments is done using reports which offer limited information on a sluggish timeframe.
A data warehouse breaks down silos by directing all information to a single data management system. Instead of collecting data locally and moving it in periodic reports, the data goes straight through the analytical funnel to one central location.
In other words, a data warehouse provides a “single source of truth” for companies. Everyone can see the same data (as long as they have the proper permissions) and can explore it however they need to in order to make decisions.
This kind of democratization of data is at the heart of business intelligence- and that’s our next stop.
Business Intelligence 101
What Is Business Intelligence?
Think of business intelligence as getting the correct information into the right people's hands in a way they can understand and apply on the job.
Good decisions depend on having the best available information to inform corporate strategy at all levels of business. Business intelligence (or BI) works to provide that information by collecting, curating, analyzing, and displaying a company’s data in a usable format.
Data visualization is a semi-recent trend in business intelligence. As you might guess from the name, it aims to present data in a visual format- graphs, 3D models, charts- which is easy for users to read at a glance.
This is more than the traditional zig-zag line on a sales chart seen in cartoons. Data visualization tools use living data so that users always find the most current information available. It’s a powerful way to spot trends, identify outliers in a batch of information, and find patterns that might otherwise be missed.
Dashboards are a popular visualization tool. They act somewhat like the instrument panel of a car, providing constantly updated information about the state of the business. Managers can drill down into a dashboard to get deeper information in categories like sales, operations and finance.
Why Use BI?
Business intelligence has a lot to offer enterprise. A few of the most-cited benefits include:
- Unique sales insights: Looking at data as a whole provides valuable information about sales trends, consumer preferences, and new business opportunities.
- Competitive advantage: Data-guided predictions give you the ability to shift strategy when it's most effective rather than taking a reactionary approach.
- Lower costs: Automating data management leads to a reduction in labor costs and lower training expenses. Both of these speak to the bottom line.
- Flexibility: BI isn't a "one-trick pony." It's a simple process to drill down into the data to find new interpretations and insights.
- Rapid, informed decision-making: Business intelligence produces practical data that helps leaders make decisions quicker. They make better choices in less time, freeing them up to focus on big-picture challenges.
Putting BI to work
The term "an ounce of prevention is worth a pound of cure" is rarely more true than it is with business intelligence. It's far less expensive to spend a little more time in the planning phase than to work around features that obviously don't work in a prototype system.
To that end, here are some “big picture” guidelines for implementing BI in your business:
- Identify the metrics you will use to monitor your business. What do you need to know? This can be as easy as making a list of your key performance indicators (KPI) and adding other criteria company leaders tend to keep track of.
- Avoid overload by focusing on core measurements. Limit the scope of your analytics at first. Remember, in the beginning, you’re also establishing good data governance and collection standards, so resist the urge to jump into the analytics deep end right away.
- Set specific goals for improving your company. Where do you want business intelligence to take you? What does success look like? Set a long-term goal or two, but focus on shorter-term targets that will be easier to evaluate.
- Outline parameters on what data users can manipulate and set permission levels. Data democratization doesn’t have to mean every employee can see all of the company’s data. For one thing, that could cause regulatory issues if you handle health or trade data. For another, it’s just bad data governance to let everyone edit everything. Work with all levels of operations to determine what permissions they need to function and thrive.
- Determine resource availability. Budget is a large consideration, but not the only one. Look at all angles of the project and predict where resources will be made available or used up. How many employees could be freed up by automating business intelligence tasks? Where could they be better used? Will the new program require outside training, and how does that fit into the project’s overall budget?
- Continue to iterate and improve the system on an ongoing basis. Business intelligence is a continual process. It improves as you gather feedback and make adjustments to bring processes in line with the specifics of your business flow. The more often you check-in, the faster you will iterate to a better system overall.
What To Do When Things Go Wrong
It’s not uncommon for executives to feel overwhelmed when setting up a BI program. They get excited to start, then slowly discover structural challenges that make it difficult to get data into the system to be analyzed in the first place.
The challenges fall into two categories: people issues and technology issues.
People issues generally stem from either a lack of training or a failure to buy into data as a transformative force for the enterprise. The team wasn't appropriately trained or didn't see why they should put in extra work to learn and use BI tools.
This leads to poor data governance, failure to set and follow consistent company standards, and reluctance to give up ownership of data (and the accompanying “prestige” of having control over it). Staff may even ignore their BI tools in favor of the older processes they’re already familiar with.
Technology issues are the result of outdated or insufficient systems. Maybe processes aren't capturing the necessary data correctly, or the data itself is low quality. Perhaps the software wasn't built using security best practices and had several dangerous vulnerabilities as a result. Maybe the business intelligence system is overly complicated and doesn't fit into an efficient workflow.
Whatever the cause, data isn’t flowing so much as it is oozing from place to place.
A savvy manager might look at those two categories and think:
“Those all look like people issues to me.”
That is absolutely correct.
Many times people think that issues with business intelligence are primarily technology-related. When you look at the human element behind these statements, you see that major technology problems are nearly always preceded by a people issue that went unaddressed. Leaders forgot to consider how people would use the data and data systems, which led to disaster.
Take the example we used earlier of software being too complicated. This reads as a failure to seek key stakeholder involvement during the development cycle. Solicit input from the people who will be actually using the business intelligence system (both at the input stage and the analytics one) and you’ll likely avoid making a system that’s more effort than it’s worth.
It's also important to recognize and remind stakeholders that:
BI implementations are never completely done.
They must be constantly tweaked, reevaluated, and managed to keep bearing fruit. Employees will be more willing to offer constructive criticism and tolerance of hiccups when they understand that the BI system is ever-evolving.
Business Intelligence Tool Kit
To give you a better idea of business intelligence software, we’re going to close out with a rundown of the most popular business intelligence tools. You can scan it for ideas- or just use it as a cheat sheet during planning meetings. (You can’t google when you’re screen sharing!)
Power BI is a suite of popular business analytics and visualization tools from Microsoft. It provides end-users in companies the power to build reports and dashboards without relying on the IT department or database administrators. Along with a desktop interface, the system provides a cloud-based service with data warehousing capabilities. It can handle data discovery and data preparation, as well as create highly interactive, responsive dashboards. Power BI is a direct descendant of a family of Microsoft Excel add-ins that included Power View, Power Query, Power Map and Power Pivot.
The main attraction of Microsoft Power BI is that your company's data is represented in colorful, detailed visuals and graphics that give a clear view of the status of multiple business functions. Visualizations update as new data is collected and analyzed. Armed with the latest, most accurate information, you have the power to make more effective and powerful decisions about your business. You also get consistent analysis across the entire company and can create and reuse powerful data models customized to your business.
Another advantage of Power BI is the ability to unite a wide variety of data sources and tools under one umbrella. Whether you're working with Excel spreadsheets, Hadoop data, email subscriptions, customer data from your CRM, Google Analytics or GitHub, Power BI centralizes it and makes the diverse data useful and actionable.
Pentaho Corporation is an Orlando-based software company that produces a group of open-source business intelligence products called Pentaho Business analytics. They include products for reporting, dashboard creation, data integration, data mining and online analytical processing (OLAP). There are two levels of Pentaho products —the open community edition and the commercial enterprise edition.
As an open source-based company, the company nurtures a strong and supportive community that contributes plug-ins and applications to the project. These developments are often integrated into the community edition, and then the company upgrades them and adds them to the commercial enterprise edition.
Pentaho is known for its flexibility and configurability. It is excellent for data integration, although creating high-level reports may demand the ability to query an SQL database. The software is very customizable with easy drag-and-drop and commented the company has timely customer support.
The term “self-service business intelligence tools” refers to software that allows users to create and manage dashboards, graphics, reports, and analyses without detailed knowledge of data mining, statistical analysis, or other Big Data concepts.
Because end users are not data experts, the software should be easy to set up and use. However, because self-service tools are so flexible, companies should adhere to preset guidelines for accessing secure information, data governance, and key metrics to avoid users “breaking” features by manipulating functions improperly.
Some examples of self-service BI tools:
Tableau's philosophy is that anyone should be able to create data reports and graphics without needing to be a programmer. Users can add animation, 3D graphics and cartography to make stunning visualizations quickly.
This is a cloud-based reporting and BI solution software that offers a variety of applications within its suite, including dashboards, analysis, reports, data warehousing and more. This software can be used in several departments, including sales, marketing, HR, finance and operations, within small and midsize organizations.
One of the more popular BI tools on the market, Qlik rose rapidly after going public in 2010. Their QlikView product is engineered to be easy to implement for end-users while offering lots of power for technical users. It can pull in data from a wide variety of sources, including SAP, Excel and Oracle databases.
Positioned to meet the needs of enterprise customers. Crimson Hexagon specializes in analytics for social media. Users can analyze social stream trends and use the machine learning capability of the software to find relevant conversations and threads. It has excellent interactive visualizations for hashtags and social media keywords.
Built as a dedicated platform specifically for business analysts, Alteryx blends data preparation and analytics from multiple sources. End users can run complicated predictive analytics and export the results to Excel, PDF, XML, Tableau or Qlik.
A Clearer, More Informed Future
Hopefully, this guide has shown you that you don’t need to be an engineer to implement a dynamic, productive business intelligence program in your company. Follow our program launch guidelines, don’t take shortcuts in planning, and (most importantly) always remember not to lose sight of the human element in business intelligence. Do that and you’ll be several large leaps down the road to success!