Guest Post: Big Data Is Getting Bigger! Are Retail Companies Ready?

This month’s McKinsey Quarterly carries an article, “Are You Ready for the Era of Big Data?”.  In contrasting two competitors, it states the following:

“The [One] competitor had made massive investments in its ability to collect, integrate, and analyze data from each store and every sales unit and had used this ability to run myriad real-world experiments.  At the same time, it had linked this information to suppliers’ databases, making it possible to adjust prices in real time, to reorder hot-selling items automatically, and to shift items from store to store easily.  By constantly testing, bundling, synthesizing, and making information instantly available across the organization—from the store floor to the CFO’s office—the rival company had become a different, far nimbler type of business.”

This week’s guest post explores what retailers need to do in order to take advantage of “big data”.  The author is Özgür Yazlalı, one of my former colleagues and a gentleman who has years of experience helping retailers significantly improve decisions that directly impact financial results through data-driven analysis.  I greatly appreciate Özgür’s contribution and look forward to more in the future.


As an internal consultant in a Fortune 500 retail company, one of the interesting transitions I have witnessed was the significant change in all dimensions of available data.  Day by day, not only more granular data but also more categories of data have become increasingly accessible to planning functions.

The retail store remains the closest link to the end customer of supply chains – the consumer.  Leveraging this connection requires collecting tremendous amounts of very detailed transaction data.  Consider the following examples:

– Customer:  “For this very random reason, I’ll rightfully return this product that I don’t remember from which store I bought”.
– Retailer:  “Ooops, we cannot see how much we charged you for this product, so we’ll credit the ticket price back to you.”

The emergence of social electronic media has multiplied the complexity and volume of interaction between consumers and retail stores.  Smart phones, social networks, image processing, and cloud computing have become reality in our everyday lives, further illustrating the relentless and ubiquitous nature of Moore’s Law.

The “big data” phenomenon is a great opportunity for the manager who knows how to utilize it.  The possibilities of localized assortment planning with reliable store-product category forecasts, localized pricing, and demand planning with social networks and web-based trends are now very real.  The limitation lies neither in the analytical capability nor the existence of reliable data, but rather in the legacy planning systems.

This phenomenon can also be a curse if all you know about data is limited to spreadsheet programs.  My humble observation is that there is now an ever-expanding gap between the rate of increase in the data and the spreadsheet capabilities of individual users.  Getting the big picture could now be a difficult task.  Without a better way to address this gap, you risk being limited to analyzing specific events and/or drawing conclusions based on a limited sample.

Given the increasing gap, the traditional way of building IT solutions based on business requirements documents and restricted interaction is no longer viable.  Big data requires cross-functional and data-capable analytical teams that operate as intermediary functions between business and IT organizations.  This is not a team simply put together from ex-business and ex-IT folks, but data scientists, optimization experts, experienced data and business analytics consultants that could unleash the capabilities of SQL and spreadsheets together.  Such teams not only facilitate the discussions between the two organizations but also enhance a tool design process that is more interactive with rapid innovation and prototyping.  This is critical because no one really knows a priori what assumptions and models will consistently work.

Most companies are organized as silos of functions, as are their data sets (though IT might store these datasets in the same server).  Thus, in addition to enhanced IT-business interaction, these analytical teams could work with different business functions that do not routinely communicate, leveraging data as common ground.  For example, the inventory management teams that use sales and inventory data may not know about, or may choose to ignore, trends in store traffic that consumer insights teams usually track.  Yet, an integrated analysis of these two data sets could reveal that a decreasing sales trend in a well-inventoried store is being driven by poor assortment, even if there is sufficient traffic.

Big data presents significant challenges and opportunities to businesses.  Cloud-based, functionally local solutions may help individual business functions maintain their own small data warehouses, but an holistic approach demands greater IT involvement.  Unhappily, IT organizations are not often seen as centers of innovation.  A team of skilled, experienced analysts with access to both the computational power owned by IT and the big data in which other business functions find themselves awash provides an effective means for overcoming the challenges and delivering on the opportunities of big data.  These analytical teams can be a part of the IT organization, but whatever organizational structure is employed, they must have powerful computing resources, access to all relevant data, and a voice with decision-makers. [Editorial Note:  A hyper-performance, secure, could-based platform would be the natural vehicle for data of all types from all sources where challenges and opportunities can be identified, diagnosed and the next best action determined and directed.]

In summary, big data is now more accessible, and companies must continuously explore new ways of using it to increase profitability and market share.  For retailers in particular, these opportunities are far greater than for any other industry because of both their proximity to consumers and the availability of structured data (social networks, CRM, POS, inventory, traffic, e-commerce).  Integrating all this valuable information into a predictive [Editor’s note:  and prescriptive] planning process is a difficult task requiring tighter linkage between the IT and business functions.  Analytical consulting teams with enhanced data-capabilities who could facilitate and guide this interaction are now more important than ever in achieving this goal of increased profitability and market share.


Thanks again for dropping by.  If you liked this week’s guest post, please rate the piece and feel free to leave a comment.  Until next week, remember the words of the American poet, Harry Kemp who said, “The poor man is not he who is without a cent, but he who is without a dream.”

Have a great weekend!


About Arnold Mark Wells
Industry, software, and consulting background. I help companies do the things about which I write. If you think it might make sense to explore one of these topics for your organization, I would be delighted to hear from you. I am currently employed by Incorta, but I am solely responsible for the content in Supply Chain Action.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: