Running out of stock is an expensive prospect for any retailer. Barring a suitable replacement, the retailer misses out on revenue. Even if there is a replacement, the customer might be frustrated, which could mean less customer loyalty and lower lifetime value. The pandemic made this even tougher for retailers, thanks to the combination of supply-chain disruptions, increased spending on consumer goods, and labor force shortages. Recent research shows that by May 2020, the frequency of stockouts within the United States had increased to more than 35% — more than double the pre-pandemic level of 14%.
How should retailers handle these increasingly frequent stockouts? Multiple experiments in the past year conducted by Instacart, where one of us is a data scientist, found that customers respond better if they’re warned there’s a chance an item is out of stock than if they find out after ordering. These experimental results suggest that, as with so many things, honesty is the best policy.
Traditional Responses to Stockouts
Typical strategies used to prevent and mitigate stockouts can be expensive. One common stockout prevention strategy is to increase the amount of inventory kept on-hand. However, among industries where products have an expected shelf life (e.g. the grocery industry), such an approach can lead to increased waste. In addition, increased storage capacity can be expensive, necessitating some combination of greater space and/or reduction in product selection. Most significantly, this strategy is unavailable to marketplaces which do not directly manage their own inventory.
Even retailers who have the resources necessary to invest in top-of-the-line inventory management technology are not immune to this problem. While improvements to inventory management and predictive analytics may ameliorate the issue, such investments cannot resolve the issue of stockouts entirely. This is due to the fact that inventory management systems cannot know with certainty what a store’s stock level will look like for a given item hours or days in advance. The problem is even worse for marketplaces like Instacart, because they are a step removed from the retailer’s inventory management system.
As an example, imagine that one day you are craving sushi for lunch. After scrolling a nearby store’s website, you decide to order the California rolls in advance so that they are ready for you to pick up as soon as you go on your lunch break. Unbeknownst to you, minutes after you placed your order, another customer who was already at the store picked up the last set of California rolls. Arriving at the store, even a full refund is unlikely to allay your disappointment. You have fallen victim to the fundamental problem of online retail: There is rarely a guarantee that an item visible to you at the time of order creation will actually be in stock by the time the order is fulfilled. The natural consequence of inventory data becoming outdated in the interlude between order creation and order fulfillment is that some customers won’t receive the item they ordered (i.e., a stockout).
Physical stores and online retailers experience this problem of stockouts differently, and have different strategies for prevention and mitigation. Customers shopping in person can reach out to employees who can then take action to minimize the loss of customer satisfaction (e.g. checking backstock or sharing when additional stock is scheduled to arrive).
While not all online retailers are able to act as concierges in the same manner as a store employee, they do control the virtual interface that the customer uses. As a result, online retailers and marketplace facilitators can deploy a host of strategies to mitigate stockouts, including proactively recommending replacements, or dynamically changing which items are displayed on storefront. For example, it’s standard for retailers to remove items from the storefront when the inventory management system reports that the balance-on-hand is zero. Going a step further, a retailer might downrank that same item within the search results if the inventory data shows a single unit remaining. The drawback to such an approach is that the retailer will invariably suffer from some amount of forgone sales.
This virtual interface between customers and a platform like Instacart can share critical information with customers when stocks are low. We propose that honesty is often the best policy to prevent stockouts. Specifically, our analysis suggests that a policy of sharing item availability information with customers leads to a 4.9% relative increase in order frequency and a 5.33% relative increase in revenue per customer over the longer term (more on that in a moment).
Being Honest with Customers
Our analysis is grounded in two field experiments conducted from October 2021 to April 2022. During this time (between the Delta and Omicron waves of Covid-19), there were pronounced nationwide inventory shortages as supply chains struggled to keep up with surging consumer demand. In response, Instacart initiated a series of experiments with the goal of preventing the stockouts that would normally arise in the wake of these supply shocks.
Prior to the pandemic, Instacart had presented search results using relatively simple logic: the product was either in stock (in which case Instacart displayed the item as purchasable to customers) or Instacart’s catalog coded the item as “unavailable,” whereupon Instacart removed the item from storefront. Seeking to protect customers from ordering out-of-stock items without constraining the selection of available products, the company adopted a more sophisticated policy of sharing more information about item availability.
Over the course of these field experiments Instacart appended a “Likely out of stock” label to an item’s description on its storefront if Instacart’s machine learning model estimated the probability of the item being in stock as falling below a certain threshold. Further experimental changes included relabeling and changing the functionality of the associated button that allowed customers to add that specific item to their order. Instead of a “+” sign, the button read “Request.” In addition to adding the item to the customer’s order, clicking the button took the customer to the replacement menu where customers were given the option of choosing a specific alternative, or disabling replacements for that item.
The goal of the first, one-month long field experiment was to understand the impact of this information-sharing policy on customer purchasing behavior and customer satisfaction. Over the course of this experiment, more than 850,000 customers were randomly assigned to treatment or control groups. The intent behind the second, six-month long field experiment was to understand the implications of sharing the item availability information on longer-term revenue.
It is worth noting that the outcomes of these experiments were not obvious in advance. The phenomenon of people placing higher value on items that appear to be scarce (scarcity effects) is well researched within the field of social psychology. Customers, fearful of future constrained supply, might purchase more units of the item than they would otherwise (hoarding). Similarly, customers may make the assumption that low item availability is indicative of high popularity, and revise their assessment of the product’s value. If customers assume that other customers’ patterns of behavior embody hidden information, and that it behooves them to act with the presumption of that hidden information, then the results are herding effects (customers’ influencing one another solely through purchasing behavior).
If either scarcity effects or herding effects are pervasive, then a policy of sharing low item availability information with customers could very well increase the incidence of stockouts. Recent research by Timm Teubnera and Antje Graul has confirmed the success some companies have enjoyed in employing these sorts of cognitive biases to improve conversion, e.g. notifying the customer that there is “Only one left in stock!” However, unless a company has extremely accurate inventory data and/or perceives relatively small losses in customer lifetime value in the face of stockouts, then actively spurring demand in the context of low item availability can be a risky proposition. Even if a stockout does not occur, there is still the risk that exploiting scarcity effects and/or herding effects can result in reduced customer trust. As Teubnera and Graul note, “many customers may increasingly perceive scarcity cues as misleading or outright scams.”
The Short-Term Impact
The most discernible effect of this new experience was that customers didn’t buy products that were running low: There was a 25% relative reduction in the proportion of items ordered that were likely to be out of stock. This result supports the hypothesis that customers’ preference for “reliability” outweighs any potential psychological biases incurred from scarcity effects and/or herding effects. In other words, people are more worried about getting what they ordered than about getting what everyone else is buying.
In addition to reducing the purchase of low availability items, Instacart observed a 3.29% decrease in the proportion of items replaced, and a 2.6% decrease in the proportion of items refunded. Replacements and refunds are natural outcomes of stockouts, so this finding is very intuitive.
As a result of the information sharing policy, we also found a 3.28% relative reduction in the proportion of replacements that were associated with a negative customer rating. We hypothesize that this result is due to the availability information changing the customer’s reference point. These “reference effects” were laid out decades ago in the postulates of prospect theory by psychologists Daniel Kahneman and Amos Tversky. Their theory posits that the loss an individual experiences when an item is taken away will tend to be felt more deeply relative to an equivalent scenario where the individual must forgo the gain of that same item. This psychological tendency is commonly referred to as loss aversion, and could very well be at play here. Customers typically expect to receive all of the items they order, and may perceive a failure to deliver any specific item as a significant loss. In contrast, labeling low-in-stock items with a “Likely out of stock” advisory may shift the customer’s reference point into the domain of prospective gains. Not gaining the item they would have ordered is less painful than not getting the item after committing to buying it.
The Longer-Term Impact
On the face of it, the aforementioned benefits to customer satisfaction could very well come at the cost of long-term customer engagement. A policy of disclosing item availability data might protect a customer from a bad experience, but simultaneously frustrate them, thus disincentivizing them from using the platform in the future. It is not immediately obvious whether customers will reward frank disclosure of item availability with increased trust and engagement, or reduce their use of the platform in favor of one where item availability is presented as more certain.
To examine this critical question, we analyze the impact of this information-sharing policy on longer-term revenue using data from our second, six-month long field experiment. Here we find a 5.33% relative increase in total revenue per customer over the course of the six-month period. Interestingly, the majority of this increase is driven by a 4.9% relative increase in order frequency (as opposed to an increase in mean order size). We propose that this beneficial outcome is the result of three factors.
- We theorize that sharing low item availability information helps customers form more realistic expectations about the risk of stockout (i.e. the aforementioned reference effects).
- By sharing low item availability information, we empower customers who place a premium on service reliability to switch from high-risk/low-availability items to low-risk/high-availability items when making their purchases.
- We propose that sharing potentially unflattering low item availability information with customers increases customer trust of the retailer, and consequently their engagement with the platform in the longer term.
Our analysis suggests that frankly disclosing which items are most likely to be out of stock can be a winning strategy for online retailers and marketplace facilitators that are suffering from stockouts, potentially improving both customer satisfaction and lifetime value. Doing so will leave customers more satisfied and increase trust, thereby increasing the frequency with which they order in the longer-term.
Source: Harvard Business Review
