For those hooked on both data and craft beer, like me, the 18% growth number announced today may come as a bit of a shock; it exceeds any of the previously published segment growth numbers for 2013. While previous Beer Industry Production Survey (BIPS) growth numbers in recent years have consistently exceeded other predictions, this year’s gap was particularly high. Why?

To begin, a word about BIPS. BIPS continues to be the gold standard for measuring the craft segment, with over 89% of the 2013 barrels being reported (rather than estimated). This response rate makes it closer to a population survey and inherently stronger than a sample. So what is BIPS capturing that other surveys are not?

The primary answer lies not with the regional craft brewers, but with the smallest craft players: brewpubs, microbreweries with tasting rooms, and microbreweries with limited, local distribution. By all accounts, these breweries both had an incredible year and are not captured by existing data streams.

How big a difference do these breweries make? Answer: a big difference.

One of the questions we ask through the BIPS survey is about distribution. (Note to brewers: a good reason to fill that part out is to improve blog posts like this!) In 2013, we had almost 1,400 breweries tell us their distribution pattern. Of those, 61% had a significant onsite distribution presence, either as a brewpub or micro with a tap room. Those breweries significantly exceed category growth, growing at a scorching 24% in 2013. In the sample that reported their distribution, these onsite sellers represented around 8% of volume (and given the high reporting rate amongst regionals, are probably a higher percentage of the total population). Not surprisingly, 8+% of volume growing at 24% tends to pull up a category average…

These types of sampling blind spots are not uncommon and often hard to correct. You may remember the famous (and incorrect) headline “Dewey Defeats Truman” from the 1948 Presidential election. One problem that led to that famous moment: polls at the time were based heavily on lists from phonebooks and vehicle registrations, and both phone and car ownership tilted Republican. It also didn’t help that polls stopped two weeks before the election. Replace vehicle registrations with scan data and you see the problem. Our own “15 defeats 18”. Pollsters today face a similar problem with young Americans who only have a cell phone. The only way to know for sure? Wait for the election (aka BIPS).

Other smaller factors also play a role. As a production survey, BIPS also includes American craft exports, which continue to rise, whereas scan data and other streams only look at the American marketplace.

None of this is meant to criticize existing data streams, but rather to make sure users are clear what they measure. For instance, I have a high degree of confidence that IRI’s supermarket number for 2013 is highly accurate – for supermarkets. Similarly, on-premise data from a firm like GuestSciences likely captures craft trends in large, national on-premise chains. However, anyone who has toured the tasting room scene in a craft beer hotspot in the past few years knows that a large part of the dynamism of our sector occurs outside standard scan channels. So the next time you buy a craft beer at a brewpub, tasting room or local beer bar, raise a toast both to your local onsite craft beer purveyor and the stressed out economists trying to measure them.