The 2014 craft brewing growth numbers were released today. I’m a big believer in data transparency, and I know that a lot of Brewers Association brewery members (and planning members) rely on these numbers to benchmark performance, apply for loans, and a variety of other statistical purposes. For those reasons, I wanted to write a short post breaking down how we compile our Beer Industry Production Survey (BIPS) numbers and why I think they are the best data source available on the craft brewing industry.

The vast majority of our data is self-reported. 79% of the volume we reported came directly from breweries (both members and non-members, we send a green BIPS form to everyone). Now normally, you’d be worried about various forms of “self-report bias” (i.e. people tend to lie on surveys, in predictable ways). That fear is mitigated here, however, since brewers tend to know their taxable production very well (thanks tax authorities!) and the data can be easily checked against state sources in a variety of states (I know – I do it, and they typically line up perfectly).

That leaves the other 21% of volume coming from Brewers Association estimates. More than half of that estimated volume comes from a handful of regional craft breweries that can be estimated in a very tight range based on industry reports, scan data, annual reports, and press releases. That leaves less than 10% of the craft industry remaining to be estimated. I’ve been asked a couple of times recently how we come up with those estimations. The answer: it depends. If you’re in one of the states that publishes brewery production data (or will give it out in response to a FOIA request) we use that. If you’re not in one of those states, we use a variety of factors including:

1. Previously reported production

2. Estimates of capacity and average usage

Note: if I’ve ever visited your brewery, I’ve asked about your fermenters, because total fermenter capacity * 52/average turns (in weeks; which we measure in our benchmarking survey) * average production/capacity (asked about in BIPS) is a pretty darn good estimate of production. When “guess the fermenter size” becomes a competitive sport, I’ll be a top contender.

Note 2: luckily for me, brewers really love posting pictures of their new fermenters on Facebook.

3. Media reports + a healthy dose of skepticism (at the beginning of the year, people tend to be overly optimistic, later in the year, more accurate). Media reports are also used to shift that usage rate. If I read over and over that a brewery is running at full capacity, I’ll use a higher production/capacity ratio than the industry average.

I believe that these estimates are accurate, particularly in the aggregate. When in doubt, I often check them with your local Guild Director. The numbers also suggest we’re getting our aggregated estimates very close.

This year, we compiled a growth rate of 18.2% for breweries that answered our survey. For estimations, we averaged 17.8% growth. Is that gap worrisome? Not really, for two main reasons:

1. Breweries that don’t answer may be different than breweries that do answer (more on this in a second). I’ve definitely noticed that many of the most eager respondents to BIPS are breweries that had GREAT years. In contrast, breweries that struggled a bit might be less likely to answer, so a slightly lower rate makes sense (this is called a self-selection bias).

2. You could get a difference that large due to random chance. While I won’t bore you with weighted standard deviations, growth rates of craft brewers are all over the map. In an industry with growth rates in the triple digits aren’t that rare, it’s pretty likely you’d get some variation between samples, even relatively large ones like we get in BIPS.

To test this further, I even look at the estimates in their sub-parts. So for instance, micros who reported grew 3.8% faster than micros we estimated (that difference is again in the direction you’d expect if the groups were different, but is also not statistically significant). I also look at the “estimates” that come from states that publish their production data. These states do show a bit of the potential bias described in point 1 above (that breweries who don’t answer grow a bit slower on average than breweries that do), but it’s a pretty small and statistically insignificant difference in most states.

So to summarize:

• Most of BIPS (about 4 out of 5 bbls) comes from breweries reporting their production
• Those reports appear to be very accurate
• Everything else is estimated (by individual brewery)
• I have a high degree of confidence in the aggregation of those estimates

Finally, and most importantly, thanks to the ~2,000 breweries that took a few minutes out of your day to answer the survey and make it the best in the business.