Using Science, or Being Used by Science.

A lot of people think that science is real "hard". It isn’t.

Proper science is really not much more difficult that picking up the dirty socks on your bedroom floor. If you start in one corner, and work methodically, in an organised sweeping manner across the entire floor, you can collect all the refuse of the last several months, and get it into the wash.

If you haphazardly, only pickup socks "at random", or only the ones that start to pile up over the door jam, that keep you from closing out the sound of people making rude and obtrusive noises (like getting up for breakfast), then you will have a bedroom that looks like mine, and you just won’t be doing what in the world of science, is called a "good job" of picking up those socks (shame on you).

What makes science "seem" difficult, is that within each field, all the people who work there have to agree on the words they are going to use, and what they mean. Sometimes the same thing gets called different things in different scientific disciplines: If you are "floating" between disciplines that means you have to learn a lot of words, in order to sometimes get out very little information from someone else’s study. The actual "doing" of science is not that hard. It just requires attention to detail, some clean methods, and honesty. You make "experiments".

If you believe that the brewing industry’s research, is not always generating information that is relevant to making nice, fresh beer, then you may want to do some "experiments" of your own, and find out what really does makes a difference to you.

If you are completely satisfied with the research which is funded by, and produced for the industry, then I wouldn’t read another word.

Deciding such, I wouldn’t brew another drop, either, since you’ll never keep up with the industry’s continual advances towards inexpensive, quickly produced, long lasting products... unless of course you stay one step ahead of them and begin producing hop flavoured water (but you may in time, decide to eliminate those hops, as they can cause real stability problems).

Should you be one of those perverse souls like myself, that really thinks it’s fun and worthwhile to figure things out on your own, I thought it would be worthwhile to point out some of the "pitfalls" of doing experiments, and ways to make sure that you are really figuring out what you think you are, if you set off doing your own experiments.

I’m using the "real terms" here. These are some fairly simple concepts, but you sort of have to keep them in mind all the time, or you can trip yourself up. This requires paying attention, and craves a bit more diligence than you might think. Real, live, big-time scientists, submit papers all the time, with some very broad ranging implications, that after a few "editorial revisions", turn out not to show much of anything.

They make some basic mistakes in assumption, or association.

Another oft repeated boo-boo, is the continual quoting of sources, that in turn quote other sources, that when you follow all the way back to the very first author, you find it really doesn't warrant the mileage that people have got out of it. This kind of junk can continue for decades, with more and more complicated revisions of the "theoretical model" to sort of fix up the discrepancies between what was found, and what was expected. Finally, someone looks back at the original source, and finds that they were either not implying what other folks have read into it, or the experiment was either poorly designed or not repeatable. Isn't that kind of embarrassing?

Don't imagine that it's uncommon.

Industrial research is most famous for making some pretty hazy interpretations when it comes to judging "quality". They are usually quite good when it comes to figuring out cheaper manufacturing methods, but when they start drawing conclusions about what is "better".... I’d step back and take a look (Do you really believe that a certain brand of laundry detergent gives you a wash that is a certain percent "brighter and whiter" than another?).

They can have made one or several of the following mistakes – either mistakenly, or otherwise (heaven forbid).

 HOW TO DRAW WRONG CONCLUSIONS

 "VARIABLES":

This is a term used quite a lot, and sometimes even relevantly. It means simply "anything that can be different". If you are trying to figure out something, it should only be "one" thing that is different between the two things that you are comparing. With the vagaries of homebrewing (it’s hard to do things EXACTLY the same way twice), I’ve found this to mean sort of "the later in the process the better".

It would not be so hard to find out what method of capping your bottles is better, since it’s pretty easy to make sure the same things happen to your beer both before, and after that. If you wanted to find out if the barley growing on the north, or south side of your house makes better beer.... you’ve set yourself a pretty tall order. You have to make sure that they are separately but equally, germinated, dried, roasted, crushed, mashed..... well... you get the picture.

Another way of looking at it, is if you are comparing something early in the process, you’ll have to do it over and over again, before you can be pretty sure that it is not some other "variable" that is making any difference you might find.

The later in the process you go, the easier it is to keep things under control, and the more you can learn with a "single shot" experiment.

For me, in practical terms, this pretty much means "after the boil". It’s not too hard to split up your ferments, and keep things under pretty reasonable control.

If you have the capability of running two boils at the same time, you could split up your collected sparge, and that just puts you further back in the process.... and good luck!

If you have two exact same mashing systems with extremely delicate temperature controls that you can run at the same time.... can I come to your house and play?

"CONFOUNDERS"

This is the word used when two "variables" are "tied" to each other. Sounds funny, but is really quite simple. Let’s take this statement: "English Ales deteriorate from oxidation more quickly than Continental Lagers, because they have higher amounts of oxygen later in the primary fermentation".

Now if that was what you want to figure out, you could go around and monitor the oxygen levels during fermentation at all the English Ale, and Continental Lager Breweries, and then make some measure of "oxidation" in the finished products, and what would you come up with?

Nothing.

You have made the mistake of having your diligent work "confounded".

You have English 2 row barley vs. Continental 6 row, different germination and roasting techniques, English hop varieties vs. Continental, infused/ decoct, top fermenters/bottom, etc., etc. These are all things that are "tied" to the two different processes and "confound" your results. You can’t be sure that it is not THEM that has caused any possible differences in "oxidised states" you might have discovered, rather than the thing you wasted your time going around measuring.

Another, closer to hand example, would be when I tried to measure the effects of cooling on final taste. It was stated on a few internet forums (by one Jack Schmidling), that he had stopped using his cooler, and it made no difference at all. I thought that sounded interesting enough to look into.

I had also become a bit religious about my cooling habits, and have thought ( and advised others) that the initial cooling phase might be very important. I was thinking here about the myriad of things that might precipitate out during this break, and later have an affect on flavour.

Now, I am pretty generous with "aroma hops" at the end of my boil, and foolishly, I continued in my usual manner with this ‘spurment.

A beer that has been allowed to cool overnight, will just not have the volatile hop components of one that is force cooled. I wasn’t interested in hop flavours, but rather whether the "cold break" that happens in rapid cooling effects flavour, or is just a visual thing.........

Hop aroma has "confounded" my experiment. (There may be a way around this, can you guess what that might be?.... ah well, we often think of these things too late).

"SPURIOUS CORRELATION OR CAUSALITY"

When you want to figure something out, it usually ends up as an "A causes B" kind of conclusion. If both things move at the same time (let’s take, say: fermentation temperature and speed of fermentation). Then you say that they are "correlated". And in the case mentioned here, it’s not too hard to stretch ourselves to a statement about "causality". OK, we’ll be bold here.... "Higher temperatures CAUSES faster fermentation.".

Now that didn’t hurt much, huh?

The problem here, and this is one of the most commonly made mistakes, is that "causality" is not that easy to establish.

Here’s a classic. There is a "correlation" between higher education and income. This is real. This is measurable. People who have been to University make on the average more money than those who haven’t. Now watch out. Here comes the twister. "Getting a University education causes people to earn more money". Sounds pretty straightforward based on the facts given. Now THAT is a "spurious causality". There are two cross sections of people that are grossly over- represented at institutes of higher learning.... smart people and rich people. We know nothing about what these groups of people would have earned if they didn’t continue their education, but a good guess would be that it was still "more than the average".

Assumptions about causality can be pretty tricky to spot. Sometimes the error runs even deeper, and the mistake was made in the correlation.

My age is quite a measurable phenomenon, and so is the availability of condoms.

They both seem to be on the rise.

Now these are two things that are moving in the same direction, but are NOT correlated.

As to assumptions about causality – that my increasing promiscuity with age is causing condom manufacturers to make their wares available wherever I might be-- – this has been a grave marketing strategy error on their part.

I promise.

"BIASED SAMPLING"

If you wanted to measure something in your beer, you might have noticed before, that the absolute first glass that you pull off your keg is "not quite like" the rest of them. If it is not filled with yeast and looks like a chocolate milkshake, there is at least a higher concentration of some flavour components right over the yeast cake.

Now, if you wanted to make a statement about "the whole keg", based on the first glass, that would be a "biased sample".

When you are measuring stuff, it’s not to hard to get a "representative sample" of the goods, and make a pretty good guess that it pretty much tells you what is going on with the rest of the stuff, too.

If you are going to make some judgement about "taste" or "quality" then the problem gets a bit trickier. One can arrange a tasting in the best faith, but whose tastes do they represent?

There is no such thing as a "non-biased sample" when you are selecting a cross section of people.

You might as well get used to the idea, and decide (and declare, if you are passing the information on), just what sort of sample bias you’re going to have among your tasters.

Some folks think that the only valid analysis of taste can be done by trained professionals within the industry. Horse puckey. That is only shifting the bias of the sample in a direction where we already know the previous results of their judgement. As an example, I had an acquaintance, who worked as a professional taster for a "Mega Brewery", who also happened to be an avid home brewer, and particularly liked producing English style Ales (his employer produced lagers). He used to complain "Every time they give us something where I start to think "Hey, this has a bit of character.", everyone else says "This is too ...‘something’."

He quit.

I think more appropriately, the bias of your sampling should be fitted to the bias of your prospective audience. If you’re only interested in what you think of your beer... blind tasting by yourself is great. If you want to pass on your information to other homebrewers (or are interested in what they think), then other knowledgeable homebrewers should be a perfect tasting group (it might be nice to try and describe their general background and experience.... made one batch of John Bull?). If you think you want to market something for the general public, I’d look at the industry’s professional tasters.... that’s what they’re there for.

MEASURING STUFF

I don’t think I need to rattle on about measuring if you’re chasing effects on pH, attenuation, or anything else that you can measure with "things".

If you are wishing to measure effects on taste, you have a whole new problem in front of you.

If you arranged an open tasting (where the tasters knew what they were having) with Siddhartha, Lao tze, Jesus Christ, and Mohammed, and perhaps had Mahatma Ghandi as the server, then I think you could get some "honest" results. You and I just can’t do that. The human is a peculiar animal, with an incredible capacity for learning and storing away information. These huge volumes we stuff in our heads, are stored by repeating familiar patterns. A new bit comes in, and we shove it down a worn path to a storage closet we perceive as similar.

The problem with this is, we tend to create huge circular loops of logic in order to not upset the order we have already established. Truly changing an opinion, means a whole building of closets come tumbling down and have to get rebuilt....we don’t like that.

This behaviour stretches right out over our taste perception. We "taste" (within reason) pretty much what we think we should. Sellers of beers are well aware that it is more important to have a customer get a positive association to a "label", than if the stuff behind it remains exactly the same through time. If you have once decided that something is "good", it will continue to taste "good" to you.

This creates quite a problem when trying to judge beers. The only solution, I think, is to taste "blind".

I have a pretty rabid attitude when it comes to adhering to this. If you aren’t tasting "blind", you may well just be kidding yourself (and anyone else that you pass the information on to).

There’s some junk written about how to set up blind tastings if you punch this link.

I’ve set up quite a few "blinders", and they generally are fun, but a bit of a pain in the backside as well. You don’t want to make it TOO serious, but one dominant personality can influence a lot of people by simply grunting or grimacing, so you have to sort of keep things in order.

Someone named Louis K. Bonham recently introduced me to a (for me) new concept called "triangle testing". It takes a lot of the trouble out of keeping the test "clean", and gives you more information as well.

THE FINAL PITCH

I’ve learned an awful lot about brewing by trying to do some ‘spurments. Just setting them up makes you think through the process and decide why you do the things you do. In a home brewing situation, everyone does things a bit different. In your particular set-up, you might find that the advice offered by others doesn’t give the flavours that you are interested in. You might also find that the information available in literature, could be theoretical applications drawn from the needs of an industrial product, where no-one has really "tested" it within the framework of a cellar fresh beer.

We need people to do that.

I hope you’ll be one.

This is important stuff.....

.......why, if enough people start trying to do some rigid testing on what parts of the process actually make beers taste better...... well, then.....I may no longer be considered an eccentric.... the fact that I keep my long term, elastic band deficient underwear up, by three revolutions of black electricians tape around my waste has nothing to do with it.

Unless otherwise specified, all material is copyright © 1995-2002 Bodensatz Brewing