A lot of classic recipes call for buttermilk, which isn't necessarily something you keep on hand every day in your refrigerator. Most of these suggest sour milk as a substitute, which is great for those times when you forget to use up that last cup of milk before you go away on vacation. They'll both work fine in your favorite recipes, though some things really only taste right with buttermilk.
They Need to Be Acidic
The main reason recipes call for buttermilk or sour milk is the acidity. Baking soda is a powerful leavener, and it'll give you light, fluffy muffins or biscuits, but it needs an acidic ingredient to make it work. Baking soda is alkaline, and when it combines with an acid in your dough or batter, it reacts and forms fine bubbles of carbon dioxide. In the heat of the oven those fine bubbles expand, and lift your baked goods. In a biscuit or other pastry, they'll lift and separate the flaky layers, and in a cake or muffin, those bubbles become the light and delicate crumb you're looking for in the finished product. If your recipe doesn't have enough acidity, it won't rise properly. Even worse, you'll be left with baking soda's peculiar, soapy flavor in your baked goods. That's why some recipes call for baking powder, instead: It has its own acidic ingredient built in, so you don't need any in your recipe.
They're Living Things
Both buttermilk and sour milk are living ecosystems, because they get their character from healthy bacteria. In the case of buttermilk, dairies use a consistent, specialized set of microorganisms, much the same as for sour cream or yogurt. Those give the buttermilk its acidity, as well as its pleasant, almost cheesy-tasting tang. Milk that goes sour naturally also gets its character from bacteria, but in this case they're not carefully selected, consistent strains. They're whatever bacteria happen to be in the area where you live, and they're going to give a pretty variable flavor to your milk. You could think of it as the difference between buying yeast or growing a sourdough: One gives you a consistent result, the other brings in an element of randomness and sometimes a stronger, unpredictable taste. Either one will work perfectly well in your baking, but some recipes do require that predictable buttermilk flavor in the end result.
They're Low in Fat
If you're trying to keep your baked goods as light and healthy as possible, you'll be happy to know that both buttermilk and sour milk are low in fat. Traditionally, buttermilk was the substance left over after butter was made – and by definition, after the fat was removed – so modern dairies duplicate that by starting with low-fat or non-fat milk. If you're working with soured plain milk, it'll have as much or as little fat as the milk you ordinarily buy. If you're concerned about your food's fat content, it's probably low-fat or non-fat already. One perk to keeping buttermilk in your fridge is that you can use it in soups or sauces to reduce the amount of high-fat cheese called for in the recipe. It adds much the same flavor with fewer calories and less fat, so it's a great substitution.
They're Easy to Duplicate
If you don't bake often enough to keep buttermilk on hand all the time, and you use up your milk quickly enough that it doesn't spoil in the fridge, you can still have sour milk to bake with anytime you need it. All you have to do is warm your milk slightly, and stir in some acidity in the form of a spoonful of lemon juice or vinegar. In a few moments your milk will thicken and curdle, and you're ready to bake. This quick-soured milk won't have all the flavor you'd get with buttermilk or naturally soured milk, but you can fake it. A good wine or cider vinegar has lots of flavor in its own right, and that will carry over into your baked goods. You can also keep dry buttermilk flour in your pantry, and add a spoonful or two to your dry ingredients for flavor. You can even use non-dairy milk, if that's how you roll. Your best bet for baking is soy milk, because it's got a high protein level that's comparable to dairy milk, and it curdles in much the same way.