Probability

Overview

Probability is the likelihood that a particular event will occur. Probability is used to estimate the chances of many different types of events happening. Insurance companies use probability to estimate how likely a particular driver is to cause an accident during the next year. Engineers use probability to predict how often critical pieces of equipment, such as jet engines on passenger planes, will fail. Gamblers in casinos routinely make wagers based on their understanding of the laws of probability, while investors make even riskier gambles on the rise and fall of the stock market or the price of a bushel of corn. Although probability is one of the most commonly used forms of mathematics in everyday life, many misconceptions exist about its formulation, meaning, and impact.

Fundamental Mathematical Concepts and Terms

Probability calculations are generally straightforward, though as the number of possible outcomes grows, the math required can become somewhat involved. Consider a simple example involving a single die (the singular form of dice), in which we wish to determine the probability of rolling a 4. The calculation for probability includes several elements. Outcomes are all the possible results we could achieve; since the die has 6 sides (1, 2, 3, 4, 5, and 6), and any of the six could land on top, the total number of possible outcomes in this experiment is 6.

Next, we must determine the total number of ways in which the event of interest could possibly occur; in this case, a roll of 4 can occur only one way. By dividing this value (the number of ways our desired outcome can possibly occur) by the total number of possible outcomes, we can determine the probability of the 4 being rolled, creating this equation: Probability = Desired outcome / Total Outcomes, or in numerical terms, P = 1/6 = 1/6. Thus we conclude that the probability of rolling a 4 on a single toss of the die is 1/6, or 1 in 6. We could perform the same calculation for each of the other values on the die, demonstrating that for each side of the die, the probability is also 1 in 6.

Interpreting this value is relatively straightforward: a probability of 1 in 6 tells us that if we roll the dice a large number of times, we will, on average, roll a single 4 for each six tosses of the die. If we wish to find out about how many 4s we will roll in 600 rolls of the die, we multiply the probability by the number of rolls, which are often called experiments or trials; in this case, we use the following equation: 1/6 × 600 = 100. This result tells us that over the course of 600 rolls, about 100 will be 4's.

This same procedure can be scaled up to evaluate events with thousands or millions of possible outcomes. If the names of each person living in the U.S. were written on slips of paper and one slip was randomly drawn, what chance would John Smith of Cloverleaf, Iowa, have of being drawn? In this example, only one John Smith exists in Cloverleaf, providing only one possible way to reach the desired outcome. The total number of people in the U.S. in 2005 was approximately 300,000,000, which is the total number of possible outcomes. John Smith's chance of having his name drawn is 1 in 300,000,000. If the drawing were held using the earth's entire population of 6,400,000,000, John's chance of being drawn would drop by a factor of 20.

A Brief History of Discovery and Development

While the very first game of chance cannot be specifically identified, historians are certain that these probability contests have been enjoyed for millennia. Ancient civilizations left behind small dice-shaped pieces of bone called astragalia, which apparently facilitated the earliest contests similar to modern dice games. Throughout the early history of man, gambling remained popular, with little apparent attention paid to the laws of nature and mathematics, which made the toss of the colored stones or polished bone fragments so maddeningly unpredictable.

During the sixteenth century, Gerolamo Cardano (1501–1576), a scholar of medicine, astrology, and philosophy made the first known attempt to explain the function of chance in gambling and other endeavors. Cardano was the first to deduce that an event's probability of occurring is determined by dividing the number of ways the event could occur by the total number of possible outcomes. Cardano explained that a roll of a single die has six possible outcomes, while a pair of dice can land thirty-six different ways; he also wrote about the statistical logic of a primitive ancestor of the modern game of poker. Unfortunately, Cardano's science was somewhat limited by his intense belief in astrology, which he used to predict future events of human lives. Perhaps his most successful prediction was naming the date of his own death far in advance; when the predicted date of his death arrived, Cardano insured his own correctness…by committing suicide.

A century later, mathematician Blaise Pascal (1623–1662) was asked why the odds of throwing a single six in four throws of one die do not equal the odds of throwing four sixes in twenty-four throws of two dice. Pascal accepted this challenge, then went on to devise the theory of probability as it is currently understood, in many cases applying these principles to the popular pastime of gambling. At age nineteen, Pascal also constructed the first mechanical adding machine.

While various other mathematicians added to the body of knowledge regarding chance and gambling in the decades that followed, the next major advance occurred in 1928, when John von Neumann put forward the basic concepts of game theory in a paper analyzing the probabilities associated with various poker hands. While game theory has found application in fields such as economics, its application to games of chance also continues, particularly given the advent of powerful, inexpensive computers.

Real-life Applications

SECURITY

The ability to conceal data from outsiders has been valued by military commanders for centuries; historians have uncovered evidence of military codes dating back more than 4,000 years. The Allied victory in World War II was hastened significantly when the Allies broke a presumably unbreakable code used by the Japanese, thus becoming privy to numerous confidential communications. Today, numerous applications for encoding and decoding data exist, most of them based on the fundamental principles of probability.

One critical use for this technology is data encryption, a technique for encoding data so that it is unreadable without a specific number, or key, which allows an authorized user to decrypt and read the message. Data encryption has become a critical technique as electronic transfers of sensitive financial data have become more routine. Many commercial websites now transfer buyers to a secure site, at which data such as credit card numbers is encrypted before it is transmitted from a user's computer.

Encryption works because of the laws of probability. An encrypted message can be read by any person with the proper numerical key, meaning that for a message to remain secure, the key must be virtually unguessable. Ever faster computers have made it possible for simple encryption schemes to be broken using a brute-force approach, in which the computer simply tries key after key until the proper one is located. Preventing this type of attack requires a large enough number of possible keys that the likelihood of guessing the proper key by chance becomes so small that it is not worth attempting. An encryption key's resistance to brute force attacks is measured as strength, with a more secure key being described as stronger encryption.

As of 2005, one of the most widely used encryption schemes is found in Microsoft Internet Explorer, where it encrypts data sent from computer users to the Internet. This encryption scheme uses a 128-bit encryption key, meaning that in order to read the encoded data, an inter-loper would have to correctly guess a 128-bit number. Since this length of key would theoretically take a modern supercomputer several hundred years to crack, 128-bit encryption is considered adequate for routine applications such as online shopping. In cases where additional security is desired, such as military applications, longer keys significantly increase the number of possible keys, producing a commensurate reduction in the odds of randomly guessing the key.

A related use for encryption techniques has recently appeared in the rapidly growing field of forensic computing. In the course of criminal investigations, law enforcement personnel frequently need to locate computer files related to a crime, a process much like finding the proverbial needle in a haystack. A typical computer hard drive contains hundreds of thousands of files, most of which arrive as part of the operating system or are installed with user applications; a basic installation of Microsoft Windows XP places between 10,000 and 30,000 separate files on a computer hard drive.

Unlike a computer user who knows where most of his important files are saved, a police investigator searching a computer for files with evidentiary value has no idea what the needed files are called, or in which directories they reside. Since it is impractical to manually open and read every file on the computer, encryption methods now allow investigators to automatically eliminate more than 90% of the files on a computer, permitting the investigator to focus on the remaining files.

This file-sorting system is based on the principle of encryption, in which any file can be processed to produce a unique identifying code. By creating these unique codes, or file signatures, of all the files installed by most operating systems and commercial applications, investigators have created a massive reference library for law enforcement purposes. Investigators can use this library to scan a suspect's hard drive, automatically eliminating any files which match the signature keys of known files while leaving the files which might have evidentiary value behind. The system works only because the number of potential file signatures is enormous; in the case of the MD5 algorithm, the total possible number of unique file signatures is 10³⁸, or a one with 38 zeros after it, making the odds of two files having the same file signature almost an impossibility. By reducing the number of files to be examined, this library enables investigators to more rapidly and more efficiently search hard drives, gathering evidence they might otherwise overlook.

GAMBLING AND PROBABILITY MYTHS

While the ancestors of today's dice games predate recorded history, the modern game of craps is far more recent, and is attributed to twelfth century Crusaders besieging a castle in Arabia. Most of today's other casino games also can be dated back to the Middle Ages, however one type of wagering can rightfully trace its lineage back more than twenty centuries. The longest-running wagering event practiced today is the ancient sport of horse racing.

Numerous archaeological finds support horse racing's claim as the most ancient form of gambling. A Hittite document dating to about 1500 B.C. describes in detail the process of breeding and training horses for the purpose of racing, while the Iliad provides a complete account of a chariot race. The Olympic games in 624 B.C. included specific rules for horse racing in contests of various distances, and the Romans soon added the concept of handicapping, or betting against the house. While the popularity of horse racing has risen and fallen over the centuries, today's racing, while faster and more refined, is virtually unchanged from the ancient contests held in Europe. While the advent of modern statistical analysis and computer equipment has provided the tools to analyze the mountains of statistical data available on past races, the ability to correctly predict the outcome of a horse remains an elusive goal.

While the interpretation of probability projections is fairly straightforward when applied to events which occur many times, the laws of probability become far less intuitive over short periods of time. One common probability myth, often cited by gamblers, is that numbers, horses, or players can become due, meaning that since they have not won in many plays of the game, they are now more likely to occur. This faulty line of reasoning is based on the understanding that over many thousands of plays, each number will appear a set number of times, hence the gambler assumes that the longer a value goes without appearing, the more likely it is to appear soon. Unfortunately, this belief is unfounded. In the case of completely unrelated events, such as the spin of a roulette wheel, the odds of the next spin are unchanged by the result of any previous spins. If the number 14 has not been spun on a particular wheel for six weeks, the odds of it appearing on the next spin are still exactly the same as they were before. The laws of probability do not provide for events to occur simply because they have not occurred previously.

A second probability myth, ironically, is the exact opposite perspective of the previous view. This perspective says that particular numbers can become "hot," or more likely to be spun. In adopting this philosophy, an observant gambler might notice that the number 27 had been spun on the wheel several times over the course of a short wagering session. The gambler, acting on the theory that numbers can become hot, now concludes that the number's frequent appearance in past spins makes it more likely to appear in a future spin, and he will wager heavily on this particular number. Once again, the laws of probability and chance dictate that, assuming the roulette wheel is functioning correctly, the chance of a future spin cannot be predicted by how often a particular number has appeared in recent spins. Regardless of how hot a number appears, it is no more likely to appear on the next spin of the wheel than any other value. Ironically, the theory of hot numbers, which says that the same number will come up many times together, is the exact opposite of the theory of coming due, which says that a number will appear when it has not been spun for some time. While gamblers subscribe to both philosophies (and back up their philosophies with their wallets), both theories cannot simultaneously be right; in truth, probability theory says that neither theory is correct, and that past events do not impact future spins of the wheel.

PROBABILITY IN SPORTS AND ENTERTAINMENT

Many sports rely on probability to predict future events. Baseball is among the most statistically-oriented sports, with numbers available for almost every aspect of the game. A player's batting average is a measure of the percentage of times he hits safely, expressed as a 3-place decimal value such as .333. While this value allows an assessment of a player's past performance, it is also useful in predicting his future effectiveness. For instance, a player batting .200, which can also be expressed 2:10, 1:5, or 20% can be predicted to hit safely 20% of his times at bat, or 1 time in 5 attempts. For this batter, the odds against him hitting safely on any given trip to the plate will be 4:1. Baseball batting averages are calculated using an involved set of rules, meaning that a player batting .200 will generally make it to first base safely more than 20% of the time; for this reason, some managers prefer to use a player's on-base average, which includes walks and errors in the player's success ratio.

Bowling is a popular sport in which players actually receive two chances to succeed, in the form of two shots (if necessary) to knock down all ten pins. Statisticians have used mountains of data from previous bowling competitions to calculate the odds of a professional bowler making a variety of shots. For example, when a professional bowler steps up to roll his first ball in a frame, the objective is to knock down all ten pins, scoring a strike. For the second shot, the odds of clearing the lane depend on which pins remain standing; three pins standing close together have much higher odds of falling than two widely separated pins. For most bowlers, a split is one of the hardest shots in the game, requiring the player to slice the ball to the outside of one pin, knocking it across the lane to hit the other one. Even for a professional, splits are long-shots. According to the Professional Bowlers Association, the 7-10 split, in which two pins remain at opposite sides of the lane, has been attempted 400 times in televised matches. In all these attempts, the professionals have managed to convert only three, putting the odds of a professional making this shot at 3:400, or about 1 time in 133 attempts.

How often do miracles occur? The term miracle has several different meanings; in theological language, it refers to an act of God that defies the laws of nature, though its most common use today refers to any seemingly impossible event that actually occurs. When the Boston Red Sox finally broke the decades-long curse of the Bambino and won the World Series in 2004, fans proclaimed the victory a miracle. When a jet airliner crashes and one or two passengers walk away without injury, many label their survival a miracle. And in a handful of cases where a single individual has won a state lottery, not once but twice, writers routinely throw out the term to describe this odds-defying run of luck. Ironically, the term is applied almost exclusively to positive events like those described, ignoring equally improbable turns of probability which lead to unexpected death or injury.

While no statistical definition of miracle exists, an estimate can be made based on common language. To most people, the expression "one in a million" describes something quite rare, though still achievable. People frequently use this expression to describe a job they truly love or a dear family member or friend, suggesting that this level of probability does not rise to the level of miracle status. For this discussion, we must conclude that a miracle is much rarer than 1 in a million; for simplicity's sake, we will assume that a probability of 1 in one billion qualifies an event as a miracle. In other words, a miraculous event is one which occurs only once in every billion opportunities. To get some sense of this level of probability, one billion seconds would take more than 30 years to elapse.

In determining how often these miraculous events occur, it becomes important to recognize that while odds of 1 in one billion are almost unimaginably low, these odds apply not just to a single person, but to many millions of individuals. For example, assume that any single person in the United States has a miraculous, or one in a billion, chance of being struck by lightning in a given day. With odds like these, any single person can safely go on with his life without worrying about storm clouds. But when these odds are applied to the entire 300 million people in the U.S., the equation changes dramatically since each of the 300 million provides another opportunity for the miraculous event to occur. Now, across the entire population, the odds of a lightning strike in a day become 300 million in one billion, or roughly 1 in 3.33. At this probability level, some individual in the U.S. would be struck by lightning every three days, making the miraculous seem almost routine, since many of these strikes would undoubtedly be covered on national news. Fortunately, lightning strikes appear to be infrequent enough to reach even the so-called miraculous level proposed here. But given the large number of citizens in the U.S., it seems statistically likely that one in a million events actually occur on the North American continent several times each day.

PROBABILITY IN BUSINESS AND INDUSTRY

Some business endeavors require a calculation of probabilities, even though little data on which to base the calculation is available. Complex pieces of machinery like the NASA space shuttle are notoriously hard to estimate reliability projections for, due largely to the massive number of components involved. Some components are simple; for example, a tire on the space shuttle is one of the more dependable components. Other components contain thousands of parts; the shuttle's main engines are among the most complex propulsion systems ever designed. In order to calculate the odds of an accident occurring in a single shuttle flight, the chances of failure for each individual component must be calculated, then combined with those of the other components to produce a composite estimate of the ship's chances of returning safely.

As the number of components rises, the process becomes increasingly difficult; because of the shuttle's complexity this process becomes virtually impossible to carry out accurately for such a machine, sometimes forcing engineers to make an educated guess. Unfortunately, these guesses are sometimes given more credibility than they deserve. Prior to the shuttle Challenger's loss on the twenty-fifth shuttle mission, engineers had assessed the shuttle's chance of a catastrophic failure at 1 in 100,000, meaning the ship could have flown every day for 300 years while suffering only one major failure during that time. Unfortunately, these overly optimistic assessments appeared to ignore previous experience with unmanned solid rockets, which suggested an accident rate closer to 1 in 25 or 1 in 50 for the boosters alone. To date, actual experience with the shuttle system has led to 2 shuttle accidents in 113 missions, suggesting that the probability of loss is far closer to the 1 in 50 value than the 1 in 100,000 estimate.

Most consumer products sold today include a warranty period, during which the manufacturer agrees to either repair or replace the product if problems occur. For most products, users expect the item to last far beyond the warranty period; new automobiles typically include a three to five year warranty, even though most buyers expect to drive a new car for twice that long. In some cases, manufacturers attempt to estimate the likely service life of a product by providing a measure called mean time before failure, or MTBF. For example, a computer monitor might be sold with an advertised MTBF of 50,000 hours, which equates to 10 hours of use, 5 days per week, for more than nineteen years. For most customers, nineteen years is longer than they typically keep a monitor, so they will feel comfortable with this purchase. However, MTBF is not the same as a warranty or a minimum lifetime; rather, MTBF provides the mean, or average lifetime of this product model before failure. In other words, half of the products will last longer than MTBF (50,000 hours in this case), but the other half will fall below the average, failing at some point less than the advertised lifetime.

If MTBF does not give a minimum lifetime, how should it be interpreted when trying to assess a product's potential service life? First, if the MTBF has been correctly calculated, the buyer can expect that the item will provide the rated service life or more half the time, so if he buys twenty of the monitors, he can expect at least ten to last 50,000 hours or longer, in some cases perhaps much longer.

The other monitors in the group can be expected to last for varying periods of time, with most of them lasting close to the average lifespan of 50,000 hours and a few failing as the time-span grows further from the mean. In a few cases, monitors might actually quit working within the original warranty period, meaning they would be replaced by the manufacturer. Unfortunately, MTBF calculations for complex electronic equipment can be impractical or impossible to calculate mathematically, meaning that in some cases the MTBF is based largely on engineer intuition and experience with similar parts, rather than actual experimentation.

One of the most exciting moments in a teenager's life is when she finally receives her driver's license. But soon after this triumph may come a rude surprise: car insurance for young drivers is often several times as expensive as for older adults. Why do insurers charge teen drivers more?

Insurance companies are among the largest users of statistical and probability data. Specialists called actuaries spend their days determining exactly how likely events are to occur, allowing the insurer to charge correctly for its policies. Actuarial tables provide summaries of this data; for example, an actuary could use one of these tables to determine that a 45-year-old man in good health is likely to live to be 82 years old, and that his odds of dying next year are 1 in 14,400. Using these probabilities, the insurer can then determine how much to charge the man for a life insurance policy which pays $100,000 to his family in the event of his death.

These probability tables allow insurers to provide discounts to specific customers, such as those who don't smoke, since they have a higher probability of living longer. Automobile insurers also use actuarial data to predict which drivers are more likely to be involved in an accident, in which case the insurer will be obligated to pay for repairs. Using this information, insurers then give lower rates to drivers who have lower odds of having an accident and higher rates to those with higher odds. Based on past experience with millions of drivers, insurance companies know that the odds of a teenage driver, especially a male, having an accident are much higher than for a 30- or 40-year-old. Since the company is more likely to pay a claim for these young drivers, it is forced to charge higher premiums in order to cover the expected losses. As long as young drivers continue to have more accidents in general, even safe teenage drivers will continue to pay higher premiums for auto insurance. In a few cases, actuarial data has shown that certain groups, such as Honor Roll students, are less likely to have accidents, and some insurers now offer discounts to students with strong academic performance.

OTHER USES OF PROBABILITY

In 2001 Russian engineers fired braking rockets to bring the aging Mir space station back to earth. The re-entry was carefully orchestrated to insure that most of the station would burn up in the earth's atmosphere, and any surviving pieces would land harmlessly in the Indian Ocean. Recognizing the incredibly long odds of losing, restaurant chain Taco Bell made an astonishing offer. The company floated a 40-foot square target featuring the words "Free Taco Here" in the Indian Ocean off the coast of Australia. The company then widely advertised that if the remains of the Mir station hit the target, Taco Bell would give one free taco to every person living in the United States. Mir eventually landed thousands of miles from the target, and the company avoided having to serve 300 million free tacos. However, executives at Taco Bell apparently recognized that even the unlikeliest of events occasionally occurs; the company took out an insurance policy in advance just in case the falling station defied the exceptionally long odds and hit the target.

Sometimes the seemingly impossible can be accomplished due to an audience's lack of statistical savvy. Consider this simple magic trick. A magician, claiming to have psychic powers, stands before a crowd and announces that he has noticed an odd coincidence: although there are 365 possible birthdays in a year, he has psychically observed that two of the individuals in this particular audience happen to share the exact same birthday. He then asks a series of questions to help locate the unlikely pair, and after confirming this fact, moves on with his act. Was it psychic power, or simple probability?

To most casual observers, the large number of possible birthdays seems to make the prediction a long shot at best. But considered in terms of probability theory, it begins to look far less magical. Assume that the crowd consists of 12 people. The magician has a 0.5073 chance of being correct, one better than in two. With a bit of showmanship, most psychic performers are able to easily dismiss the predictions they miss using a variety of explanations. But in close to half of this performer's appearances, he will shock the crowd by appearing to do the impossible, when in fact he has simply made a smart bet based on the simple laws of probability.

In a few instances, probabilities are used to attract attention or create fear. Newspaper and magazine headlines during the mid-1990s warned air travelers to avoid planes with fewer than thirty seats, based on statistics which seemed to indicate that these smaller planes were several times more likely to crash than larger jets. But this probability was based on a classification system which grouped small commercial planes in the same category as helicopters and some other types of planes, unrealistically inflating the numbers for the category and making the commuter planes seem less safe. Eliminating the other types of equipment from the equations produced probability figures demonstrating that the smaller commercial planes are approximately as likely to crash as their larger cousins.

Potential Applications

As computational power continues to double every two years, the ability to apply probability theory in new ways will lead to further applications for this powerful tool. In some cases, these applications may involve major improvements in current applications, such as forecasting weather patterns or predicting when and explaining why freeways suddenly become congested. The ability of faster computers to crack increasingly complex codes will lead to an escalating battle between code-writers and code-breakers.

In other cases, advances in probability theory may well result in unforeseen applications. Based on mathematical advances made by eighteenth century mathematician Thomas Bayes, scientists are just beginning to develop software which is comfortable dealing with concepts such as "probably" and "more likely" rather than the simple yes or no typically required in computer programming. Google and other search engines already use rudimentary forms of Bayesian reasoning to answer search queries. Potential future applications include cameras which would visually examine a patient and warn a physician of symptoms making the person more likely to suffer a stroke.