《联邦党人文集》背后的统计学幽灵（Ghost was the Federalist Papers behind）

《联邦党人文集》背后的统计学幽灵（Ghost was the Federalist Papers behind） 《联邦党人文集》背后的统计学幽灵(Ghost was the Federalist Papers behind) The statistical spectre behind the Federalist Papers In September 1787, the draft constitution of the United States was circulated to the States for discussion. A group of opposition opponents, under the pseudonym of "anti Federalists", published a large number of articles to criticize the draft. Alexander Hamilton worry, he found a former Foreign Secretary (later Secretary) John Jay, and New York congressman Madison, together with Publius (Publius) under the name of published articles, explain to the public why the United States needs a constitution. They walk fast, usually within a week will be published 3-4 new review. In 1788, they wrote 85 articles published, this is the famous "Federalist Papers". The "Federalist Papers" published, Hamilton insisted on anonymity published, so these articles from the hands of the end, became a detective. In 1810, Hamilton accepted a political opponent's Duel challenge, but he was determined not to shoot for the other, out of Christian faith. A few days before the duel, Hamilton knew he didn't have much time. He listed a list of authors of the Federalist papers. In 1818, Madison presented another author list. The two lists are out of line. In the 85 article, the author of 73 articles is clearly defined, and the remaining 12 are controversial. In 1955, Fredrick Mosteller, a professor of statistics at the Harvard University, found Wallance, a young statistician at the University of Chicago, David, who fooled him into doing research with him. He said to Wallance, "can you come to new England this summer to join me in a small project?"" Mosteller wants to identify the authorship of the Federalist Papers by statistical methods. This is not a small subject at all. Both Hamilton and Madison are good writers, and their style is very close. From the part of the text that has determined the author's identity, Hamilton wrote 9.4 words, and Madison wrote 11.4 words. The average length of each sentence of Hamilton is 34.55 words, and Madison is 34.59 words. As far as the writing style is concerned, Hamilton and Madison are twins. Hamilton and Madison spent about a year writing these articles, and Mosteller and Wallance screened the author's identity for more than 10 years. They are dealing with big data in an age without computers". The project is time-consuming. Nearly 100 students at the Harvard University helped them with the data. The students with the most primitive way, type the text of "Federalist Papers" of the play out, then cut out each word according to the sequence number of the alphabet, put these words to be arranged together. One of the students was tired, stretched and heaved a long sigh. His breath too hard, just put into a good word blowing as the willow catkin. A room of students instantly petrified, it is estimated that many people even destroy his heart. Mosteller Wallance and this is to find a needle in a haystack doing embroidery. They must first weed out the words that are not available. For example, the Federalist Papers often talk about "war", "legislative power", "executive power" and so on, But these words do not reflect the author's writing style because of the theme. Only the "in", "an", "of", "upon", these prepositions, conjunctions and so on, can show the subtle differences in the style of the author. Slowly, they saw a gleam of light in the tunnel portal. A historian kindly told them that a 1916 paper mentioned that Hamilton always uses "while", while Madison always uses "whilst"". Just one clue is not enough. "While" and "whilst" do not appear enough in these 12 authors' pending articles. Moreover, Hamilton and Madison will sometimes write an article, or maybe they will change each article, if Hamilton Madison's "whilst" to "while"? When the students grouped and glued each word, they found that Hamilton had two pages of "upon" on each page, and Madison hardly ever did. Hamilton prefers to use "enough", but Madison seldom uses it. Other useful words include: "there", "on", and so on. In 1964, Mosteller and Wallance published their findings. They concluded that the authors of the 12 articles were likely to be Madison. They are most unsure of the fifty-fifth, Madison is the author, the winning rate is 240:1. The research caused a great sensation. The most striking are not constitutional researchers, but statisticians. Research by Mosteller and Wallance released a spectre of statistics from the bottle. The spectre is the Bayes rule. Bias's law was first proposed by Bias, a mathematician in the eighteenth Century, who wanted to prove that God existed as a first cause. His idea is this: Bayesian sitting with his back to a table, an aide to throw the ball on the table. With his back to the table, Bias didn't know where the ball had landed. He then asked the assistant to throw a ball, and report the ball falls on the first ball on the left or right, if the second ball in the first ball on the left, then it means that the first ball is more likely to fall near the right side of the table. If the assistant continued to throw the ball, throw a ball every time the report falls on the first ball on the left or right, then, the Bayesian will become more and more accurately infer the original ball where. He believes that by trying so far, we can eventually trace back to the original causes of everything in the world. Bias himself did not take this thought too seriously, and perhaps he did not even convince himself, so he forgot about it. In 1774, French mathematician Pierre - Simon Laplasse found the Bayesian rule again. Laplasse's concerns are more mundane: how can we find the real rules when there is a huge amount of data, but there may be a variety of errors and omissions in the data?. Laplasse studied the proportion of births between boys and girls. It was observed that boys seemed to have more births than girls. Is it true that the hypothesis holds true? Laplasse constantly collects new birth records and uses them to infer whether the original probabilities are accurate. Each new record reduces the range of uncertainty. Laplasse gives the expression of the Bayes rule we're using right now. Strictly speaking, the Bayes rule is at least referred to as the Bayesian Laplasse rule, but Laplasse himself later gave up the idea. He found that if the amount of data was large enough, the overall law could be inferred by directly studying the samples. In statistical terms, Laplasse himself turned from a "Bayesian" to a "frequency doctrine"." The frequency of nationalist Bayesian rule as great scourges. The most important problem for the Bayesian school is that the fortune teller has entered the temple of science. In the frequency view of the critics, science is the study of objective facts. We can only observe a replicable phenomenon repeatedly, and accumulate enough data until we can deduce the meaningful laws. The critics insist that nothing that happened in the past will never happen again. In extreme terms, from the frequency point of view, the plane will not collide: they do not hit until they collide. The frequency of nationalist yizhengciyan, but in reality, the problems need to be solved but and frequency of nationalist worlds apart. For example, the frequency of activists may tell the doctor, chronic obstructive pulmonary disease is the symptom of dyspnea, but the doctors want to know is, if I see a patient breathing difficulties, what is the probability that he may be suffering from chronic obstructive pulmonary disease? People with asthma may also be gasping for breath. At this point, we have to abandon the perfect doctrine of frequency and accept the laws of ignorance that gradually discover the laws of approximation. The Bias rule assumes that we initially had a prior probability, you can start from the intuition and even then, you would start to observe new information, so as to obtain the posterior probability, at this time, you can use the posterior probability to modify the prior probability of the first, through the "trial and error", and gradually formed a new and more accurate understanding. The Bias rule allows you to be ignorant, allow you to make mistakes, and even encourage you to make mistakes. So long as you can correct every mistake you make, you can get closer to it. Compared with Puritan Puritans who "cut, eat, sit, sit, sit, sit, sit down", Bayesian people are simply a bunch of confused, boisterous hippies. 250 years ago, the Bayes rule was born. After that, it was forgotten for 50 years and was exiled for 150 years. For a long time, the statistician Bayes rule during the Cultural Revolution as if people carefully conceal mentioning, do not want to bring their friends and relatives overseas. However, always in the fire burning. Even Laplasse himself gave up the idea that French and Russian artillery are still using Bayesian laws to calculate how shells can be played more accurately. During the Second World War, Alan Turing developed the Bayes rule and cracked the German Navy's code. British geophysicist Harold Jeffries used it to speculate that the core is liquid, perhaps molten iron, or mixed with a small amount of nickel. Poincare, a famous French mathematician, testified before the trial of Dreyfus, a Jewish officer. (Zola, a famous writer, wrote my accusation, accusing the court of being racially inclined), He cited the Bayes rule to prove that Dreyfus was innocent. In 1950s and 60s, a rebel in Bayesian school. The study of Mosteller and Wallance was a model that was successfully applied to large data 200 years later. Dennis, Lindley, and Jimmie Savage find a more rigorous mathematical basis for the Bayes rule. Jerome Cornfield uses the Bayes rule to study the possibility that smoking can cause lung cancer. His research was strongly criticized by the critics. The standard bearer of the frequency doctrine, the famous statistician Fisher, not only does not acknowledge this conclusion, but even argues that it may be cancer that causes smoking! At the Harvard University business school, Robert Shlaifer and Howard Raiffa teach MBA students how to use Bayes rules in uncertain situations. Their efforts have finally borne fruit, and economists are more familiar with Bayesian laws than statisticians. Even Greenspan, a former Fed chairman, admitted that economic decisions can not be separated from the Bayes rule - does he really use the Bayes rule? That's another matter. Perhaps, if he does use it, the financial crisis will not explode. When Paulson was to serve as secretary of the Treasury, his predecessor, Lubin, warned him that Washington was a world of probability. Paulson knew right away that he was referring to the Bayes rule. In the last 30 years, the Bayes rule was finally delivered from oppression. In the past, one of the main problems encountered is the Bayesian calculation it is very troublesome, with the development of computer, especially when mathematicians discovered the Markov Monte Carlo method (MCMC), Bayesian school suddenly Conditions are becoming better and better. shine. A field with vigour and vitality of the "Bias revolution" happened: life scientists used it to study how genes are controlled by the educators; suddenly realized that the students in the learning process is actually using the Bias rule; fund manager Bias rule to find the investment strategy; Google Bias rule improved search function, help users to filter spam; unmanned the car roof sensor to collect the received traffic and traffic data, obtained from the application of the Bias rule updating map information. Bayesian laws are widely used in artificial intelligence and Machine Translation. The Bias rule became fashionable. In the era of big data, the multitude information does not let us become wiser, on the contrary, with the increase of information, the noise is more and more, we are still the same as in the past at a loss, we still like exposure to the Beijing haze like can not see the future. Bias's law tells us to admit their ignorance, cautiously probing the road ahead, ready to give up their correction, even once believe things: when the world changes, we must change the view.