The scientific method is a process of asking questions and finding answers. It is the essence of the human experience. Every three year old knows how to do it. A toddler asks a question, a grownup answers, and the toddler asks “Why?” Lather, rinse, repeat. Until exhaustion sets in and the grownup, past the point of frustration, finally answers “Because I said so!”
Good scientists excel at this game because it lasts generations; they never take “Because I said so!” as a valid reason, and continually ask “Why?” A lazy scientist either states “Because I said so!” loudly enough that all the other scientists stop asking “Why?” And nothing upsets a lazy scientist more than another “Why?” after he says “Because I said so!”
Because, you see, “Because I said so!” is equivalent to “’Shut up’, he explained.” As are “settled science” or “consensus” – these phrases are designed to close off debate and have a vote and move on to the next order of business. This is not science.
What happens between the asking and answering is where things get interesting. We ask a question which is only the first step in the process. We then propose an answer (or theory as to why this ought to be so). Note that his is not the actual answer; it is what we would imagine the answer to be. The theory is always rejected until it is confirmed by independent observations. This point is important to understand – any change to our existing understanding of the universe is automatically rejected; it is the responsibility of the person advancing the theory to prove it is correct.
To test whether the theory is correct, we devise an experiment that has a chance of either accepting or rejecting the theory. We must take care so as to not bias the results in favor of either acceptance or rejection. We then perform our experiment and observe whether the theory is supported or not. If the theory is supported, we have unlocked a new understanding of the Universe! Almost.
To keep everyone honest, a good scientist makes his findings known (publishes his results) and invites others to review and criticize his work. Some people look at his experiment, others devise additional experiments, and everyone compares notes, showing whether the observations warrant accepting or rejecting the theory. Once there has been independent confirmation of the theory by multiple people using (preferably) multiple methods, the theory may be accepted – this is what some non-scientists consider “consensus”.
Consensus is one of those warm fuzzy constructs that comes from compromise. Party A gets something and party B gets something; neither gets all of what they want, both get what is most important to each them. That way nobody feels bad about the outcome. It is a “win-win” for both parties.
In terms of science, acceptance of a theory is better referred to as “win-lose”. Proponents of a new theory have the burden of proving that their theory better explains the universe than the old, established theory. If so, the new theory “wins” and the old theory “loses”. Sort of.
Newton’s Laws are still taught in undergraduate physics because most of the world does not move at the speed of light. They have wide applicability in designing buildings, automobiles, and most everything we use on a daily basis. But they do not apply at the limit of the speed of light. We do not formulate a new toothpaste based on relativistic considerations. At least not yet.
An example of a new theory failing was quite recent, in terms of science. During the 1990s, “cold fusion” was all the rage, when some scientists purported to observe fusion at room temperature (as opposed to elevated plasma temperatures, which is why it was called “cold” fusion). Nobody could reproduce their results and the theory is still largely discredited (although some die-hards do believe that it really, really works). However, at the time millions of dollars were set aside for institutes to study this phenomenon.
Scientific evidence does not work as evidence in a legal setting. Scientists do not adhere to a “reasonable doubt” standard. It is probably more correct to state that they adhere to a statistically significant standard. That is to say a theory is accepted if the results can be shown to be more than just a statistical anomaly.
This standard is somewhat loose as to what the definition of “statistically significant” means in practice. In general, the more serious the outcome, the more statistically significant you want the result to be. Do we launch a nuclear counterattack if we have a 5% chance of a false positive or a 0.001% chance of a false positive? I know which I would choose and why.
Just because a theory has been accepted does not mean that another theory cannot take its place in the future. Just look at what happened to Newton. All because Einstein had the gall to ask “Why?”
And so it continues.