Planning and Executing Credible Experiments. Robert J. Moffat

Чтение книги онлайн.

Читать онлайн книгу Planning and Executing Credible Experiments - Robert J. Moffat страница 13

Planning and Executing Credible Experiments - Robert J. Moffat

Скачать книгу

(Dec. 7, 1854). “Dans les champs de l'observation le hasard ne favorise que les esprits préparés,” translated as “In the fields of observation chance favours only the prepared mind.” Lecture, University of Lille. http://en.wikiquote.org/wiki/Louis_Pasteur.

      1 1.1 Following the guide in Appendix D, Section D.1, download and install the statistical language R, which is open source and free. Please consider this software tool essential.

      2 Following the guide in Appendix D, Section D.2, download and install LibreOffice, open source and free.1.2 LibreOffice is compatible with msOffice documents. LibreOffice can even read and write antiquated *.doc and *.xls files of obsolete versions of msOffice better than ms does. The interface is more accessible and less bloated than msOffice. Consider this optional, but highly recommended and free.

      3 1.3 Following the guide in Appendix D, Section D.4, consider R‐Studio. Please consider this software tool optional.

      Notes

      1 1 Variations of the quote are attributed to Albert Einstein and to William Ian Beardmore Beveridge. In The Art of Scientific Investigation (1950), p. 65, “A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it.” http://en.wikiquote.org/wiki/William_Ian_Beardmore_Beveridge.

      2 2 https://quoteinvestigator.com/2014/03/29/sharp‐axe asserts no evidence of Lincoln writing this. Having grown up on a small farm with one chore of clearing hundreds of pines out of our pasture, I (RH) can attest to its advice. Would rail‐splitter Lincoln not agree?

      Richard Feynman

      Science, medicine, and engineering depend heavily on experiments. Business does too, where product design and marketing experiments must account for the added complexity of human nature and taste. In every case in any field, an experiment must be credible or it wastes time and resources.

      Engineering problems are solved using three tools: insight, analysis, and experiment. Usually all three are brought to bear on any given problem; each complementing the others. A sudden insight or inspiration may be enough to suggest what must be done next, a new analysis or a new experiment, but to actually get an answer takes work.

      This book is about the strategy and tactics of experimental work – the techniques whereby one plans and executes an experiment which insight or inspiration has suggested. Our strategies and tactics are a superset which includes the concepts of design of experiments (DoE) and extends beyond.

      Have you heard how designed experiments improve results while reducing the amount of data needed? In our experience, few engineering students have learned DoE in their undergraduate labs, so we are including DoE concepts. Many DoE techniques were pioneered by Ronald Fisher (1890–1962) for agricultural experiments in the early 1900s. As the advantages of DoE become known, we all benefit.

      The purpose of an experiment is to get provably accurate, relevant, and credible data – data that are reliable enough to serve as the basis for answering questions and making decisions. Most experiments arise out of questions which must be answered, such as “Does this device behave the way it is supposed to?” or “How much cooling do we need?” or “What is the relationship between X and Y?” or “Which of these designs performs best?” In many cases, such questions lead to experiments, and the data from those experiments lead to decisions.

      The three key words are “accurate,” “relevant,” and “credible.” One point to remember is that when the work is all done, it will be your signature on the report – and that report will be around for a long time! It is not enough for you to be personally convinced that your results are accurate. You must be able to establish the credibility of the work “beyond a reasonable doubt” or at least well enough so that a prudent engineer would be willing to accept your results as valid when you are no longer around to answer his or her questions.

      The process of establishing credibility begins with the experiment plan and only ends when the results have been presented in such form that they can easily be understood. The experiment plan must make provision for the appropriate checks and balances: baseline checking, repeatability tests, and the other diagnostics which guard against error. Showing agreement with a baseline dataset is one of the most convincing pieces of evidence that can be offered to support the credibility of an experiment. The data presentation must include a quantitative description of the residual uncertainty in the results.

      In large measure, that is what this book is all about: designing and executing experiments for credibility. Before we get to the main issue, however, there are some key points to consider about experiments in general.

      A quantitative property can be measured. A categorical property can be recorded but not measured.

      The act of measurement is an ordering in a scalar system involving a “less than, equal to, or greater than” test. We assign a value to the measurand by comparing it with a standard interval and counting the number of intervals equal to the measurand. The only attributes of any system that can be measured are those which can be put into one‐to‐one correspondence with points on the real number line. Since only the real number system has the order property, only real numbers (scalars) can be measured.

      2.2.1 Examples Not Measurable

      Only the simplest attributes of systems can be directly measured – the rest are inferred.

      Many times people seem to recognize this problem but don't know how to describe their malaise. They want information but find themselves talking about measurements. For example, a former governor of

Скачать книгу