Planning and Executing Credible Experiments. Robert J. Moffat

Preface

      Laboratories (and businesses) need people who can answer difficult questions experimentally. If data are needed for decision‐making, this book is for you.

      Science, medicine, the environment, agriculture, and engineering depend heavily on experiments. Business does as well, when surveying the added complexity of human taste and markets. In any of these fields, an experiment must be credible or it wastes time and resources. This book offers a tested guide, developed and used for decades at Stanford University, to equip novice researchers and experienced technicians to plan and execute credible experiments.

      This book prepares the reader to anticipate the choices one will face from the launch of your project to the final report. We come alongside the reader, emphasizing the strategies of experiment planning, execution, and reporting.

      The foundation of this book originated in “Planning Experimental Programs,” a set of class notes by Robert Moffat, our first author. Our second author, Roy Henk, is one of thousands whose own experiments benefited from Moffat's notes. This book blends our approaches to developing and planning experimental programs.

      Bob Moffat: Our goal for this book was to collect, organize, summarize, and present what we think are important ideas about developing, running, and reporting reproducible, provably accurate experiments – with minimum wasted effort.

      My contribution has to do with the process of developing experimental programs that can be trusted to produce trustworthy data. I used the word “developing” because “new” experiments generally require quite a few changes to eliminate unforeseen (but finally recognized) errors before they qualify as “good” experiments. This is called “the shakedown phase.”

      This raises a question, however: how can you spot an error in a “new” experiment? The only way I know is to run a test for which you already know the answer. We have the three conservation laws we can trust (conservation of mass, momentum, and energy), but they are a bit hard to work with. I think it generally requires less work to simply stretch the operable domain of the proposed experiment until it includes some conditions that have already yielded data that you trust. That data become the “qualification test” for your experiment.

      I've been designing and running experiments for more than 60 years, and my experience has been that the development of a good experiment is always iterative. It takes a lot of work during the shakedown period to eliminate all the sources of error. Surprisingly, this aspect of experiment planning isn't generally “taught.” That's really what started this book! The suggestions presented here reflect things I learned “on the job” during my first 10 years in industry and then refined during the rest of my career at Stanford.

When I got my BSME from the University of Michigan, I was offered a job in the General Motors (GM) Research Laboratories (GMR). I stayed with GMR almost exactly 10 years, rising from working on little problems like calibrating instruments to developing a high‐precision, high‐temperature wind tunnel for testing aircraft engine thermocouple probes up to 1600 °F, finding what caused the bearings to fail on an experimental engine, and developing high‐effectiveness heat exchangers for gas turbine engines. In the last two years of my stay, I was part of a small group of engineers who designed, built, and tested a high‐efficiency, two‐spool regenerative gas turbine that, starting from 30° below zero Fahrenheit, could deliver 350 HP in less than two minutes. That was developed for GM trucks and buses, but a descendant of that engine powered the US Army's M1A1 battle tank – the one designed for cold weather.

      The increasingly interesting problems I encountered in the last three or four years made it clear to me that I needed to go back to school – there was too much that I didn't know.

      I applied to Stanford with references from GMR and was offered a scholarship, which lead to a PhD, a Stanford professorship, 15 years as head of the mechanical engineering department, and 25 years of research and teaching on engineering problems. In my spare time, I worked with Dr. Alvin Hackel, a Stanford pediatrician, to develop the Stanford Transport Incubator. Dr. Hackel provided the medical insights and I did the engineering. His patients, usually newborn and sometimes premature, often needed intensive medical care during transport between hospitals. That incubator won the American Society of Mechanical Engineers' 1987 Holley Medal for Service for the Benefit of Mankind and is displayed on the cover.

      I retired and began to put this book together.

      Now a word from my colleague, Roy Henk.

      Roy Henk: With the aim to continually improve, experiments can fill one's personal daily life. My experimenting began on a small farm with vegetables and livestock (plus honeybees). Later experiments involved concrete mixing, internal combustion engines, molten metal, and measuring ore. At Virginia Tech, I joined low‐noise precision wind‐tunnel tests on flow transition; I have used wind tunnels at Stanford and in Japan at the National Aerospace Laboratory of Japan (NAL), even joining tests in a Mach 5 tunnel. NAL together with JAXA is Japan’s equivalent to NASA. My research used water tunnels at the US Naval Research Lab and at Stanford, and Bob’s notes. Ten years in industry had me designing advanced components of commercial aerospace engines. Compared to air flow around an airplane, thermo‐fluid flow inside an aerospace engine is about the most challenging area of classical physics. Those engines have flown on aircraft for years without incident.

      I have thrilled with experimental success, yet failures too provided valuable data and lessons. For example, as a young researcher I did not realize the advantage of randomizing the order of data collection. How did this matter? In the middle of one particularly costly data run, a valve failed staying open. Had I randomized, I would have had two valuable datasets at lower resolution. Instead that data had

Скачать книгу