experiment plan:
Level 1: What happens?
Level 2: By what means does it happen?
Level 3: What are the underlying physics?
It is important to identify the level of the motivating question and ensure that all following decisions are aimed at progress at that same level. The different levels lead to different experiments for a given situation.
For example, consider a program aimed at studying the effects of high free‐stream turbulence u' on the value of the heat‐transfer coefficient h on a flat plate.
The Level 1 question would be: “How does the heat‐transfer coefficient h vary with free‐stream turbulence u', all other factors remaining fixed?” This question would be answered by a series of experiments in which h was measured for different values of u' for some specified set of conditions.
A Level 2 question would be: “What changes in the structure of the boundary layer were responsible for the changes in h?” This question would be answered by measurements of the velocity and temperature distribution within the boundary layer, seeking a correlation between high h and some recognizable change in the boundary layer structure.
A Level 3 question would be: “How have the momentum and energy transports within the boundary layer been altered by the turbulence?” This question would be answered by measurements of the mixing length and turbulent Prandtl number or the turbulence intensity and dissipation length scale within the boundary layer.
Note that these different levels require entirely different types of measurements, so it is important to know which level is most important to the client, the ultimate customer.
There is an implicit presumption in modern fluid mechanics and heat transfer that low‐order events can be predicted from high order, i.e. that knowledge of the turbulence transport mechanisms will allow calculation of the boundary layer structure, and that knowledge of the boundary layer structure will allow prediction of h at the surface.
This hierarchical assumption leads some experimenters to immediately start work at Level 2 or Level 3, seeking an elegant and powerful answer to the Level 1 question. In general, however, it seems to work best if one directly addresses the Level 1 question first over the entire range of the desired conditions before attempting to shift up to Level 2 and Level 3.
It is particularly important not to inadvertently mix levels in the same experiment plan. One may pursue two, or even three, levels of questions within one series of tests, and that is frequently done to save testing time, but one should remain aware that each level requires different data and that the different datasets are addressing separate objectives.
4.5 Strong Inference
A particularly powerful approach to selection of the “right” question is described by John R. Platt, professor of physics and biophysics at the University of Chicago (1953), as “strong inference.” The essential features of strong inference are (1) the formulation of more than one alternative hypothesis concerning the major question, and (2) the execution of a set of experiments that have the possibility of disproving each hypothesis. The power of the method lies in the fact that, once a hypothesis has been disproved, then no further work need be done along that line. Note that if an experiment simply supports a hypothesis, not much progress has really been made, because the very next experiment may disprove it. To use strong inference, one must assemble systems of hypotheses that fully enclose the main question so that at least one hypothesis must finally survive its experimental test.
Platt lists the following steps:
1 Devise alternative hypotheses.
2 Devise a crucial experiment (or several of them) with alternative possible outcomes, each of which will as nearly as possible exclude one or more of the hypotheses.
3 Carry out the experiments so as to get clean results.
4 Recycle the procedure, making subhypotheses or sequential hypotheses to refine the possibilities.
Identification of the motivating question corresponds to the step devising alternative hypotheses. The specific experimental objective is, then, to test these hypotheses by experiment.
Platt's contribution is his reminder to seek disproof for hypotheses, rather than support, as a more economic strategy.
4.6 Agree on the Form of an Acceptable Answer
Once the question has been determined, the next step is to agree on the form of an acceptable answer. A clear agreement on the form of an acceptable answer helps ensure that both parties have the same question in mind.
Thinking about the form of an acceptable answer may, in fact, lead to refining the question, especially if you play “devil's advocate” and look for “silly” answers. If “silly” answers are not precluded by the form of the question, then the question needs to be refined.
The question “What is the effectiveness of this heat exchanger at its design conditions?” needs only a single number for an answer: “The effectiveness is 0.86.”
Another example, using instrument calibration, has more latitude. If the calibration question was: “Does this instrument meet its accuracy specification?” then a satisfactory answer would be a simple “Yes” or “No.” If, however, the question was “How do the readings of this instrument compare with the true values, over its range?” then an acceptable answer would require a table of values, or a chart, listing the indicated value corresponding to each true value.
Note that a simple request to “Calibrate this instrument” does not establish which of these two answers would be acceptable and does not qualify as a desirable motivating question.
4.7 Specify the Allowable Uncertainty
The third point that should be dealt with is uncertainty. How much uncertainty can be tolerated in the answer to the motivating question?
For most heat‐transfer situations, an uncertainty of ±5% is “Olympic‐quality” data. Much of our present heat‐transfer understanding was developed from historic and legacy databases with ±20% uncertainty. Often, the client already has some data or has a design approach that works reasonably well but wants to reduce the uncertainty. In such a situation, to produce more data with the same uncertainty as the existing dataset will not advance the art, nor will it satisfy the client.
The allowable uncertainty should be described in a manner consistent with current conventions. For most engineering work, the uncertainty is specified at the 95% confidence level and presumes a symmetric probability distribution, centered around zero error. For example, if the uncertainty is quoted as ±10% at the 95% confidence level, this means that 19 of 20 repeated trials of the same measurement act will yield a value within ±10% of the quoted value. The possible error is, by agreement, equally likely to be positive or negative.
In some cases, the client may wish to specify a “one‐sided” tolerance, for example to demonstrate that a product is better than its competition. A one‐sided test must be selected before the experiment is run. The one‐sided test must be clearly indicated in the report so that the results can be interpreted accurately in terms of the one‐sided error allowance. Otherwise, the experiment will be reckoned as a two‐sided or
