Chapter 5
Decisions and selection
Introduction
This is a review of and commentary on some results in decision science which are applicable to devising schemes for rational materials selection .
The methods described here are techniques of last resort in two senses. They are inferior to all techniques based on physical modelling where such models are appropriate, and otherwise they are universal in applicability and superior to any other technique. 'Professional' judgement without problem-structuring aids has been shown to be inferior to judgement which uses decision science techniques, not because the techniques are so good but because the human mind is so limited.
Background
Many materials information systems are more than simple databases; they attempt to guide users and to help them make decisions based on the information therein. However these decision support facilities have generally been devised without reference to the large body of knowledge concerning technical decision making that has been accumulated in schools of management science over the past forty years.
Materials selection is sometimes taught as if value judgements can always be avoided, but as designs progress from conceptual to embodiment and final stages, the amount of relevant information increases in volume and detail and materials selection can become entirely constrained by details. This implies potentially lost opportunities unless the value judgements which decide the details can be structured so that their influence on materials choice is explicit.
This review is intended to provide some essential background in decision science to those who are constructing the next generation of materials information decision support systems. It brings past work on materials selection by judgement into the same framework for criticism and further development.
Decision analysis
Decision theory is a body of systematic, rational techniques used to help decision makers decide what course of action to take. It is increasingly rich in the variety of techniques for representing decision problems.
It has been shown that the need for satisficing decisions is far more common than the need for optimal decisions and bounded rationality is the term used to describe a wide variety of techniques which do not strictly adhere to principles of rationality but which approximate to them. A simple illustration of the necessity of bounded rationality in practice is the impossibility of deciding how much effort to put into assessing the different alternatives before the decision is made: a truly rational estimate could only be made either after that assessment has already been done or in a situation where there are no limits on the decision makers' resources [Sim81, Daw88].
Rationality
Rationality can be defined as adopting a set of rules which will be adhered to in the course of decision making [Wat87], and decision theory supplies some sets of rules. Rational decisions are thus coherent and consistent; a rational process is one that avoids contradiction either at the time or in the future [Kee76].
For materials selection we can say that we want the following consistency criteria to be satisfied for any rational decision:
Satisfying these requirements is harder than it looks.
Human judgement
A very substantial body of research shows unequivocally that professional judgement based on intuitive or holistic appreciation of a decision problem as an undivided whole is never superior to statistically derived weighted averages of the relevant attributes. Professionals are very good at identifying the relevant attributes because they know what information to look for, but are demonstrably poor at actually weighing the evidence and producing a judgement [Daw88].
The research findings are largely based on clinical, commercial and educational correlations but there is no evidence to suggest that engineers are any better. All people make poor intuitive (holistic) judgements when when two or more attributes are incomparable directly: attention shifts from one to the other without actually weighing their relative importance. What is worse, people think that their decisions are better than they really are due to the additional selective memory problems of learning from experience. Natural mental processes that 'systematically lead to irrational decisions include honouring sunk costs, being swayed by framing and grouping effects, systematically misjudging probabilities on the basis of representativeness or availability (to memory), and thinking in scenario terms' [Daw88]. The reasons are not hard to find; it is very hard to think in a way in which one does not think: human minds are better for some things than others and judgement is something we are poor at.
The selection problem
Materials selection in practice involves many different uses and types of materials information: material properties, availability, standards, codes of practice, established uses, current local experience etc. Also important are the best predictions for, and estimates of the costs of obtaining, some unavailable information, e.g. optimum processing conditions, in-service lifetime, future availability.
Academic research and database developers have concentrated on the irreducible core of materials selection, where every property value is usually considered to have identical reliability or quality, for a closed set of materials. It is also assumed that the property information is sufficient and appropriate for the selection task (whereas in reality it is more common that the conditions of service do not match at all well the types of measurements which were made to obtain the properties available). It is this unrealistic core problem which is the subject of this chapter. If uncertain or time-dependent information is important then more techniques from decision science become appropriate (decision trees, risk preference, judgement techniques using probability etc., see [Kee76], [Wat87], [Daw88] or [Neu90] ).
Multi-attribute problems
If there is only one property to be considered then the core selection problem is defined to be trivial: the materials are ranked according to the values of that property. Difficulty arises when there are a number of properties (a number of attributes) for each material, where an overall ranking is required and where a physical model cannot be formulated. The need is to reduce a variety of measures down to some common currency so that an overall measure can be derived. This common currency is termed 'error signal' in control engineering, 'payoff' in game theory, 'fitness' in evolutionary genetics, 'reward' in psychology, 'net present value' in accountancy and 'utility' and 'value' in economics and decision theory.
Value functions calculated from an aggregate of properties are defined so that they are considered identical if they are strategically equivalent, if they lead to the same final ranking of the alternatives. Utility functions however are 'measures' in the technical sense, like temperature measured in Celsius or Fahrenheit, and are not so malleable: they must be unique under linear transformation. This distinction is taken from Keeney and Raiffa [Kee76].
In materials selection, the set of attributes which will be used to base a selection is predefined: it is the set of materials properties which are available. This situation is slightly different from those for which decision analysis is usually applied where attributes tend to be defined and assayed specifically for the decision to be taken. Thus the methods can often assume that the attributes satisfy some constraints which may not be true for materials properties. For example, it is recommended [Kee76] that attributes be chosen such that they are:
Since these recommendations cannot always be followed, materials selection often involves the use of 'proxy' attributes (properties) which are more appropriate according to these criteria.
There is a common variety of materials selection problem which looks as if it is a single attribute type but where it must be decomposed into the multi-attribute case. This is where a simple property can be defined, but where no data for that property exists: an example might be the slip-resistance of a surface. If an appropriate specific test is available then it is easy, but if measurements according to the test have not been made then proxy properties such as friction, loss coefficient, and elastic modulus must be combined in (preferably) a physical model or (if necessary) a value function.
The need to use proxy properties does not usually arise when physical models are constructed because such models tend to be expressed in terms of well-characterized, physically based stable properties such as stiffness, density, thermal expansion etc. which depend directly on chemical bond type and crystal structure and for which data is often available or can be estimated. It is the capricious properties that depend on relative kinetics or microstructure which tend to need decision techniques because physical models of these properties are, by definition, complex and highly variable (see Chapter 3).
For the purposes of discussion it is sometimes convenient to anthropomorphize the situation and to assume that we have an 'expert' for each of the attributes we need to consider, and to set the core problem as one of deriving a formal negotiation procedure which will produce an overall view taking every expert's view into account.
Merit indices
A general procedure for deriving merit indices for new problems has recently been devised by Ashby (see Chapter 2) [Ceb91]. These merit indices are a compact representation of a physical model.
Currently merit indices are derived using simple mechanics and thermal conduction theories but the Ashby method also enables them to be derived using any well-founded theory which provides numerical models of materials behaviour. More complex accountancy would also model other aspects of material costs within an organization. So a merit index would be possible which took account of capital charges on the value of material stock, where the dwell time as well as the cost depended on the material. These more sophisticated models will not always be exactly accurate (unlike simple mechanics) so the merit indices derived using them would have a measure of uncertainty which would also have to be modelled and predicted. Progress in this area of extended merit indices can be expected to be rapid since it only requires that existing materials science research be viewed in a slightly different light.
The Ashby method is also significant for purposes outside the 'core' problem of materials selection since it directly indicates areas of information: properties, combinations of properties and materials, which should have priority in continuing information gathering. Conversely the value function methods only indicate materials which should be assessed further, the help they give with deciding which properties are important arises from sensitivity analysis which identifies unimportant properties. Sensitivity analysis is an under-used technique that can be applied to any materials selection method.
Theorems and pitfalls
A non-trivial comparison function or negotiation procedure is usually termed a multi-attribute value function because it takes into account multiple attributes (materials properties) and allows the calculation of some overall value which ranks the alternatives (materials) in order of merit.
There are several classic results of which the designer of a value function should be aware: Condorcet's Paradox, the non-inferior set, Arrow's Impossibility Theorem and Simpson's 'Paradox' in addition to the von Neumann and Morgenstern axioms for the existence of such functions, and the mutual preference independence theorem which defines the appropriate conditions for linear additive functions [Kee76, Wat87, Daw88, Neu90].
Simpson's paradox
Simpson's Paradox; 'paradox' is only counter-intuitive for innumeratte people. It simply states that if a number of variables are grouped into sets then the mean average value of the variables taken together is different from the mean of the means from each set [Daw88].
Simpson's 'paradox' often appears in misleading advertising and in statistics manipulated to demonstrate a political point. The classic example is an investigation of sex discrimination at Berkeley (University of California) where women had lower rates of admittance than men for the university as a whole, but where within each department women were more likely to be admitted than men. A hidden distribution or unequally-sized groups is the cause of the apparent paradox. In the Berkeley case it was because women tended to apply to popular departments with lower admission rates, e.g. English in preference to engineering.
The lesson for devising strategies for choosing between alternatives (materials) is to be very careful in designing user interfaces to decision support software. The software should not allow problems to be stated in such a way that Simpson's effects can be obtained. This is a real danger because typically the system will have done some grouping of data into sets itself and such groupings may be hidden from the user who may then attempt to take an average over the groups rather than over the whole data set.
Condorcet's paradox
The Marquis de Condorcet noticed in 1785 that even if every member of a committee holds perfectly self-consistent views, the decisions of the committee (which decides by majority voting) can be inconsistent and in fact usually are. This has an analogy with Simpson's 'paradox' but ranked orderings, preferences, are being manipulated instead of numerical averages and the difficulty is real rather than imagined.
The manipulation of preferences might be thought to be an ideal method for arriving at overall rankings of materials in a selection problem without having to compare incommensurate quantities. Reducing the problem down to preferences would appear to give an ideal common 'currency'. However consider three materials, lead, PVC and mild steel, and consider a materials selection committee of three: a corrosion expert, a weight expert and a strength expert. Their preferences can be stated thus:
low corrosion lead > PVC > steel
low weight PVC > steel > lead
high strength steel > lead > PVC
Majority voting, considering pair-wise choices, will prefer lead to PVC by 2 to 1 (corrosion and strength prefer it, density does not), similarly PVC will be preferred to steel, but also steel will be preferred to lead! This particular example is cyclic which is unusual but most situations lead to similar inconsistency.
Thus while there are rare occasions when the decisions of the selection committee are consistent and the final ranking is agreed, common experience suggests that these will be as rare as unanimous decisions in human committees.
The non-inferior set
Further thought shows that unanimous committee decisions can only occur when one alternative dominates all the others, that is when one alternative is better than the others on all the relevant properties. Thus while Condorcet's result means that there is no complete ranking, nevertheless a committee can unambiguously agree that the dominated alternatives (materials) should be excluded even if it cannot agree on the ranking of those not excluded. That set which is not dominated is termed the 'Efficient frontier', the Pareto optimal set or the 'Non-inferior set' [Wat87, Neu90].
(a) example (b) dominating/dominated regions
Figure 5.1 Dominance relationships: the axes are two different attributes
The dominated alternatives are those for which there is no trade-off to be made, they are just unambiguously worse. Figure 5.1 (a) illustrates that materials A, B and C form the non-inferior set, with E being dominated by B, D being dominated by B and C, F being dominated by A and B, and G being dominated by all the others. Figure 5.1(b) shows the regions dominating (light shading) and dominated by (dark shading) D.
The non-inferior set contains all good candidates for selection but deciding between them means that the designer has to make some trade-off between desirable and undesirable properties. Trade-offs are thought by most to be an essential part of good design, therefore some balancing of incommensurate quantities will be required and Pareto dominance is of use only as part of the evaluation.
In a materials selection information system, excluding all dominated materials from the display simplifies the users' view. In the real world however there are always other factors to be taken into account, so keeping at least all the 'near-misses' in view is probably sensible. A different colour or symbol for dominated materials (for each material type: metal, polymer etc.) on the visual display might be helpful to users. (Methods for displaying and manipulating shortlists were discussed in Chapter 2).
Dominance induces two weak orderings on the materials, weak because of the many tied positions, where each material is ranked by two measures: the number of other materials that it dominates and the inverse of the number of materials that dominate it. All the materials in the non-inferior set tie for first place on the second of these rankings. No existing materials information system (or any decision analysis software for that matter [Bue88]) displays either of these two measures, easy though they are to produce.
Arrow's impossibility theorem
Since Condorcet's result shows that majority voting is not a generally useful method for consolidating a group's preferences, in 1951 Arrow tried to find what method would be satisfactory. The answer is that there is no such method, it provably does not exist.
To produce a grouping method which works, more than simple preferences are required: either the intensity of each expert's opinion or some common currency for the experts' evaluations is required, or both.
The 'Borda' rule is commonly used for ranking sports teams based on their ranked orders over a number of different competitions, e.g. combining football league with football association rankings. It produces an overall ranking purely from individual preferences and is not dictatorial. It works by giving each alternative a score equal to its rank, thus the third team gets a score of 3 etc., then adding up all the scores and re-ranking them with the lowest score first. Thus using just the strength and weight criteria for the lead, PVC and steel example earlier, PVC gets (1+3), steel gets (2+1) and lead gets (3+2), giving an overall ranking of steel > PVC > lead.
Now suppose that lead is removed from consideration: the new scores become steel (2+1) and PVC (1+2), so steel ~ lead. Thus the overall preference between steel and PVC has been affected by something other than the preferences of the 'experts' with respect to just steel and PVC. This is irrational.
The influences of non-significant information on choice are termed framing effects. The Borda rule is a concrete example.
Von Neumann and Morgenstern axioms
Arrow's impossibility theorem shows the need for constructing aggregate value functions for material properties where the physics (or accountancy) of the problem does not suggest a model and hence a merit index and a reduction to the trivial case. Von Neumann and Morgenstern showed that given certain conditions (the axioms), a monotonic value function with certain characteristics can always be associated with each attribute (material property) even if it might appear only to have discrete values. This is redundant for a property such as 'strength' where there is already a clear numerical function, but it is much more useful for a property such as 'susceptibility to corrosion' where numerical values can be derived in comparison with the values of defined end-points, usually taken as zero and one.
Recently it has been shown that the minimum score (0) for any property should be associated with the minimum value of that property among only those in the non-inferior set, and not the minimum among all alternatives (materials). If this is not done then these 'irrelevant' minima distort the scale, falsify trade-off possibilities and can even 'hide' whole regions of preferred solutions [Sch90]. We know that these minima are irrelevant because dominated alternatives can never be candidates for selection. However it does mean that the scores for a particular property will then depend on precisely which other properties are being considered for each (ephemeral) selection decision. This may be disturbing for engineers using a software system which implements this technique.
The difficulties of constructing a value function using the von Neumann and Morgenstern method for an individual property are insurmountable for one-off, semi-routine materials selection situations because their overhead of effort is so high; but perhaps they are not insurmountable for database compilers (see below). Such a function could be linearly re-scaled with a new zero point to fit with any particular ephemeral non-inferior set as an engineer uses the system.
Multi-attribute aggregate value
Having obtained either values or utilities for all the individual attributes, the problem remains of aggregating them to produce a single ranked order of preferred materials. The construction of a multiple attribute utility function can only be done in classical decision analysis by designing questionnaires which ask for people's views about the value of two or more attributes together. It has the effect of mixing the values of each property (the vector x, which represents property values on some common numeric, non-dimensional scale) together with a set of weights (the vector w) for balancing trade-offs.
The value is a function of both weights and properties:
v = f (w . x)
Note that the properties x might include some qualitative properties which have been assigned numerical values through the use of the analysis of the type shown to be possible under the von Neumann and Morgenstern rules.
Two value functions v1 and v2 are 'strategically equivalent' if
v1(x) = T(v2(x))
where T() is any monotonic function (e.g. square-root, logarithm, etc.) because they both produce the same preference order of alternatives (materials) [Kee76]. Note that this means that asserted differences between some multiplicative and additive types of function may be unimportant since taking logarithms will transform one into the other. Note also that strategic equivalence does not in general imply:
v1(x) ~ v2(T(x))
For very important materials selection decisions the extremely time-consuming questionnaire may be justified but it only has value for a particular group of people at a particular time and place. There is also a practical limit of only about six properties that can be included, the limit being determined by the time it takes for a subject to answer a sufficiently comprehensive questionnaire [Fie88].
It is likely that responses to questionnaires will be inconsistent ('irrational') so extra questions are required to provide redundancy. A methodology exists for providing numerical measures of the divergence from coherency and for pin-pointing inconsistent assessments using an eigenvalue approach so that interviewees can re-assess their answers and produce data on which a well-behaved value function can be based [Hug90].
A less-general, linear-sum version of the value function is often used, where the function g() is some non-linear function and where the sum of the weights is 1.0:
v = S(i = 1,n) { wi . g(xi)}
where S(i = 1,n) {wi} = 1.0
This might seem trivial but there are good theoretical reasons why a function of precisely this form is appropriate under certain well-defined conditions [Kee76]. In most cases g() is the identity function (see below).
Mutual preference independence
If three or more properties are mutually preference independent then a theorem states that there exists a linear additive function which describes their aggregate value and that it is the only appropriate function [Kee76, Wat87, Neu90].
Two attributes (properties) are preference independent if the numerical trade-off between them is independent of the values of all the other properties: the pair are in a sense orthogonal to the other properties (though the relationship is not commutative, if properties x1 and x2 are preference independent of x3 and x4 it does not follow that x3 and x4 are preference independent of x1 and x2). If the trade-off between two properties x and y is such that Dy units is an appropriate compensation for one x unit, then preference independence means that whatever the values of any other property z, the trade of Dy for one x is unchanged.
In a materials context this could mean that the trade-off between strength and weight would be independent of the required resistance to corrosion, or that the trade-offs between different methods of manufacture would be independent of any in-service requirement and yet some in-service properties would nevertheless be dependent on manufacture.
Any sub-space of the decision space spanned by preference independent pairs is mutually preference independent (which is a commutative relationship). This is a strong condition which implies that the decision procedure has interesting properties. If expressed for n attributes this yields a linear additive value function of the form:
v = S(i = 1,n) { wi . xi}
What is more, the theorem states that this form of the value function is unique (under positive linear transformation) and that no other function is appropriate to mutually preference independent decision spaces [Kee76].
If the situation is not well-defined enough for a numerical physical model to be appropriate, physically based reasoning can still give insight into which pairs of properties are preference independent of others, and therefore which sub-spaces are describable by linear additive functions. In addition qualitative physical models can show that combining properties can lead to preference independence where there was none previously. In a design problem of deflection under thermal inequilibrium (see [Ceb91]), the sub-set of properties stiffness (E), specific heat (Cp), density (r) and thermal conductivity (l) have no preference independence. However stiffness (E) and thermal diffusivity (l/r.Cp) probably are preference independent of all the other properties.
Pricing-out and extended dominance
'Pricing-out' is a technique whereby all the variation in one or more properties is described in terms of one of the other properties, usually monetary cost; though weight is often used in aerospace design. The technique requires that
Every property priced-out reduces the dimensionality of the problem, so in combination with physically-inspired combined properties aimed at increasing preference independence, and merit indices which also reduce dimensionality, the selection can be greatly clarified. Any sub-space of the design problem that can be simplified in this manner can then be used in sensitivity analysis to observe effects on other more intractable properties. If the dimensionality can be reduced enough it might be possible to decide directly on a trade-off between rather unfamiliar property combinations, such as between (K1c)2/3/r and l/a. If there is a physical model for such a trade-off then there is no problem, but if it is a matter for engineering judgement then the unfamiliarity means that the engineer would have to experiment with sensitivity analyses to see how the trade-off behaved in practice.
When the number of effective attributes has been reduced by any of the techniques described above, the dominance relationships (the non-inferior set) can be re-assessed or 'extended'. This will nearly always have the effect of reducing the number of good candidates. There is a further effect in that if the effective range of one of the remaining properties becomes small among the remaining materials: for example, if all remaining materials are similar in corrosion resistance, then it might be a useful approximation to assume that the other properties are preferentially independent of corrosion resistance [Kee76]. Such an approximation might not have been valid on the original non-inferior set.
Displaying multiple attributes
A conventional scatter plot effectively communicates the necessary information if only two attributes are relevant, and the non-inferior set can be identified by eye. Figure 5.1 is a scatter plot, so too are the Ashby materials selection charts (Figure 2.3) [Ash89a,b]. Contours of value function(s) can also be added easily to such plots. A third discrete attribute can be displayed by use of colour or different symbols, e.g. to distinguish classes of materials or to distinguish compressive from tensile properties, but continuously valued third attributes or contours are much harder to add and to interpret. The approach taken by the package EMS is to structure the decision problem in terms of pairs of attributes (properties or merit indices [Ceb91]).
For numbers of attributes between three and about ten an 'icon' type of display is sometimes used where each alternative material is represented by a symbol which varies in systematic ways with values of the attributes (a review is given in [Lew90b]). A serious problem with this type of display is that the significance of different aspects of the icon has to be learned so the technique is inappropriate for problems where the relevant set of properties is subject to rapid change.
Figure 5.2 A polygon multi-attribute display showing an alternative (grey) against an ideal (black)
The polygon display is used, together with a subsidiary value function (see below) in the commercial PERITUS materials database [Per91]. One advantage of this type of icon display is that the different properties can be clearly labelled and the user does not have to rely on an overall, learned 'feel' for the data. It is thus appropriate when the set of properties being considered changes frequently. The polygon usually displays each attribute on an axis with zero at the centre and the maximum value for that property at the extreme radius, in which case the candidates for selection (alternatives) all form shapes within the ideal regular polygon. Alternatively a particular candidate can be chosen as a reference or a target and the other candidates scaled appropriately. PERITUS also defines a 'balance' function which measures how far from the ideal shape the polygon for a material is, irrespective of whether it is good or bad.
The weighted bar-chart display is effective at showing properties, weights placed on properties, and data describing materials, if the number of properties and number of materials are less than about five or six [Neu90]. Figure 5.3 shows an example of a bar chart with four properties and three materials. Materials A and B form the non-inferior set since they both dominate C.
Figure 5.3 A weighted bar-chart display
A particularly useful feature is the scaling of the bar for each property to be in proportion to the weights. This means that importance to the aggregate value is represented by the same length on each bar. A recent software package V*I*S*A uses, in addition to the chart shown in Figure 5.3, two interactive bar-charts: weights for each attribute and aggregate values for each alternative [Bel89]. The user presses 'up' and 'down' keys to modify weights and sees the bars for the alternatives shift in response and the weights of other attributes change (they are normalized to sum to 1.0).
Predefined value functions
For materials selection from a closed set of materials in a database, von Neumann and Morgenstern's work shows that it is possible for the database compiler to construct numerical functions for all the material properties which would otherwise have only qualitative measures. This is only feasible if it has to be done only once for the entire life of the database. These numerical values could then be compounded into an aggregate value function to be used in selection decisions.
The scheme might be to construct value functions for each property and then allow each individual user to set weights to balance their relative importance in an aggregation function. The mutual preference independence theorem shows that a linear aggregation function is an adequate representation if, in the users' view, each property-pair tradeoff can be considered to be independent of all the others (which is unfortunately only a first approximation for materials selections problems). Note that 'independent' here means independent with respect to the specification of the task for which the material is selected. Any individual material's properties are in reality highly correlated: e.g. strength with stiffness and melting point [Fro82], but that is irrelevant here. However, there is a maintenance problem when new materials are added to the database.
Requiring the user of a materials selection system to supply weights even for only that sub-set of properties deemed to be relevant is a significant imposition, apart from not actually being particularly effective. It is difficult for an engineer to articulate such weights a priori and it is almost as difficult for the engineer to re-assess such weights even after several iterations of experience with a single selection problem [Zuc89]. There are at least two structured techniques which can help the user generate weights systematically, the hierarchies [Pah84, Wat87] and the (misleading and flawed) pairwise 'digital logic' [Die83] comparisons.
The initial structuring techniques help the a priori estimation but do not help re-assessment. In any case engineers are much happier manipulating concepts they are familiar with, such as density and strength, than with abstract concepts such as weights. However it must be faced that the comfort of decision makers must come second to their professionalism if there is a conflict. Once physical models have been used to their fullest extent, any remaining selection decisions should be made and documented by some kind of an aggregate value function method rather than by holistic 'feel'.
There is evidence that when past holistic decisions are re-analysed in terms of a value function and the implicit weights the expert 'used unconsciously' are extracted, these weights are more extreme than the weights which the expert articulates explicitly [Daw88]. This implies that holistic decisions effectively take fewer attributes into account than do explicit aggregate value function methods, which could explain why holistic decisions are usually worse.
Normalization and updating
The decision analysis literature generally assumes that the information gathered for the purpose of supporting a decision is then used only for that decision, and if further decisions are required later then the information will be re-evaluated first. This is not the case with decisions made using database information. The information in databases has certain characteristics, constrained by the ways in which databases have to be managed, which conflicts with the requirements of some decision analysis techniques.
All the aggregation functions described above suffer from implicit changes to their effective weights if new data is added to the database. Often a normalization is used of the form
xi = (yi - bi) / (ti - bi)
where 'ti' is the highest, and 'bi' the lowest, value for that property of all the materials in the database. This produces a set of non-dimensional property values that all lie between 0 and 1.
Adding information on new materials to a database produces no problems if the properties fall within the limits already defined, but if, say, a new, extremely corrosion resistant material is added then all the materials values for the corrosion property will have to be re-scaled to take into account the new end point. This has the effect of compressing the corrosion scale for pre-existing materials. This does not affect their ranking with respect to corrosion, but has the implicit effect of decreasing the weight given to corrosion as compared to other properties. This means that the overall ranking, using unchanged weights, of pre-existing materials can change when a new material is added even though that new material may not be significant in the final ranking, which in turn means that the set of value functions and weights, considered as a single multi-attribute function, is not independent of irrelevant information. This is irrational.
The alternative course of action is not to re-scale the data when adding new information, but to extend the scale to give the new material a corrosion resistance of, say, 1.1 when the previous high point was 1.0. The values assigned to different material properties will then depend on the order in which materials are added to the database, which also does not seem desirable.
If the property data were re-scaled at each use of the database, using the minimum of the then current non-inferior set [Sch90], then the entire set of weights would be different for each specific selection decision; but it would also mean that database updates would not affect past decisions if the new data was for a dominated material. The only useful conclusion seems to be that when the database is initially compiled, a scale should be derived for each property (using the von Neumann and Morgenstern axioms and associated method if necessary) such that the absolute maximum and minimum conceivable values are taken as the end points from the beginning.
Materials selection
Materials selection is an engineering problem and as engineers we would like to think that the matter of rationality can safely be assumed. Unfortunately, as soon as the comfortable support of physical modelling is removed, subtle differences in the way weights and preferences are expressed can lead to inconsistencies and hence, by definition, irrationality. Most engineers might think the Borda aggregation rule a poor tool, but few would notice that it violates important conditions of rationality.
Many engineers faced with selecting materials based on a number of properties would begin by identifying the most important property, create a short-list, and continue by considering the less important properties in turn. This technique is known as 'lexicographic ordering with aspiration levels' and although simple it rarely captures the whole problem [Kee76]. This is because the sequence of selections has a greater influence than the difference in the importances of the properties warrant. Far better is to consider each property on its own, for all the materials, and then combine the shortlists by some structured technique such as a multi-attribute value function.
Satiation and good all-rounders
For many materials selection problems the utility of a material to the designer does not scale linearly with respect to its properties: more usually there is a target value which a property must meet and then any higher values are useful but not essential. The utility of a property also tends to satiate with a law of diminishing returns (see Figure 5.4).
Figure 5.4 Diminishing returns as a property improves
It is found that materials which are good all-rounders in their properties are often preferable, that is they have higher overall utility than do other materials which score equally well on an aggregate value function but which have some extremely good and some very bad properties. This preference for all-rounders is a consequence of utilities with diminishing returns and is consistent with the empirical observation that engineers typically weight properties non-linearly with respect to their values [Fie88].
Clearly a better selection decision could be made if utility relationships were known rather than just value functions, but utilities are specific to a particular design task and vary even for a single material depending on the use to which it will be put. Developing utility functions for semi-routine materials selections appears to be more trouble than they are worth unless some generally applicable functional form could be devised.
Hopgood's function
Hopgood considered both additive and multiplicative aggregation functions for materials selection and decided that both appear to have an important defect: unimportant properties have too great a negative influence [Hop89].
vj = S(i = 1,n) { wi . xij}
vj = P(i = 1,n) { wi . xij}
In these formulae there are 'n' relevant properties and the overall value for material 'j' is given by a value function which adds or multiplies over all the properties (vj is the value of material 'j' taking into account properties 'i = 1,n'). Hopgood preferred the multiplicative function because it favours 'all-round' performers rather than materials with extreme values of some properties, but modified it to take account of the 'unimportant property' problem.
The point is that the distinction between a good or a poor value for an unimportant property should not be so significant as the same distinction for an important property, yet the simple additive and multiplication functions are symmetric and appear not take that into account. Hopgood suggested an asymmetric multiplicative function AIM ('alternative inference method') of the form:
vj = P(i = 1,n) { c1 + (wi- c2).(xij-c3) }
where c3 is the mid-point of the scale used for the property measurement (i.e. it would be 4.5 if the scale ran from 0 to 9), c1 is a scale-shift term chosen to ensure that all resulting values are positive, and c2 is the mid-point of the weights. (If the weights are then defined to be between 0 and 10, and if c2 is not used (set to zero), then the scale shift term c1 turns out to be 46.)
The principle behind the multiplicative formula is that of the modification of prior odds by new evidence:
new odds = likelihood ratio * prior odds
This is a procedure known as Bayesian updating which derives directly from the principles of probability.
Each property is treated as a new piece of evidence with the likelihood ratio equal to the property value (score) times the weight given by the user.
likelihood ratioi = wi . xi
The prior odds for a material are irrelevant because they are the same for all materials and so do not affect the ranking. Taking the likelihood ratios for all the properties leads directly to a multiplicative form of the function. The scale shift terms were introduced to create a value function of the right 'shape' so that unimportant properties are treated sensibly. Hopgood's function is an honest attempt to use scores and weights directly, but in a form inspired by the principle of Bayesian updating. However the decision analysis literature indicates that such a method is probably unnecessary and perhaps inappropriate.
If we agree with Hopgood's assertions concerning either unimportant properties or the more suitable behaviour of multiplicative value functions, then that implies that the simple linear aggregation function is inappropriate, which, by the mutual preference independence theorem, means that mutual preference independence is inappropriate to materials selection (and possibly for more general types of selection task). Such conclusions would however be premature: additive functions with normalized weights do in fact provide correctly for the treatment of unimportant properties.
There is, however, some disagreement whether weights should be normalized or not: the view being taken that this removes the possibility of changing the weight for one property 'independently' of the others [Hop91]. There is no 'correct' interpretation for this since it depends on the context in which the selection task is being performed and how one wishes to design a user interface. If the weights are unnormalized then the ordering produced by the value function of a given set of materials is unchanged, so long as the same weights are used throughout. It is when comparisons must be made between selection tasks using different weights that care must be taken that the function performs in the required manner.
An unorthodox aggregate value function, somewhat similar to an additive AIM, was developed specifically for a materials selection system developed as a programming exercise [Saw86]. This was noteworthy because the entire source code, in Turbo™ Pascal, was published and placed in the public domain. It used the following value function:
v = (1/n) . S(i = 1,n) abs|{ wi . (xi - pi)|
where the user had to pick both a weight (wi) and a 'pivot' (pi) for each property. This implicitly meant that the user had some control over the shape of the utility function for each property, since that is determined by the pivot.
The PERITUS functions
The PERITUS materials database and selection system includes two multi-attribute value functions which perform a minor role. The preferred means of interaction with the system are the manipulation of short-lists satisfying aspiration levels and the use of polygon visualizations.
The first, relatively standard, value function is termed a 'performance' function, and is a linear additive aggregate value function which is interesting because it removes the need for the user to supply weights: the weights are implicitly equal for all properties and instead the user manipulates the scales of the properties by specifying a target level ti for each:
performance = (1/n) . S(i = 1,n) (xi /ti)
The 1/n term appears so that values can be coarsely compared between different selection tasks even when the number of relevant properties may be different in each case.
There is a second function which calculates a 'balance' of properties intended to show how similar a candidate material is to the ideal target in terms of the spread of properties irrespective of whether it is better or worse:
balance = (1/n) . S(i = 1,n) abs|(xi /ti) -performance)|
The 'odd-fish' effect
There is a problem with linear additive value functions, and perhaps with all aggregate value functions, that occurs when engineers experiment with a variety of different sets of weights in order to exercise such a system to see if it gives the answers they expect on problems they are already confident with.
Value functions do not exclude any alternative materials, they just rank the entire set slightly differently depending on the weights. What happens is that as the engineer 'homes in' on the precise set of weights which best describes the problem, the materials scores on the value function tend to bunch up into a narrow range, and yet slight changes can still lead to unexpected materials appearing, or 'odd-shaped fish appearing in the trawl net' as it has been termed by one of the participants in an experiment by Zucker [Zuc89]. This is an example of incremental instability.
Zucker used Hopgood's function for most of his studies but informally reported that the linear additive value function as used by the PLASCAMS polymers database led to the same effects. Unfortunately Zucker did not report the actual numerical values of the properties used in his tests so it is just possible that the peculiar behaviour was due to a particular kind of 'error' in the database itself. This is not entirely unlikely since the selection was performed using poorly defined properties such as 'glossiness', 'impact resistance' and 'mouldability' and the database compilers may have been slightly inconsistent in the way they scored different materials according to those measures. It is unlikely that the scoring was performed according to the von Neumann and Morgenstern axioms so that the difference between scores of, say, 3 and 4 could have been very much more (or less) important than the difference between scores of 5 and 6. Since the value function takes those differences to be equivalent, it would not be surprising that the overall behaviour would appear pathological.
Generating weights
The time-consuming and abstract process of generating weights is the greatest disincentive to the use of aggregate value functions by engineers. The clearest method is probably through the use of an influence hierarchy [Pah84, Wat87].
Figure 5.5 shows a hierarchy of the influences that particular aspects of the selection problem might have on choosing the best material. There are two things to note about it. First, the attributes in the hierarchy are not necessarily immediately identifiable with particular materials' properties: that requires another stage of analysis; and second, the weights of particular aspects are treated in the same way as probabilities (by multiplication) as one descends the tree although there is no uncertainty in the situation.
Figure 5.5 Hierarchical method for setting weights
In order to split up aspects into more detailed attributes in this way it is necessary to be sure that they are conditionally independent, analogous to the requirement that independent probabilities should have zero covariance [Kee76]. At least 18 software packages, not all of them intended for commercial use, have been produced which can aid this task of hierarchical decomposition. Some can be used to edit and move branches of the tree while preserving the local and global weights properly [Bue88, Bel89].
A significant problem with attribute (property) hierarchies is that they are not unique and that forming them depends on how an engineer sees the problem. For example, in the design of an implantable prosthetic device it would be possible for corrosion to be considered a sub-attribute of toxicity rather than prosthesis lifetime; worse, it is possible that different hierarchies could be appropriate for different materials in the same selection task. If such different trees led to different weights then those materials would be truly non-comparable.
Different structures will usually lead to different weights through framing effects (the distortion of human judgement that arises from the influence of non significant facts), though a set of weights which is valid for a number of different trees (different ways of looking at the problem) is likely to be robust and reliable.
Choices versus policies
The core problem of materials selection is often considered to be a solitary act performed by an individual, yet in the real world it has consequences that extend over time and which influence other materials selection choices in the future. The point is that there is a distinction between a single selection and a policy which would stand for all such similar selections in the future. It is in the interest of organizations that individual choices should be consistent and that they should reflect or synthesize a definite policy. The recording of materials selection decisions as part of design audits is intended to encourage such activity.
The point is that if a design decision is made on a certain basis, then when that design is re-evaluated later either for incorporation into a larger design or for another production batch, the material selected could be re-evaluated quickly. If a record is kept on which materials have been approved for company use since the first selection, then a recorded selection decision can be 're-run' on the new data to see if an improved material is available or, more likely, whether the criteria used in the past are no longer appropriate.
An interesting difference between single selection and selection policy is that a policy should explicitly take into account probabilities of error. This means that the skewness of probability distributions used to define value functions does have an effect since large organizations may be more or less risk averse than individual engineers.
Expert systems
Expert systems, if they produce an overall ranking of a sub-set of materials, must still abide by the same rules of rationality as any other form of preference or weighted value function. The certainty factors, Bayesian probability assessment, support-logic, evidence rules and related techniques are all governed by the same rules, and the same normalization and update problems, as discussed here for linear-additive and multiplicative functions. They add nothing to the rationality of selection, but they do add a quantity of case-based experience (heuristics) together with a pattern matching system to identify which past experiences are relevant to the current problem.
Although expert systems are often advertised as being easy to update by simply adding more rules, it is in fact well known that the weights and probability assessments within such systems are very fragile with respect to the addition of new information. New data usually requires re-assessment of existing weights in order to prevent the enlarged system becoming self-inconsistent, a very much more serious problem than the inability to re-run old decisions which is the characteristic of databases.
Proposal
It therefore remains only to decide which type of value function(s) to use for materials selection when physical modelling proves impossible. The multiplicative functions, including AIM, are attractive in their simplicity and usefulness but their mathematical basis is extremely complex in terms of the preference independences they require and imply (so complex that it has not been covered in this review, see [Kee86] for further details). On the other hand there is an enormous weight of practical experience with linear additive functions, and their justification in terms of mutual preference independence is relatively understandable and provides a crucial link to physical insight, modelling and merit indices.
The question then arises of how to present linear additive models to practising engineers and how to take account of the decreasing utility of a material's property above a particular level. Weights are initially harder to visualize and to manipulate (because of their implicit dependence on property range) than target properties, but weights enable a very clean hierarchical decomposition technique for complicated problems. Target properties, as used in PERITUS, would work well together with 'pivot' or 'saturation' property values to model non-linear utility relationships.
Perhaps we should leave the complicated problems to those who are happy with weights and concentrate on representations of simple problems: those which manipulate range instead. We do not want to introduce too many parameters but there is probably a minimum of two per property required, even without introducing utility nonlinearities. These would be the target value and the appropriate baseline (zero) value for each property. Hopefully the baseline could remain unchanged for long periods and would probably be predetermined by the database compiler.
Figure 5.6 A general purpose utility function
If we wish to represent a non-linear utility relationship then it should be done on a property by property basis (it may not be needed for all properties) and the aggregate would be a multi-attribute utility function of the same linear additive form. The shape of the individual utility functions is problematic: what we want is a curve which becomes greater than zero at the target value, which is less than 1 for all conceivable property values (assuming utilities to be normalized to lie always between -1 and 1), and which nearly saturates at a utility of, say, 0.95, at some specified property value.
In addition it has to have sensible behaviour for values less than the target, probably reaching a utility of -1 for a property value equal to the baseline. For properties where a non-linear utility is not required, a parameter-less linear function is required to transform values (which are usually 1 when the property equals the target) to utilities (which have just been defined to be 0 when the property equals the target).
Any function of the form shown in Figure 5.6 could probably work adequately, it would be scaled by the target and saturation parameters (in the appropriate property units) which would be readily understandable. The invariant shape for all properties would be adequate as a first approximation. No great accuracy is required since the basic technique is not itself accurate. Perhaps its greatest benefit in practice would be the imposition of a structured, systematic way of approaching decision problems irrespective of how the structure is based in theory.
Summary
This chapter has attempted to show both the necessity for and feasibility and usefulness of constructing value functions (or something else which behaves in the same way) to compare different materials in cases where the physics or accountancy of the situation does not collapse the problem to the 'trivial' case. It has been shown that constructing such functions contains many pitfalls for the unwary and some of the more important difficult cases have been described.
It is concluded that being able to update a database by adding new materials without violating certain basic principles (update sequence independence and freedom from value drift) is incompatible with ensuring that old selection decisions continue to give the same materials rankings when the same, old weights are used on the new data. This can be ameliorated by initially defining property scales with wide limits and then never changing them.
The practicality of value functions for everyday engineering materials selection decisions seems poor, especially because of the burden it places on engineers of articulating weights or target scales. However the conclusion is inescapable that the techniques should be encouraged wherever appropriate because holistic judgement appears to be so untrustworthy. Expressing decisions in this form would convey some benefit in recording selection decisions for design audits, for justifying decisions to management and for explaining the rationale for decisions to new members of a re-design team.
Footnotes
1. Rational: (1) Having or exercising the ability to reason. (2) Of sound mind. Rationality is not used here in contrast to emotion, it is used to describe the way in which decisions can be taken, not what should be decided. Strictly speaking, rationality only specifies how not to decide.
2. Value functions are required usually only to give an overall preference order whereas utilities should be "measures" in the sense that differences between utilities are meaningful. The von Neumann and Morgenstern method does in fact produce measures.
3. Minimizing weight for a brittle beam in bending where the bending moment is due to temperature gradients [Ash89b, Ceb91].
4. This is what the UK Plastics and Rubber Institute commercial polymers database PLASCAMS does, though it uses identical scales for both numeric properties (such as strength) and semi-qualitative properties (such as surface finish). One problem is that scores for a product property may be only monotonic rather than being proper measures as required by the von Neumann and Morgenstern rules.