Robert Shaw
This paper records the development of stipulative
definitions for use in policy analysis. The work was undertaken in the Ministry
of Research, Science and Technology, to support the separation of research
funding into practical categories.
RULES FOR STIPULATIVE DEFINITION
METHODS OF STIPULATIVE DEFINITION
OPERATIONAL RESEARCH AND OUTPUT RESEARCH
THE PRACTICAL DEVELOPMENT OF STIPULATIVE
DEFINITIONS
AN EVALUATION OF THE DEFINITIONS DEVELOPED
This text is published to assist
policy analysts when they address a commonly faced problem. Although the paper is made available because
it may be useful, it was in fact written as a part of a real policy development
project and it has been deliberately left in its “raw” state in order that the
real flavor of the work may remain.
Frequently in policy development
projects it is necessary to draw firm distinctions between categories. Usually the categories are established with
a view to their use in the making or budget decisions. Policy analysts have to establish a rational
basis for the distinctions they draw, and they have to be able to convince
others that their classifications are reasonable.
This paper addresses one approach
to the establishment of categories for budget, and other, purposes –
stipulative definition. The paper is
deliberately written in a style that should be accessible to analysts, and
although the example of policy development described (developing definitions of
public good and operational research) is drawn from a need within New Zealand’s
science policy, the techniques outlined should be found to transfer into other
areas of policy development.
Finally, the text does not deal
with the methods of conceptual analysis, but this methodology is an important
complement to stipulative definition.
Some of the skills needed for conceptual analysis are also needed in the
development of stipulative definitions.
Analysts with a background in philosophy might well be those in the best
position to develop these areas of policy development in practical contexts,
however this should preclude others from trying their hand.
1.
This paper records the development of stipulative definitions
that were needed to operationalise New Zealand’s new science policy. Making explicit the nature of such
definitions and consciously adopting a methodology that was consistent with
this understanding of stipulative definitions developed the definitions. The work was undertaken within the Policy
Division of the Ministry of Research, Science and Technology.
2.
Hence, the paper deals with the following topics:
a)
the nature of definitions;
b)
principles to be followed when making stipulative definitions,
and the methods used to make stipulative definitions;
c)
the need for definitions of operational and output research
in the implementation of New Zealand’s new science policy;
d)
the procedures adopted to develop the two definitions and
conclusions which were reached about the definitions as the project proceeded;
e)
general conclusions about the development of stipulative
definitions.
3.
Recent legislation such as the Public Finance Act 1989 and
the Foundation for Research, Science and Technology Act 1990 provide a new
framework within which New Zealand’s public sector must operate. Included in both those pieces of legislation
are definitions that relate to the funding of scientific research. Naturally, the operational definitions used
in addressing funding questions must be consistent with the legislation. However, what is in the legislation is not
by itself sufficient to guide the practical decisions of officials.
4.
The establishment of the Foundation for Research, Science
and Technology, the restructuring of science providers (research
organizations), and the likely provision of greater commercial powers to
science providers has emphasized to officials the importance of a comprehensive
set of definitions which will facilitate both communication and good
decision-making.
5.
Definition is a complex and exacting process. Experience has shown that the process itself
sometimes leads to valuable insight, and decisions taken as definitions are
developed occasionally have profound, or unexpected, consequences beyond the
“definition development” project.
6.
Definitions are one instrument that officials use to make
public policy practical. Definitions
used in making science-funding decisions were expected to have substantial practical
repercussions. If good definitions are
not developed officials may not be able to bring about the intentions of
Government.
7.
Galileo said that all definitions were arbitrary (Two New
Sciences, tr. Crew and De Salvio, New York, 1914, p.162). It must help the process if we understand
why Galileo was mistaken and the means by which definitions can be made
practical and robust. Hence, we sought
to understand the conceptual processes involved, and alternative methodologies
that might be adopted, when stipulative definitions are developed.
8.
The literature on definition is extensive. The first distinction which we should make –
both to assist our thinking and because it has practical consequences – is
between the ‘purpose’ and the ‘method’ of a definition. The purpose of a definition is what it is
trying to do; and the method is the means that it adopts to achieve its
purpose. Different definitions have different
purposes, and they achieve them by different means.
9.
In developing our definitions for science it will be
necessary to pay attention to both purposes and methods.
10.
There are three purposes of definition according to Robinson
(Definition, Oxford University Press, Oxford, 1954, p.18):
a)
Word-word i.e. a class of nominal definition;
b)
Work-thing, a class of nominal definition;
c)
Thing-thing, i.e. real definition.
11.
Word-thing definitions may be of two kinds:
a)
Lexical definition (reporting the customary or dictionary
meaning of a word;
b)
Stipulative definition (establishing, announcing or choosing
one’s own meaning for a word).
12.
the broadest interpretation of word-thing definition would
be: any process by which any individual brings another individual to know the
meaning of a verbal symbol or any sort of word.
13.
Lexical definition is a form of history. In giving a lexical definition we are
explaining how a word has been used in the past. It is not uncommon for people to have unrealistic expectations of
lexical definitions. Some
administrators evidently believe the Concise Oxford Dictionary contains all the
distinctions necessary to administer any programme.
14.
It is likely that everything is capable of lexical
definition, but lexical definition has its drawbacks from out
point-of-view. Many concepts that need
to be given practical meaning are not capable of lexical definition. Government’s policies are often promulgated
as only broad ethical imperatives.
Wittgenstein’s great contribution to philosophy was to explain some of
the ways in which concepts work. Lexical
definitions can never be expected to replace concepts, nor can they be expected
to adequately provide the distinctions needed to operationalise concepts that
become Government policy.
15.
Another drawback with lexical definition is the flux of
language. If you watch television or
listen to children you cannot but be impressed by the evolution of language.
16.
Consequently, the conclusion was drawn that stipulative
definitions, but not lexical definitions, may be useful in the restructuring of
science administration. Only
stipulative definitions could bring the degree of exactness and certainty which
is required in the taking of funding decisions over a period of years.
17.
Parliamentary counsel, mathematicians, Humpty Dumpty, the
White Rabbit and Adam all believed in the value of stipulative definition. “Whatsoever Adam called every living
creature, that was the name thereof.”
18.
By ‘stipulative’ word-thing definition is meant the explicit
and self-conscious setting up of a meaning-relation between some word and some
object, the act of assigning an object to a name, not the act of recording an
already existing assignment.
19.
Stipulative definitions have advantages and
disadvantages. People involved in the
process of stipulation will do well if they understand these.
20.
Stipulative definitions are not made for their own
sake. They are made for some further
purpose. Keeping this in mind, consider
the following summary of advantages and disadvantages.
a)
Stipulative definitions are the only form of definition
which can remove ambiguity;
b)
Ambiguity is not removed if the words used in the
stipulative definition themselves contain ambiguity;
c)
Stipulation is most straight-forward when it goes from a
concrete object to a word. The
stipulation of definitions for abstract concepts is frequently pointless. Kekes has made this point in his notion of
“essentially contested concepts”;
d)
In some circumstances abstract concepts have to be
formulated into stipulative definitions so they can be used in practical or
theoretical situations. It is from this
that stipulative definitions derive their great utility. Scientific discourse draws upon stipulation
of necessity;
e)
Stipulative definitions are not helpful if persons involved
in the discourse will not subject themselves to their discipline;
f)
Unrestricted stipulation is not acceptable. If we are not careful people will feel
deceived by stipulative definitions.
Stipulation can also lead to falsehood because it hides difference;
g)
As Humpty Dumpty shows us, it is not reasonable to stipulate
without regards to consequences;
h)
Largely because of its association with science, stipulation
can be used to impress people (consider “garbologist” or “geophagy”);
i)
Artificial definitions have to be maintained by labour. Lexical definitions maintain themselves;
j)
Stipulation places a burden upon readers. Many scientific writers have made themselves
unreadable;
k)
The habit of stipulating one’s meanings for words is said to
bring with it the habit of avoiding analysis.
In many instances we are under an obligation to show what we understand
by concepts established within academic disciplines;
l)
Stipulation may be a form of abbreviation. Hence, it may unreasonably hide the
complexity of a thing.
21.
From the above list of advantages and disadvantages for
stipulative definitions it is possible to derive rules which we may usefully
lay upon ourselves (based upon Robinson):
a)
Stipulate as little as possible;
b)
Do not stipulate unless there is good reason to believe that
a phrase which already covers our designation is too cumbersome for our
purposes;
c)
Do not stipulate unless you are certain there is not in
existence a name for the thing you want to name;
d)
Do not stipulate one word for two different things;
e)
Do not use stipulation to change the emotional force of a
word;
f)
Ensure that stipulation is unlikely to deceive;
g)
Once a word is stipulated we must ensure that in our work it
is never used in another sense (hence, Decarte’s review);
h)
Words in titles should always be lexical words;
i)
Get friends to criticize our stipulate definitions
(Faraday’s rule – read his discussion of electrolysis);
j)
It is frequently better to invent a new word rather than
stipulate a meaning for an old one (in New Zealand’s bureaucracy “output” means
“goods and services”);
k)
We may remind ourselves to be responsible in stipulation.
22.
There are at least seven methods by which a stipulative
definition may be established. They are
the methods by which a person may be taught the meaning of a new word.
23.
Fortunately, words tend to fall into classes according to
the ways in which they are defined in common practice. Here it is proposed to comment upon three
methods which are particularly relevant to the development of definitions
required for science administration: the method of analysis, the method of
synthesis, and the denotative method.
24.
In this method the meaning of a word is conveyed by naming a
bigger class in which the object falls, and then naming something which
distinguishes the object from the class as a whole. This is probably the most common method of stipulative
definition.
25.
Examples from Parliamentary Council (who draft New Zealand’s
legislation) abound: from the Public Finance Act 1989 – “Grant” means an amount
of public money provided by the Crown to finance some purpose. And – “Benefit” means an amount of public
money provided by the Crown for the benefit of some person. The advantage of this method is that it
tells you something about the category. The disadvantage is that it may be
elaborate and cognitively demanding.
Consider –
“Mode B” means (a) In the case of outputs, an appropriation
of public money for the acquisition of goods and services of a non-capital
nature relating to a specified class of outputs or a programme; (b) In the case
of capital contributions, an appropriation of public money to increase the
amount of the Crown’s net asset holding in a department; (c) In the case of
benefits and grants, an appropriation of public money for the making of
payments of benefits or grants on behalf of the Crown. [S.2 Public Finance
Act 1989)
This example
is clearly cognitively demanding.
26.
This method of word-thing definition indicates the thing
meant by mentioning its relation to other known things (which may of course be
other words).
27.
The synthetic method has the great advantage that it is
always possible to define things in this way, because everything is unique in
its relation to other things. Another
advantage is that comparisons often make things very clear. Children will understand “middle C” when you
play the scale of notes. It is more
difficult to understand “middle C” by analysis or synonyms (another method not
discussed in the present paper, but obvious).
28.
Frequently, the synthetic method is combined with the
analytic method. Consider –
“Minister” means the Minister of Finance or any member of
the Executive Council acting on behalf of the Minister of Finance.
[Ibid]
In this example an analytic definition
is supported by a synthetic provision (… acting on behalf of …). But, consider –
“Operating surplus”
means any excess of departmental revenue over expenditure…[Ibid]
29.
This method consists of mentioning examples of things to
which the word applies. These examples
may be either particular things to which the word applies, or sorts of thing to
which the word applies. If the examples
are easy to comprehend, and there is consistency in their content, this method
is appealing. But, upon reflection it
may be seen that this method in a sense never actually defines the word. In stead, the reader is left to gain the
concept by induction.
30.
This method is quite unlike the analytic and synthetic
methods, and it is not difficult to find people who believe this is the only
appropriate way to develop definitions.
It is probably the most direct way to develop some abstract concepts. Peters (Ethics and Education, 1965)
demonstrated this in educational philosophy for example. So far as stipulative definitions are
concerned the denotative method is on way of bringing greater certainty to an
analytic approach. Consider this
example which beings as analytic and then becomes denotative (just to be
certain!):
“Public Security” means any security pursuant to section 53
of this Act or any provision of any other Act; and includes any loan or credit
agreement, guarantee, indemnity, bond, note debenture, bill of exchange,
Treasury bill, Government stock, and any other security representing part of
the public debt of New Zealand [Ibid]
31.
Three characteristics of the debate on science policy stand
out. Cabinet papers demonstrate the
characteristics well. They are:
a)
Definitions abound;
b)
Definitions are not well developed;
c)
Definitions developed in one place for one purpose do not
have an easy correspondence with those elsewhere.
It has often
been the case that definitions for categories of research have been developed
for particular practical purposes without regard to the overall framework of
science policy. When in 1990 the
Ministry of Research, Science and Technology was established, one of its instructions
from Government was to implement proposed reformation in science administration
in a manner which ensured policies were consistent, and consequently it was
intended that policies be integrated.
This in turn necessitated attention to the relationships between
definitions.
32.
Legislation provides a settled set of stipulative
definitions upon which it is appropriate to base further work.
The following definitions appear in
legislation and relate to science policy (Attachment 1 gives all the definitions):
Class of Outputs, Cost, Mode A, Mode B, Mode C, Outcomes, Outputs, Public good
science outputs, Research, Science, Science outputs.
33.
Central to recent legislation is a framework established by
definitions: the outcome/output framework.
The definition of Outputs at first appears to have both analytic and
synthetic components. The analytic
component is straightforward. Outputs are goods and services that may be
produced. The
Synthetic component is a list of who
may provide the goods or services: “… produced by a department, Crown agency,
Office of Parliament, or other person or body.” Hence, the synthetic part of the definitions brings no greater
specificity to the definition because anyone can produce the goods and
services. We should ask if it was necessary
to define a common word that has a known meaning to now mean “good and
services”.
34.
In the case of science outputs, both goods and services may
be embodied in writing reports. In
other words, reports are movable property produced (goods) or reports may be
the record of work completed (services).
Hence, we can conclude that an organization purchasing science outputs
cannot go wrong if its purchases research reports.
35.
The question is: could such an organisation purchase
anything other than reports? They could
purchase services that are not record in reports. Work could be done which relates to science, perhaps a patent
registered or a conference held.
36.
It would appear that they could also purchase in-service
training for science teachers, or equip school or private enterprise science
laboratories. These would all be
services which relate to science. In
contrast, “Public good science outputs” is a more narrow concept.
37.
The Ministry of Research, Science and Technology, in the
middle of 1990 had to attend to the definition of ‘public good science outputs’
and a raft of other definitions which were required as a consequence of PGSO
appearing in the Foundation for Research Science and Technology Act 1990.
38.
In the Act the following definitions appear in s.2
Interpretation:
“Public good science outputs” means
science outputs –
a)
That are
likely to increase knowledge or understanding of the physical, biological or
social environment; or
b)
That are
likely to develop, maintain, or increase research skills or scientific
expertise that are or is of particular important to New Zealand; or
c)
That may be of
benefit to New Zealand, but are unlikely to be funded, or adequately funded,
from non-governmental;
“Research” means scientific research and includes scientific
development, and services;
“Science” includes the social sciences; and “scientific” has
a corresponding meaning;
“Science outputs” means goods and services (including
information) that relate to research, science or technology.
39.
The statutory definitions have not proved sufficient for
officials who must make practical Government’s science policy. The New Zealand Government until 1990 funded
scientific institutions by having Parliament make annual appropriations of
money. From 1990 Government policy has
been to purchase classes of scientific outputs (goods and service = research). The Foundation for Research, Science and
Technology is an impartial purchasing agency which operates under the direction
of Government.
40.
The definition of PGSO is composed of three logically
distinct, synthetic components. In
other words, there are three separate definitions, and only one of these
definitions must be satisfied for a research project to be classified as
PGSO. The following points are worthy
of attention with regards the expression ‘PGSO’ itself:
a)
Each of the three components is a definition of a particular
kind: (a) is analytic and denotative, (b) is analytic, and (c) is analytic and
synthetic.
b)
Complexity is brought to the definition of PGSO because PGSO
itself is a complex descriptor, which is in part further defined in the Act and
in part not defined at all.
c)
Defined further, as indicated above, is “science outputs”
‘science’ is clearly associated with its lexical definition and because of the
synthetic addition (“including the social sciences”) has a clear presumption
towards scientific disciplines; and, ‘outputs’ are goods and services (consistent
with the definition in the Public Finance Act 1989).
d)
Left undefined is the notion of ‘public good’. This expression is used in the discipline of
economics, although this usage was not that discussed by Members of Parliament
when they were considering the Bill both at the Committee stages and as it was
read through the House of Representatives.
41.
It is difficult to know if ‘public good’ ought to be
considered as a concept and subjected to conceptual analysis, or whether it
ought to be taken as grist to the mill of stipulation and defined to facilitate
practical purposes. The term ‘public
good’ usually appears in science policy to describe research which does not
have immediate application within a sector of the economy. Such work is said to be ‘non-appropriable’
and it is generally accepted that responsibility for funding it lies with
Government.
42.
Another example of the use of the term ‘public good’ is to
described statistics which ought to be gathered and promulgated at Government
expense. Here, aside from arguments
about appropriability, there are concerns bout the desirability of having a
neutral organisation collect the information and adopt methodology which will
ensure the credibility of the result.
Hence, the concept of public good as it is evidently being used in
statistics policy is rather different from that in science policy.
43.
It is important for administrators to be able to distinguish
between research that ought to be funded as a PGSO and be (subject to the
national priority setting process), and, research that should be undertaken by
government departments. This latter
class of research is funded by appropriation through departments and is known
as operational research. It may be
consistent with the definition of PGSO in the Act.
44.
The distinction between operational and output research
became particularly critical in the transition to the new national science
policy: the funds for operational research that was classified as output
research were transferred from departments to the Foundation.
45.
Consequently, the definitions of PGSO (= output research)
and operational research had to be developed with care; and they were developed
at a time when scientists and research managers were passionately concerned
about research funding decisions.
46.
The Ministry of Research, Science and Technology was asked
by Cabinet to review research in government departments to consider its
classification in December 1989 and the project was begun in the middle of
1990. The Ministry conceptualized the
project as involving two activities:
a)
The development of stipulative definitions for PGSO and
operational research.
b)
An examination of extant research in light of the
definitions.
Attachment
2 is the methodology adopted by the Ministry after consultation with
departments. The present paper is
concerned only with ‘a’ above.
47. Robinson’s
account of how stipulative definitions ought to be developed was a useful place
to begin a consideration of methodology.
When practical work began three things became apparent and were not
controversial:
§
The distinction between the analytic and synthetic constructions
was not always very obvious, and this did not appear to have any practical
consequences;
§
The complexity of the definitions being developed required a
strategy which alternated denotative with analytic/synthetic methods. In effect we would say: here is an example
of operational research upon which we are all agreed – now that criteria led us
to that conclusion. Or alternatively,
we believe this is a criterion – what examples of research projects would be
classified as operational if this criterion was applied by itself or in
conjunction with other criteria.
§
It was helpful to take some Wittgensteinian instructions
that related to conceptual analysis and use them, admittedly in a somewhat
corrupted form, to prompt consideration of aspect of the definitions being
developed. Some examples were: On what occasion, or for what purpose, might
we wish to emphasise this criterion? What
kinds of actions may follow from the making of this distinction? What actions or circumstances were
associated with the perception of a particular criterion?
48.
Hence, in October 1990 the tentative stipulative definitions
were broken into their analytic and synthetic elements and each element was
considered against known characteristics of existing research projects. Then, the reviewers worked the other
way. Projects considered to be clearly
operational, or clearly output research, were taken as examples and compared
with other projects and eventually with the elements of definitions.
49.
Given the context, a Government department implementing
innovative Government policy, it was appropriate that the definitions be based
upon legislation. Hence, we began with:
“Output
research” must:
a)
Satisfy the
definition of “public good science output” given in the Foundation for Research,
Science and Technology Act 1990; and
b)
Be funded, at
present, in part or in full, by a vote; and
c)
In principle,
be capable of being identified, successfully undertaken, and effectively
managed, apart from related work which is the responsibility of a department.”
And
“Operational
research must:
a)
Satisfy the
definition of “research” given in the Foundation for Research, Science and
Technology Act 1990; and
b)
Be undertaken
to facilitate the delivery of an “output” (as defined in the Public Finance Act
1989) and which appears in the corporate plan of the department.”
50.
A committee of senior officers from the Ministry of
Research, Science and Technology considered these definitions and speculated on
how successful they would be in making the distinctions which those officers
thought necessary if the new science administrative provisions were to operate
successfully. Of most concern was the
desirability if subjecting all government research requirements to an integrated,
consistent priority setting process. If
departments were able to establish research projects which they considered in
the national interest without reference to the national priority setting
process then that process simply would not achieve its objective.
51.
In passing we may note the extent of the determination of
Government that New Zealand’s adopts a highly centralized, plan approach to
science policy.
52.
On 11 October 1990, a group of officials from three
departments met to attempt to use, and review, the criteria. They considered four research projects which
were described in writing, and orally, by the departments responsible for those
projects. The projects were:
a) Public
understanding of the climate change issue
b) Forestry
options for the mitigation of global warming
c) Mainland
ship rat biology in relation to control
d) Norway
rats in Breaksea Sound, Fiordland, their impact and eradication.
53.
Five important discoveries were made when projects were
considered:
It
was discovered that decision-making was facilitated if decisions were taken in
the correct order (see later).
The
definitions, as originally drafted on 11 October 1990, were not sufficiently
practical to be of use in the classification of projects.
i.
The criteria which were useful were not sufficient in themselves
to be taken as definitive – a group of criteria had to be considered together
and a judgment made.
ii.
It would be more difficult to establish criteria for “output
research” than it would be for “operational research”.
iii.
There was little point in developing a definition of output
research that did not include a reference to the content of the research
itself. This is because government
funded output research is defined annually by content areas through the priority
setting process. Hence, whilst a project
may satisfy the definition of a “public good science output” unless it appears
in the output categories it will not be funded. It is important to note that this point applies only to the
classification of existing projects (which was the goal of the definition
development project being described here).
If it was applied to long-term proposals it would distract from the
priority setting process.
54. As a result of
the meeting on 11 October 1990 there was developed a list of “indicative
criteria.” These were criteria that
were thought to be relevant factors in making judgments about research
projects, but they were not sufficient in themselves (when taken individually)
to form the basis of a decision. It was
considered likely that the criteria could be considered in some sort of
cumulative fashion and hence render a decision in individual cases. No-one suggested giving them numerical
weightings, although this would have been the next logical step.
55. The indicative
criteria, developed by considering research projects, were:
Criteria
|
Operational
|
Output
|
|
Specified as contributing to the delivery of an output |
Will be directly linked to an output that is approved by
Government, and will generally appear in the corporate plan. |
Will be within one of the public good science output
classes approved by Government. |
|
Specific research objectives |
Research objectives will be derived from management needs. |
Research objectives need not be specific, rather goals or areas
or investigation may be specified. |
|
Time constraints included in project brief. |
Will be specific and derived from management needs. Generally ‘short-term’ projects. |
Unlikely the completion date of the programme will be
critical. |
|
Specified researchers |
There may be particular researchers who have become
identified with management needs. |
Alternative researchers may be envisaged. |
|
Duration of the research programme. |
Will depend on nature of managements needs but generally will
be within three years. |
Longer term projects. |
|
Integration with policy development. |
Mandatory that the project be closely integrated with
provision of specific output(s) of department |
Research may be integrated with policy development if this
is appropriate. |
|
Appropriation of the research findings. |
Research findings will be used by departmental
managers. Use to others of secondary
interest. |
Research findings may be of interest only to researchers, but
may also be used by managers in the public and private sectors. |
|
Number of beneficiaries |
Generally 1 or 2 only |
If more than 3, strong suggestion is output. |
55.
The indicative criteria were found to be of two kinds: those
that derived from the purposes of the research funding (appropriation of the
research findings, number of beneficiaries, integration with policy
development), and, those that were characteristics of the research itself
(specificity of research objectives, duration of the programme).
56.
There were two approaches to decision-making
considered.
On
17 October 1990 a committee of 14 people from ten government departments met
and considered the definitions and the indicative criteria and the proposed
strategy for decision-making on projects.
They:
a)
Commented on each of the indicative criteria in turn. Those that related to the content of the
research had little credibility. Examples
were the objectives for the research and the duration of the project.
b)
Were concerned about the future application of the
definitions being developed. It was
thought obvious that the definitions developed in the project would influence
future science funding decisions and the operation of the system itself. It is beyond the scope of the present paper
to pursue this.
c)
Ranked the indicative criteria in order of importance.
60. The definitions were modified to take account
of the committee’s concerns and the following version of the definitions was
used to review 370 projects on 19 October 1990:
DEFINITIONS FOR THE REVIEW OF EXISTING
PROJECTS
This research must:
a)
Satisfy the
definition of ‘public good science output’ given in the Foundation for
Research, Science and Technology Act 1990: and
b)
Be included
within one of the output classes in the attached list of 40 classes; and
c)
Be funded, at
present, in part or in full, by a vote.
This research will be closely integrated with the provision
of specific departmental outputs which are agreed between the Chief Executive
and the Minister (as defined in the Public Finance Act 1989).
When reviewing projects to determine if they are output or
operational research the following criteria are indicative of classification:
Dimension
|
Operational
|
Output
|
|
Appropriation
of research findings |
Research findings
will principally be used by departmental officials in the production of their
outputs |
Research
findings may be of interest or use to a wide range of persons |
|
Integration
of the research with departmental management or policy development |
Research
will be designed to include close interaction between officials and
researchers and integrated closely with other work in the department |
Research may
be designed without immediate reference to the presumed users of the research
outputs. |
|
Establishment
of the research |
Research
will be initiated by departmental managers (funders) who identify the need
for research, and the establishment of the research may resemble a tendering
system. |
Research may
be initiated by managers or researchers who focus on the desirability of
working in a particular area.
Providers, rather than funders, determine the research specification
to a considerable extent. |
|
Researchers
orientation |
Researchers
will be attempting to address departmental needs, and sometimes provide
outputs broader than research alone. |
Researchers
may have less regard for concern outside of the research itself. |
Note: The above list of features is indicative
only of a classificatory decision – and is in an order of importance.
61.
Information on each of the 370 projects had been provided in
writing by departments and the projects were related to the departments
corporate plans. The definitions were
applied without difficulty and the reviewers formed the view that there was
consistency in the decisions taken.
62. The
experience of using the definitions with a large number of projects resulted in
a proposed modification to the definition of operational research. The modification was really a rewording of the
alternate expressions considered earlier “must contribute to the delivery of an
output” and “will be closely integrated with the provision of an output.” It was found that there were a limited
number of outputs which generate legitimate proposals for departmental
research.
63. Hence,
it was proposed that operational research be redefined as follows:
Operational Research is research which
is funded by a department for one of the following purposes:
a)
To support
advice given to a Minister of the Crown;
b)
To support
departmental purchasing decisions;
c)
To facilitate
the execution of statutory duties;
d)
To support
departmental advisory services to industry or the community
e)
To enable NZ
to satisfy international obligations;
f)
To evaluate
policies or practice in state funded organizations (including departments).
64. At
this time it was decided that the 40 output categories would be re-defined to
ensure that they were comprehensive.
That is, they were to cover all possible areas of state funded research,
and hence included all academic disciplines and all sectors of the economy.
Consequently,
the questions “was it within 40 contestable pool output?’ was no longer
available as a first filter for decision-making on research projects, because
it would always be answered in the affirmative.
65. Given
the development described above, and the fact that it was found that the
definition of operation research was relatively straight-forward, it was
concluded that it was best to:
a) Define
operational research,
b) Define
contestable pool, or output research, as “research which was not operational
research”.
66. Earlier,
a set of rules was given for the development of stipulative definitions. Here the questions are asked: to what extent
were the rules followed in the development of definitions of operational and
output research, and, what conclusions may be drawn about the process of
developing stipulative definitions.
67. It
was necessary to deliberately adopt a strategy to develop the definitions for
three reasons:
a)
The terms ‘output’, ‘contestable pool’ and ‘operational’
were being used in many contexts and people admitted they were not clear as to
their meaning.
b)
The meaning of the terms affects important decisions about
the state funding of research in New Zealand.
The terms are central to the new approach to science administration.
And, hence
c)
It was desirable that people from many Government
Departments were involved in developing the definitions. This was for two reasons. First, the definitions had to be easily
applied throughout the state sector, and second, it was desirable that
departments saw that the definitions were neither arbitrary nor partial.
68. The
project resulted in a substantial simplification of the definitions because it
was realized that operational research could be defined in a very acceptable
manner and that it was unnecessary to persevere with the task of defining output research for it is all
research which is paid for by Government but which is not operational.
69.
The stipulative definition being advocated here are unlikely
to deceive, nor do they have any hidden emotional force. The process adopted to develop the
definitions ensured that they were neutral in all senses. Finally, it appears that the definitions
being suggested will be maintained readily because they have face validity and
are a direct refinement of the way that the terms operational and output were
being used by persons involved in science administration.
70. The
science policy project demonstrated that Robinson’s theoretical approach to the
development of stipulative definitions is helpful in practical situations. Galileo was wrong: all definitions are not
arbitrary. Those proposed as a basis
for decisions on science funding in New Zealand in 1990 are certainly not
arbitrary.
71. The
methods by which stipulative definitions may be established interact in complex
ways in many practical examples. This
was seen in the definitions, which are in New Zealand legislation, and the
project reported in the present paper provides further examples.
72. Policy
analysts concerned to bring a rational approach to the establishment of
critical definitions in the development of public policy should be able to make
direct use of the text on definitions, stipulation, and the methods of
stipulation.
73. It
is important that analysts do not use stipulative definitions in situations
where other approaches might be more fruitful.
[There is as tendency for people to attempt to resolve policy issues
quickly by the use of stipulation – watch that, you need to know when
conceptual analysis is called for.] To
assist with the identification of situations where stipulation is appropriate
some rules are given to guide those considering the use of stipulation. As a general first rule: only stipulate as
a last resort.
74. Given
that you are going to develop stipulative definitions in a critical situation,
it is important that you adopt procedures that will result in there being a
good level of acceptance of the definitions.
Stipulative definitions will not continue to be used without some
maintenance and the maintenance work is much easier if the audience for the
definitions has a good understanding of their development, importance, and
credibility. In the case of the science
project described, important funding decisions were made using these
definitions and they were challenged.
They withstood all challenges, and contributed to the acceptance of the
funding decisions.