Consensus conferences and scenario work-shops have been developed in Denmark within a
democratic perspective that assumes it is both possible and necessary to establish a dialogue
with citizens about technology politics. In consensus conferences, the citizens have the role of
a citizen panel, which will set the agenda for the conference. In scenario workshops, a group of citizens interacts with other actors to exchange knowledge and experience, develop common visions and produce a plan of action. The focus of both methods is to create a framework for dialogue among policy-makers, experts and ordinary citizens. Both methods are also characterized by their ability to create new knowledge.
Consensus Conferences
A consensus conference is a public meeting, which allows ordinary citizens to be involved in the assessment of technology. The conference is a dialogue between experts and citizens. It is open to the public and the media. Usually it is attended by some members of the Danish Parliament.
The citizen panel plays the leading role: it consists of about 14 people who are introduced to the topic by a professional facilitator. The citizen panel formulates the questions to be taken up at the conference, and participates in the selection of experts to answer them. The panel has two weekends for this preparation.
The expert panel is selected in a way that ensures that essential opposing views and professional conflicts can emerge and be discussed at the conference. Good experts are not only knowledgeable but also open-minded and good communicators with an over-view of their field.
An advisory/planning committee has the overall responsibility of making sure that all rules of a democratic, fair and transparent process have been followed.
On the first day of the conference, the experts present their answers to the questions from the citizen panel, from the point of view of their field of expertise. The following morning is reserved for clarifying questions and for discussions between the expert panel, the citizen
panel and the audience. The rest of the second day and the third day are reserved for the citizen panel to produce a final document, presenting their conclusions and recommendations. Consensus on attitudes and recommendations is achieved through open discussion.
Thus the final document is an expression of the extent to which the citizen panel can reach
consensus.
On the morning of the fourth day, the citizen panel reads the final document to the experts and the audience, including the press. The experts have the opportunity to correct misunderstandings and factual errors, but at this point they are not allowed to influence the
Views of the citizen panel.
Scenario workshops
A consensus conference focuses on society’s use and regulation of technology; a scenario workshop starts with a problem looking for solutions. The solutions can be technological, regulatory or maybe a new way of organizing and managing certain problems.
A scenario workshop is a local meeting that includes dialogue among four local groups of actors: policy-makers, business representatives, experts and citizens. The participants carry out assessments of technological and non-technological solutions to the problems, and develop visions for future solutions and proposals for realizing them.
Before the workshop can take place, a set of scenarios is written, describing alternative ways of solving the problem. They have to be different with respect to both the technical and organizational solutions described and the social and political values embedded in them.
In the workshop, the scenarios are used as visions, and as an inspiration in the process. The participants are asked to criticize and comment on them to enable them to develop visions of their own —not to choose among, or prioritize, the scenarios.
The workshop process, which may last for one or two days, has three principal steps:
To comment on, and criticize, the scenarios by pointing out barriers to realizing the visions
To develop the participants’ own visions and proposals
To develop local plans of action.
The process is guided by a facilitator and takes place in ‘role’ groups, ‘theme’ groups and plenary sessions. Dialogue among participants with different knowledge, views and experience is central. Various techniques can be employed to accomplish good dialogue
and the production of results in the form of identification of barriers, of visions and of proposals for action to be taken.
The Danish Board of Technology has used the scenario workshop method for three topics. First, was “Urban Ecology”, 1991–93, which is treated in more detail later. Second was “The Library of the Future”, 1995–96. Third was “The Future of Education”, 2000-2001.
This article describes the format of the two methods, giving examples and experience from practical projects as a background to discussing some strengths and weaknesses of each method. Expectations of future development in Denmark and other countries are discussed.
Institutional setting
Consensus conferences and the scenario workshops were developed by The Danish Board of Technology (DBT), which is an independent institution established by the Danish Parliament. It is financed by the state with 13.5 million Dkr (approximately 1.6 mill ECU) a year, and has a permanent staff of 13. The DBT has, since it was established in 1986, experimented with, and developed, participatory methodologies, which allow ordinary citizens to be involved in technology assessment. According to the Law (no 375, 14 June 1995) that is the basis of the Board, its work has two main purposes:
q To initiate independent technology assessment by carrying out investigations and comprehensive assessments on the possibilities and consequences of technology for society and the citizens.
q To further a public debate on technology by communicating the results to Parliament, other political decision-makers and the Danish population.
A basic principle in the work of the DBT is that technology assessment should: include the wisdom and experience of ordinary citizens/lay people; integrate the knowledge and tools of experts; respect the political processes and the working conditions of policy-makers; and build on the democratic tradition in Denmark (Klüver, 1995, page 41).
Cultural context
DTB’s understanding of technology assessment has a background in Danish democratic traditions (Andersen and Jaeger, 1997; Joss, 1998; Klüver, 1995). As technology becomes more and more integrated, influencing more circumstances in life, citizens should have a democratic right to influence its development (Bijker, 1993; Sclove, 1995). This viewpoint opens a discussion about democracy and technology assessment.
The most widespread definition of democracy is closely linked to the national state and for many years seems to have functioned adequately. However, because of the reduction in Parliament’s power, partly because of decisions in the European Union (EU) and partly because of the decentralization to local authorities, Parliament’s role as a forum for societal decision-making has weakened. Thus we have seen signs of crises of the representative democracy in the last few years (Sørensen et al, 1996). An-other weakness concerns who is actually represented by Parliament. Members of Parliament are selected from the political parties, but the number of Danes who are members of a political party has been decreasing
in the last couple of decades. The ideals of the representative democracy are built on what some political scientists have described as “aggregative political processes” (March and Olsen, 1989), in which, the will of the people is discovered through bargaining among rational citizens
who are pursuing self-interests within a set of rules for governance by majority rule. Order is seen as based on rationality, and the primary outcome of a political process is the public policies and the allocation of resources. Leadership involves the arrangement of coalitions among interests.
There is also a tradition of participatory democracy, in which citizens take part in political processes. Participation takes several forms, from citizens just expressing their opinion without any kind of obligation, to ‘user-boards’ in which they have a high degree of responsibility, actually becoming engaged directly in running a local institution such as a school.
In this light, we can see the work of the DBT as a way to develop methods combining both forms of democracy. Both consensus conferences and scenario workshops involve citizens. Their task is to create solutions, recommendations, develop visions and express demands to be used in relation to political decision-making. The two methods may overcome some of the limitations of both concepts of democracy, and in some cases may build bridges between the
local, national and even the international level.
Consensus conferences in practice
Purpose of organizing a consensus conference
In this period of history, the development of science and technology continually provides us with good and relevant topics for a consensus conference. For instance, in medicine and the environment, society is confronted with difficult decisions, involving conflicting attitudes to technology. Questions concerning what we, as a society, want to use technology for, if we want to use it, and how the use of it should be regulated?
Danish politicians, according to the experiences of the DBT and a recent study, are satisfied with the opportunity given by the consensus conferences to obtain information on how ordinary citizens consider these difficult questions. The conferences provide the politicians with information, which they normally do not have, either from the media or from the experts
themselves. Also, politicians know the limitations of expert knowledge and the possible interests and biases involved in experts’ assessment. Therefore they use results and final documents, for example, when they need to find out where the most important conflicts are, related to a given decision on technology.
Topics
During the last thirteen years, the DBT has conducted more than 20 consensus conferences, for example on: irradiation of foods (1989); how we should apply the increasing knowledge on human genes (1989); the future of motoring (1993); treatment of infertility (1993); limit values and risk assessment —chemicals in food and environment (1995); sustainable consumption
(1996); telework (1997), genetically modified food, electronic surveillance, road pricing and gene testing, 1999-2002.
According to the DBT experience a good conference topic is: of current interest; requires expert knowledge, which is also available; is possible to delimit; and involves conflicts and unresolved issues regarding attitudes to questions such as applications and regulation
Recruitment for citizen panel
The recruitment and composition of the citizen panel is a crucial factor in a consensus conference. It is not possible to create a 14-person panel, which is representative of the Danish population. The DBT recruits participants by sending invitations to a random sample of 2000 people. Those who want to participate must write a letter to the DBT with some information about themselves and their motives to be part of the panel. Among these (about 120–150 applicants) the citizen panel is composed of participants with varied backgrounds regarding age, gender, education, occupation and geographical location. The citizen panel
should consist of non-experts, but they are expected to be able to raise critical questions and to work independently.
Consensus and conflict
It is especially important to obtain consensus in relation to two phases. First, concerning the selection and formulation of main questions to be asked of the experts. These questions also constitute the agenda for the whole conference. Secondly concerning the conclusions and recommendations in the final document, which is another visible product of the work of the whole panel.
Sometimes the conclusions reached by the citizen panel are controversial according to other groups in society. For example, in a conference on “The future of motoring” the main recommendation of the panel —14 members of which 11 were car owners —was, gradually to double the price of gasoline to reduce car traffic. This did not lead to a doubling of gasoline prices, although the question is still alive and discussed. There are too many ‘holy cows’ involved in it, which the discussion after the conference showed very clearly. However, it showed the ability of the method and the process to lift the participants out of the selfish way of thinking as car owners, and put broader perspectives on the problems of transport and the environment.
In a conference on infertility, the citizen panel recommended that sperm donors should not be allowed to stay anonymous, if the child later wanted to know whom its father was. This is an example of how sometimes a lay panel can reach conclusions which are not evident or easy.
Strengths and weaknesses
Can recommendations from consensus conferences make a difference? Yes, they almost certainly will, but not always in the way envisaged. Since the DBT conference on irradiation of food, Denmark has prohibited this kind of preservation. The question is regarded as controversial, for instance, in today’s debate on food safety.
Since the DBT conference on mapping the human genome (1989), Denmark has prohibited companies from claiming a DNA health profile from their staff and from job applicants.
Consensus conferences represent an opportunity to hear the voices of people who normally are not asked about their attitude to technology. An important contribution from the conferences is the information given to politicians, experts and society as a whole on the ideas and concerns of ordinary citizens. This is, in our opinion, the most important aspect of new knowledge produced in the whole process.
The history of consensus conferences also shows many examples of recommendations that were not listened to, and results which have no documented impact on technology politics. The question is whether the most important results of the conferences are their direct impact on specific decisions, or that they communicate new knowledge to politicians on ideas and concerns of ordinary citizens, and new self-confidence and awareness to lay people on the role they may play in setting an agenda for society’s debate on technology.
No matter how good or democratic the contributions that consensus conferences give to
decision-makers, they are of no use, if the development or application of new technology is not an object of political decision-making, but designed and decided on far away from both the public and the politicians.
Consensus conferences do not promise any miracles, and it is very important to make this clear to the citizens before engaging on the work. They will have to see themselves as advisors to Parliament: voluntary and unpaid advisors, without any guarantee that their efforts
will be taken into account in future decision-making.
Scenario workshops in practice
Example of urban ecology, 1992–93
The scenario workshop method was developed in the early 90s, very closely related to the need for new and integrated ways of handling environmental problems. In 1991, the DBT agreed on “sustainable housing and living in the future” as a topic for a new project. As a
preparation for the Rio Conference in 1992, the Government had formulated an official policy on environmental protection and sustainable development in all areas of Danish society.
In this way, the project could benefit from a broad political consensus concerning the need to develop and transform cities and urban communities in a way which was ecologically sustainable. The concept of urban ecology became a point of departure to help the project formulate more concrete ideas of what was needed in an overall effort towards sustainable development. Urban ecology in the DBT project was defined as the interaction between people and nature in urban areas. To think and act in an ecological way implies saving resources, recycling and reusing products and materials and returning used materials to nature in a clear form.
It soon became clear that this project was dealing with an extensive process of societal transition, which obviously cannot take place overnight. The project had to comprise the whole technical infrastructure for energy, water, waste water and solid waste management, as well as daily life, habits and values of all the involved actors, including residents.
Urban ecology is concerned with the interaction among different types of technology, various involved actors (organized and individuals), different criteria for assessing technology, different types of knowledge, a broad spectrum of laws and rules from different authorities, and various places and levels of action and several possible solutions.
This multitude of aspects is what mostly confronts us as citizens in a technological world. The problem focus of the scenario workshop method, together with its emphasis on local problems and local solutions makes it necessary to handle multi-technological and even non-technological problems. This broad and open approach has been mentioned as a specific advantage of the method, because it is well suited for handling local problems and is open to citizens’ visions on innovation and technological design (Sclove,
1997).
Workshop structure
Scenario workshops were conducted in four local communities during 1992. The criteria for choosing the communities were that there should be some positive effort and experience regarding urban ecology, and that the four places should be of different size and urban development.
Each participant took part in two workshops with 20–25 participants. First, there were ‘role group’ workshops, where participants from the same role group, for example, business people, but four different localities met. The task was to develop visions using the scenarios as an inspiration. The cross-local dialogue gave new knowledge on barriers and new ideas on visions, both to participants and to organizers. Reports from the first workshops were used as
input for the next round —local workshops, arranged in the four local communities.
At the local workshops, participants were split into ‘theme groups’ according to experience and interests: the task was to agree on common vision and produce local action plans for energy, water and waste. The results from these workshops were evaluated and were competent for public and political debate. The outcome was a report and a national plan for urban ecology, which was presented at a public conference in January 1993. Subsequently, this was partly implemented by the Danish Minister of the Environment (Ministry of Environment, 1994).
Results
The results of the workshop were threefold:
q Barriers to urban ecology were identified
q Visions were developed and
q Action plans were proposed.
Results from the project in all these fields have played an important role in the Danish debate on sustainable housing and planning during the years after the conference.
The following give an idea of some of the post-project changes:
q 1993: the Minister of Environment established a national committee on urban ecology, inspired by recommendations from the national action plan.
q 1995: the Urban Ecology Committee decided to establish a Danish Centre of Urban Ecology (to support experiments and give advice to local activities), and a Green Foundation to finance activities such as the Ecological Council and the association of Green Families.
q The DBT has a fund to supply grants for local activities. It has supported hundreds of local meetings with material about urban ecology and money to arrange the meeting.
q Today the scenario workshop method is widely used in education, research and consultancy.
q The public debate in general has developed scenarios to solve urban ecology problems towards more awareness of the importance of urban ecology principles to be integrated in regulation and law making.
An evaluation among all participants shortly after the project showed that the experience had been important learning exercise and paved the way for better dialogue at local level. However, the DBT has not followed the long-term changes resulting from this project in the four communities.
Strength and weaknesses
The role of citizens in a scenario workshop is somewhat different from their role as lay people in a consensus conference. In scenario workshops, the citizens are a group of actors among other groups. The experience and vision of all the actors contribute to the proposals and plans of actions resulting from the workshop. All groups contribute with their knowledge and experience from local activities, for instance, as local residents, business people and so on. They can all be regarded and defined as experts, because local experience and knowledge is a crucial factor in this locally oriented method.
Our environmental future depends on a joint effort from all members of society. It can only partly be planned and regulated top-down by experts and policy- makers. Therefore the involvement of many citizens in vision making, identification of barriers, development of ideas, proposals and plans has to be stressed as an important advantage of the scenario workshop
method. The method can contribute to better decisions in a field, where future changes depend on the engagement and participation of many citizens.
Furthermore, the workshop process tends to bring people together, who do not usually engage in dialogue, even if they live in the same place. This is a precondition for breaking down ‘stereotyped images’, which can sometimes be an obstacle to finding solutions.
The advantage of local participation also may have a reverse side, because the results may not be able to be used more general level. More experience (more than one workshop) may be needed, as was the case in the Danish urban ecology project, to produce results which can be generalized and used by other local communities or at national level. This is a question of time and money.
The scenario workshop method needs good preparation, planning and facilitation. It may also need effort from the organizers to document and present the results in a structured way, if they are going to be used as an input for decision-making both at local level and more generally. It is not always easy to interest the press and politicians in the results of local scenario workshops.
Therefore, the success of this method depends on the existence of a ‘customer’ — somebody at local, national or even international level, who needs the results and wants to use them. This makes the process very vulnerable, because who will buy a product, which cannot be exactly described, maybe even predicted, in advance?
Use of participation in other countries
In 1992–93 a European Commission project (EC DGXIII, the Innovation Programme) was looking for an appropriate method for establishing and improving communication between different sectors in society such as the scientific community, the political community, and the public at large. The purpose was to bring the EC R&D programme more in line with the future needs of society.
The scenario workshop method was selected for a pilot study, which was conducted in 1993 and 1994 in four European cities: Ede (the Netherlands), Corfou (Greece), Preston (UK) and Mulhouse (France). This was followed by several scenario workshops in other European cities, for instance, for Local Agenda 21 projects. Training courses for National Monitors from all EU countries were arranged and followed by the production of a ready-to-use package for local organizers (EU Innovation Programme, 1999).
The Innovation Programme has also explored the possibility of applying the method in other areas, such as the provision of information and communication technology (Andersen et al, 1995; 1996). In 1997-99, this work was part of the so-called Fleximodo project, in which partners from the Netherlands, Portugal, Italy and Denmark collaborated to design new scenarios for local mobility, urban regeneration and local in-formation and communication. The general idea was to create more flexible and modular applications for the method, so that local communities can be provided with tools and procedures to organize scenario workshops, which are tailored according to the local needs and interests. The Fleximodo project was finished at the end of 1998.
In 2002 the first American scenario workshop will be held in the city of Lowell, where it is welcomed by the mayor and city planners, who hope that it will give a useful contribution to their work on a comprehensive plan for the next 20 years. The project is hosted by the University of Massachusetts Lowell and the Loka Institute and is financed by the National Science Foundation. This project will introduce a new development to the method, as the scenarios and technologies will be changed by a set of democratic criteria questions introduced by Richard Sclove, who obtained the grant from the NSF.
The consensus conference method has been applied to more than 30 conferences in other countries:
Britain, the Netherlands, Norway, Austria, New Zealand, Japan, Switzerland, Australia, Canada, U.S.A. and South Korea.
Conclusion
The two methods we have described represent a framework, which makes it possible to involve ordinary citizens in technology assessment and thus give an input to society’s decisions on technology.
The scenario workshop method needs good preparation, planning and facilitation: it may also need effort from the organizers to document and present the results in a structured way, if they are going to be used as an input for decision-making. It has been shown that results from consensus conferences and scenario workshops have had some direct impacts on decisions taken. More important, though, in our view is their indirect influence by giving politicians new knowledge about citizens’ discussions of the threats and opportunities of technology, and giving citizens new knowledge and awareness. In general, it is difficult to measure and document both direct and indirect impact of the two methods.
The two methods offer a new way of hearing ‘the voice of the people’. We see this as a supplement to well-established, well-known ways, such as elections, referenda and opinion polls. The established ways show us the will of the people regarding a well-structured set of alternatives. The two methods cannot claim to express the voice of the whole people, but they offer an opportunity for citizens to present their ideas and opinions in a more open way, which they have the opportunity to influence and structure themselves.
This is important, because society is full of people —experts, technocrats, politicians and so on —who have time and resources to set the agenda for public debate on technology. If these methods are able to counterbalance this, just a little, this is a very important advantage.
We have pointed to some of the strengths and weaknesses we see in each method, and have shown that the objectives to be reached by using the methods are common in some respects, and differ in others.
Consensus conferences usually concern a particular technology. It should be easy to delimit and arrange dialogue between experts and the public to clarify some of the conflicts involved. The conclusions and recommendations are used by politicians for debate and regulation and are widely accepted as an input of ‘the voice of the people’. Therefore the recruitment and composition of the citizen panel is crucial in this method.
In the scenario workshop, the topic is formulated as a problem, for instance, a local issue, which cannot be solved without the participation of local people. The workshop is designed to find solutions, technical or not, to the problem. The design of scenarios is a crucial challenge in the scenario workshops. The results can be used to make better and longer-term solutions to problems with many actors and technologies involved.
It is not possible here to evaluate all relevant elements of the two methods. Both have attracted a good deal of interest from national and international institutions and policy bodies. We see this as a sign that in technology assessment there is a general need for policy-makers to involve the public in decisions, to compensate for the deficit of both the market and the political system. This need is partly met by the two methods, which represent well-documented, convincing and tested efforts to involve citizens in technology assessment. In the DBT, this experience is used to become conscious that all such methods are conditioned by the cultural context of their origin and must be adapted for use in a different cultural context.
What also becomes more and more clear, from both the Danish experience and initiatives in other countries, is that there is one, indispensable criterion for success, for real changes to take place. This is that the policy-makers, to whom the results are addressed, have to be able and willing to listen and take the results seriously as proposals from the public. This also means that the institution organizing the projects must enjoy credibility with the public. If this is not so, it will be more difficult first, to find participants prepared to give the required time, and second, to make the politicians listen to the outcome.
The most important future perspectives on using participatory methods in our opinion are:
q Development of scenario workshops so that citizens can be involved earlier, at the stage of design and selection of criteria for developing technology.
q Strengthening the power of consensus conferences to present the broad approach of citizen panels as just as legitimate a criterion in technology decision-making as the more narrow approach of experts.
q Both methods have demonstrated great capacity for creating awareness of methodological innovations and diversification, and for building networks.
Notes
- More detailed information and evaluation of the method is available on the internet (DBT, 1999) and in Joss and Durant (1995).
- See Joss (1998) which gives extensive documentation of the political impact of consensus conferences in Denmark.
- Summary in “The future of motoring — results from a consensus conference”, no 3/1993.
- Summary in “Consensus conference on infertility, conclusions of the lay panel”, November 1993.
- For further documentation on both the direct and more indirect impact of consensus conferences see Joss (1998).
- More detailed descriptions of methods and results can be found in Andersen et al (1992; 1993; 1995).
- The scenarios are also available in 11 languages on the internet, EU Innovation Programme (1999).
- For a detailed description of the use of the method in this context see Mayer (1997, chapter 5). See also Bilderbeek and Andersen (1995).
- Fleximodo has developed its own homepage, Fleximodo (1999).
- Documentation and discussion on this can be found in Joss and Durant (1995) and Joss (forthcoming).
References
I-E Andersen and Birgit Jaeger (1997), Involving Citizens in Assessment and the Public Debate on Information Technology (TMV, University of Oslo).
I-E Andersen, L D Nielsen, M Elle and O Danielsen (1992), “The scenario workshop in technology assessment”, paper presented at The Third European Congress on Technology
Assessment, Danish Board of Technology, Copenhagen, Denmark.
I-E Andersen, L D Nielsen, M Elle and O Danielsen (1993), Byøkologiske øjebliksbilleder. Visioner, barrierer og muligheder for at handle, report from Danish project (Danish Board of Technology, Copenhagen).
I-E Andersen, L Klüver, R Bilderbeek and O Danielsen (1995), “Feasibility study on new awareness initiatives. Studying the possibilities to implement consensus conferences and scenario workshops” (European Commission, DG, Interfaces III, Brussels). See also DBT (1999) and EU Innovation Programme (1999).
I-E Andersen, S Stripp, R Bilderbeek and J Geurts (1996), “The local information society, development and descriptions of possible scenarios for the assimilation of the new information
technologies by the European society in the next decades” (EU Innovation Programme, European Commission, DGXIII, Brussels). See also EU Innovation Programme (1999).
Wiebe E Bijker (1993), Dutch, Dikes and Democracy, Technology Assessment Texts no 11 (Technical University of Denmark, Lyngby).
R Bilderbeek and I-E Andersen (1995), “Raising awareness among citizens: experience from European local scenario workshops on sustainable urban development”, EPTA Newsletter, 10.
M Elle (1992), Byøkologiske Fremtidsbilleder (Scenarios on urban ecology) (Danish Board of Technology, Copenhagen). See also EU Innovation Programme (1999).
EU Innovation Programme, DGXIII (1999), http://www.cordis.lu/innovation/home.html , with self-training hypertext and CD multi-media slide show, in 11 European languages.
Fleximodo (1999), http://www.cittadellascienza.it/fleximodo/fleximodo.html , is under development
and will be gradually updated from the Fleximodo project.
Toini S Floris and Charlotte Bidsted (1996), Brugerbestyrelser på tvaers —erfaringer fra kommuner og amter (User-boards across—experience from municipalities and counties) (AKF Forlaget, Copenhagen).
S Joss (1998), “Danish consensus conferences as a model in participatory technology assessment: an impact study of consensus conferences on Danish Parliament and Danish public debate”, Science and Public Policy, 25(1), pages 2–22.
S Joss (forthcoming), “Participation in parliamentary technology assessment: from theory to practice”, in N J Vigand Hpaschen (editors), Parliaments and Technology: the Development of
Technology Assessment in Europe (State University of New York Press, New York).
S Joss and J Durant (editors) (1995), Public Participation in Science: the role of consensus conferences in Europe (Science Museum, London).
L Klüver (1995), “Consensus conferences in the Danish Board of Technology”, in Joss and Durant (1995).
James G March and Johan P Olsen (1989), Rediscovering Institutions (The Free Press, New York).
Igor Mayer (1997), Debating Technologies, A Methodological Contribution to the Design and Evaluation of Participatory Policy Analysis (Tilburg University Press, Tilburg).
Ministry of Environment (1994), “Byøkologiske anbefalinger”, Betaenkning fra det rådgivende udvalg om byøkologi (Urban ecology recommendations) (Ministry of Environment, Copenhagen).
R E Sclove (1995), Democracy and Technology (Guilford Press, New York, London).
R E Sclove (1997), “Using democratic design criteria in participatory technology assessment”, (unpublished).
Eva Sørensen, Allan Dreyer Hansen and Carsten Greve (1996), Demokrati i forandring (Democracy in Change) (Projekt Offentlig sektor, Copenhagen).
This article is a revised and updated version of an article which was first published in Science and Public Policy, October 1999, Vol. 26, No. 5, PP331-340.