Demands for transparency are central to the current culture of public debate. Climate scientists and their associations, the former English football captain, individual politicians and political parties, arts organisations, BBC broadcasters, financial institutions and local government have all been criticised in the past few days for not being transparent enough. The right of the public to question the decisions of those with power and demand disclosure of information from those in authority is essential for democracy, but transparency has become a double-edged sword in many ways, blunting all kinds of cultural distinction between the powerful and the ordinary person. This has led to demands for transparency of that entity we used to call the private citizen. These demands can only control the behaviour of the citizen in general and stifle the creativity that relies on cultural exchanges between people.

That transparency would threaten the privacy of the individual at first seems obvious because wanting to know something requires the disclosure of something that someone else may prefer to remain hidden. We have always had to balance the public interest with that of the privacy of the individual, depending upon the issue. But when those in authority take up the cry for more transparency, rather than those who seek to bring authority to account, the result is less likely to be the democratic investigation of powerful interests and more likely to be the undemocratic control of the individual. Transparency is less of a threat to privacy in the sense that more information about an individual may be gathered or demanded. It is a greater threat to privacy in the sense that the actions we take and the autonomous decisions we all make in whatever capacity become subject to a panoptical scrutiny with the retrospective ability to censure or punish. This has the potential to restrict action and narrow the basis upon which we act.

f the first problem with transparency is its adaptation for use by those in power, the second is that it frustrates the purpose of disclosure. The discovery of how a decision was made or why an action was taken is sought so that conclusions may be drawn on how to proceed in future. This process may be stalled by the general assumption that knowledge or understanding is the same thing as information. While everyone who makes a particular demand for disclosure, such as a Freedom of Information request, has some purpose or objective in mind, a cultural perception that all information must be known, in an unspecific way, scrutinises the act of decision-making rather than the decision itself. That is to say that all actions and thoughts become socially undifferentiated, of equal importance in the quest for transparency. Instead of the actions of organisations being judged by their consequences, they are judged by their processes, regardless of outcome. In this way the result of inquiries carried out in the name of transparency result in the call for further transparency instead of a change in actual policy.

This confusion of information and knowledge has two further difficulties. In an age of increasing management of information, and alongside our cultural preoccupation with “sending the right message”, it is understandable that we seek an ideal of the unvarnished truth, free of spin or vested interest. What bits of information are actually relevant to understanding an issue depend, though, upon the knowledge and understanding of the issue that we already have. For example, not being a scientist, what would I do if every piece of data on climate change was at my command? Even if I could assimilate it all, how would I make sense of it? Everyone (including scientists) depends upon a process of specialist social editing so that the expert opinions of one field of human activity can be assimilated by others in society.

The results of disclosure, when they blur the distinction between power and its critics and when they confuse the distinction between information and understanding, have less force to change the situation that is being addressed. When the focus of public scrutiny becomes an argument about disclosure and freedom of information, the public interest implicit in an issue becomes identified with the degree of transparency demonstrated. Public confidence becomes identified with the competence of information management displayed by the objects of scrutiny instead of their competence at whatever they are actually supposed to be doing.

The third problem with transparency is that it poses an ideal of disclosure that cannot realistically be met. However much is disclosed, there would always be the suspicion that something is hidden. There must also come a point at which further transparency encroaches on the privacy of individuals involved or affected by the events or issues scrutinised. That boundary line between the privacy to which any person can expect a right and what should be revealed in the public interest has been erased in the so-called Climategate affair.

The controversies about hacked e-mails, withheld climate change data and the scientific veracity of some predictions of environmental damage have encouraged an unhealthy kind of cynicism, suspicious of all information regardless of its sources. Those who plea for the public to consider the bigger picture of global warming and make distinctions between what is known and what is suspected by scientists are talking across a cacophony of recriminations and counter-recriminations. The general public’s job of critically appraising the evidence and opinions of climate scientists was difficult even before these controversies.

Climate scientists have predictably been told by the government’s chief scientific adviser that they must demonstrate more transparency. Each new investigation into who sent what e-mail to whom has compounded the reduction of political and cultural discussions into arguments about who can be trusted and who cannot, or about who is more or less selective in the presentation of information. Sections of the climate science establishment (for want of a better description) and some of the more militant climate change sceptics are responsible for discouraging proper public participation around these issues. Regardless of one’s opinions on the extent and causes of global warming, these continued calls for greater transparency are reducing the real debates about climate change to a question of confidence in scientists or campaigners. That is inadequate.

Freedom of information requests for scientific data were made of senior scientists whose public institutions provide and collect data and they may well wish they had responded differently. It’s no excuse to say that data requests were inconvenient or would have been misused by climate-change sceptics. So what? That’s democracy. But private e-mail accounts were illegally hacked into and this has led to a false polarisation of the debates around global warming. It encouraged the dumbest of conspiracy theories, on both sides, and has destroyed the distinction between what is private and what the public has a right to know. The more that distinction is ignored, the more difficult it will be to get secretive organisations, companies, government departments and other bodies to respond to requests for information and its sources in the future. The assumption that disclosure and exposure are magically able to transform public perceptions in themselves rests on an equally shaky assumption that there is some clear and common outcome of opinion among the general public once it has been presented with information.

Information is important and 100% of statistics cannot be wrong, but unless there is a strategy for the use of information, a system of understanding within which new data can be placed within a process of objective debate it cannot be put to good use. The hopes that greater disclosure would improve accountability and encourage confidence in institutions and more involvement in politics seem forlorn. Institutions of all kinds strive for transparency in their processes in the hope of promoting public confidence in them, but even the fullest possible disclosure cannot on its own combat cynicism or suspicion. Norman Lewis highlighted the distinction between the trust that people reserve for people they know or interact with and the confidence they may or may not have in institutions and organisations. At the very least, confusing different kinds of relationships makes it difficult to see where privacy is useful and where one might gain by surrendering it (see http://futures-diagnosis.com/?s=privacy). Institutions cannot make friends with you just because they show you all their emails, but they can make you more confident in them by doing what they are supposed to do.

For the Conservative party, the pledge of more transparency has rebounded badly. In a curious echo of Tony Blair’s old New Labour pledge that his administration would be whiter than white, the Tories have found that disclosure has no off button and that complete transparency is an impossible promise, before they have even formed a government. The Chilcot inquiry’s lack of revelation has done little to discredit the processes by which the Iraq War started. If anything, the hearings have increased the public’s lack of confidence in public inquiries and achieved the opposite of the government’s intention. Both the Tories’ election campaign and the government’s attempt to put the Iraq War behind it rested on the assumption that transparency, or at least an appearance of transparency, could somehow change the public perception that politicians lie and closely manage the truth.

If too much faith is placed in transparency then general principles, around which information should be organised both politically and culturally, are less contested and, perversely, we risk a more complete fragmentation of public opinion. If the public is treated as if mere information is required before the correct view of its significance can be arrived at, then attempts to engage the public with big ideas or really change their attitudes will fail. If the political and cultural framework within which information fits in the public mind is not addressed by both those with power and those who hold them to account, then information will continue to be chosen for its impact and ability to change opinion in only a very limited sense. If people’s views of the world remain pretty much the same while their opinions on particular issues can radically be influenced, then we all stumble forward unconsciously.

The desire of many organisations and campaigners to reduce complex questions to a simple, informational and moral polemic of transparency is the direct result of treating the public as an amalgam of suspicious, fragmented and cynical atoms. The public is not that fragmented and is not devoid of a cultural and political ambition to co-operate. Transparency seems a logical response to the managed information that bombards us from the all sides, but it rebounds upon us as private individuals. We have a general public comprised of different interests whose information needs are serviced by specialists in different fields. Part of the function of the different fields of activity in our society is to compress information so that it can be sent outwards and made useful in other areas for the public good. This process encourages innovation because it allows good ideas from one field to be applied in another. Everybody should be accountable in ways appropriate to their professions and they should not lie, but if everything everyone ever does in private is subject to public scrutiny in the name of transparency, then every individual risks censure and punishment for the slightest autonomous decision or innovation. The retrospectively applied threat implicit in today’s transparency cannot imagine all possible scenarios in the real world, but is ready to closely manage every action as if it could.

Sean Bell is secretary of the Brighton Salon.

Norman Lewis will speak about privacy, trust and confidence at the salon “The End of Privacy? The Future of Trust in the Transparent Society” Tuesday February 16