To say that John Brockman is a literary agent is like saying that David Hockney is a photographer. For while it's true that Hockney has indeed made astonishingly creative use of photography, and Brockman is indeed a successful literary agent who represents an enviable stable of high-profile scientists and communicators, in both cases the description rather understates the reality. More accurate ways of describing Brockman would be to say that he is a "cultural impresario" or, as his friend Stewart Brand puts it, an "intellectual enzyme". (Brand goes on helpfully to explain that an enzyme is "a biological catalyst – an adroit enabler of otherwise impossible things".)

The first thing you notice about Brockman, though, is the interesting way he bridges CP Snow's "Two Cultures" – the parallel universes of the arts and the sciences. When profilers ask him for pictures, one he often sends shows him with Andy Warhol and Bob Dylan, no less. Or shots of the billboard photographs of his head that were used to publicise an eminently forgettable 1968 movie, . But he's also one of the few people around who can phone Nobel laureates in science with a good chance that they will take the call.

Cynics might say that this has something to do with the fact that Brockman has a reputation as an agent who can extract massive advances from publishers. (There's a story, perhaps apocryphal, that he regards books that earn out their advances as failures because it means he didn't ask for enough in the first place.) And he is indeed a hustler who spotted early on that there was a massive audience for writing about science, but there's more to it than that. Brockman is genuinely passionate about big ideas. He is fascinated, he told Wired magazine, "by people who can take the materials of the culture in the arts, literature and science and put them together in their own way. We live in a mass-produced culture where many people, even many established cultural arbiters, limit themselves to secondhand ideas. Show me people who create their own reality, who don't accept an ersatz, appropriated reality. Show me the empiricists – and not just in the sciences – who are out there doing it, rather than talking about and analysing the people who are doing it."

Brockman's immersion in both sides of the Two Cultures runs deep. He did an MBA at Columbia in the early 1960s and started his own financial leasing company on Park Avenue. But a friend introduced him to avant-garde theatre, thereby launching him on the primrose path into the arts.

He then got involved in the city's underground movie scene, becoming manager of the Film-Makers' Cinematheque, the home of underground cinema, in 1965, where his mandate was to produce a festival that expanded the form of cinema. He commissioned 30 performance pieces by world-class artists, dancers, poets, dramatists and musicians. The resulting festival made a big splash. "Intermedia", the term Brockman coined and used as his logo, was suddenly hot. A number of notable art-world figures were immersed in the genre, among them Les Levine, the "USCO" Group, La Monte Young, Terry Riley, Claes Oldenburg, Robert Rauschenberg, Andy Warhol (which explains the photograph), several kinetic and "happenings" artists, avant-garde film-makers and dramatists, the Velvet Underground, and composer John Cage.

This immersion in New York's arts scene also led to deep interest in science and technology. Many of the pieces at the festival were informed by artists' interest in cybernetics. They were reading and discussing books by scientists. (So much for CP Snow's castigation of arts folk for their indifference to science!) Rauschenberg suggested to Brockman that he read George Gamow's One, Two, Three… Infinity and The Mysterious Universe. Gerd Stern, co-founder of media collective USCO, who performed in conjunction with talks by Marshall McLuhan, introduced Brockman to several scientists and eventually arranged for him to meet McLuhan and his colleagues.

In cyberspace, Brockman is best known for Edge.org, a site he founded as a continuation of what he describes as "a failed art experiment" by his late friend, performance artist James Lee Byars. Byars believed, Brockman recalls, "that to arrive at a satisfactory plateau of knowledge it was pure folly to go to Widener Library at Harvard and read six million books. Instead, he planned to gather the 100 most brilliant minds in the world in a room, lock them in and have them ask one another the questions they'd been asking themselves. The expected result – in theory – was to be a synthesis of all thought." But it didn't work out that way. Byars did identify his 100 most brilliant minds and phoned each of them. The result: 70 hung up on him!

Byars died in 1997, but Brockman persisted with his idea, or at any rate with the notion that it might be possible to do something analogous using the internet. And so Edge.org was born as a kind of high-octane online salon with Brockman as its editor and host. He describes it as "a conversation. We look for people whose creative work has expanded our notion of who and what we are. We encourage work on the cutting edge of the culture and the investigation of ideas that have not been generally exposed."

As of now, the roll call of current and deceased members of the Edge salon runs to 660. They include many of the usual suspects (Richard Dawkins, Craig Venter and Stewart Brand, for example, plus Daniel Dennett, Steven Pinker, George Lakoff, Daniel Kahneman, Nassim Nicholas Taleb, Murray Gell-Mann, Nick Humphrey and Richard Thaler, to name just a few.). It's a predominately male crowd, with women accounting for only 16.5% of the members – which is probably a reflection of the fact that science is still largely a male-dominated business. There are a lot of what one might call the "digerati" – the Clay Shirkys, Douglas Couplands, Steven Johnsons and Howard Rheingolds of this world. Two generations of the Dyson clan are represented – the great physicist Freeman and his two kids, Esther and George. Edge seems biased towards the Anglo-Saxon world; at any rate, there are surprisingly few continental Europeans or Asians. Brits, on the other hand, figure prominently: names that stand out include those of Brian Cox, Charlie Leadbeater, Colin Blakemore, David Goodhart, John Barrow, Karl Sabbagh, Martin Rees, Mark Pagel, Lewis Wolpert, Matt Ridley, Patrick Bateson, Simon Baron-Cohen, Ross Anderson, Tim Berners-Lee and Helena Cronin, for example.

Asked how he had assembled this intriguing posse of thinkers, Brockman replied: "It's all based on word of mouth and reputation. Edge, contrary to how it may appear, is not exclusive. Elitist, yes, but in the good sense of an open elite, based on meritocracy. The way someone is added to the Edge list is when I receive a word from a Steven Pinker, a Brian Eno, a Martin Rees or a Richard Dawkins, telling me to do so. It's as simple as that and I don't recall ever saying no in such circumstances."

Ever since it appeared online, Edge.org has consistently been one of the most thought-provoking and interesting sites on the web. As I write, the front-page lead is an extraordinary piece by the evolutionary biologist Mark Pagel in which he argues that humans' capacity for social learning has made us less intelligent than we like to think we are. "If I'm living in a population of people," he writes, "and I can observe those people, and see what they're doing, seeing what innovations they're coming up with, I can choose among the best of those ideas, without having to go through the process of innovation myself. So, for example, if I'm trying to make a better spear, I really have no idea how to make that better spear. But if I notice that somebody else in my society has made a very good spear, I can simply copy him without having to understand why.

"What this means is that social learning may have set up a situation in humans where, over the last 200,000 years or so, we have been selected to be very, very good at copying other people, rather than innovating on our own. We like to think we're a highly inventive, innovative species. But social learning means that most of us can make use of what other people do and not have to invest the time and energy in innovation ourselves."

This essay is a perfect illustration of Brockman's idea of what Edge.org should do: to serve as a forum for big, intriguing and/or disturbing ideas advanced by intellectuals who have a track record of major achievements in their fields. He doesn't seem to have much time for the scholar who crawls along the frontiers of knowledge with a magnifying glass.

This philosophy is also what drives one of his annual rituals. Every year, on the anniversary of the launch of the site, he poses a question and invites Edge participants to answer it.

What kinds of question? "Questions that inspire answers we can't possibly predict. My goal is to provoke people into thinking thoughts they normally might not have." In previous years, the questions have included:

What do you believe even though you cannot prove it? (2005)

What is your dangerous idea? (2006)

What are you optimistic about? (2007)

What have you changed your mind about? (2008)

What will change everything? (2009)

In 2010, Brockman's question was: "How is the internet changing the way you think?" He received 172 replies in the form of mini-essays of varying length. These were published on the Edge site in the usual way, but 150 of them have now been collected between hard covers under Brockman's editorship. The result: a whopping hardback, How is the Internet Changing the Way You Think? The Net's Impact on Our Minds and Future, published by Atlantic Books.

Reading it over Christmas, I was intrigued by the book and emailed John Brockman to discuss some of the thoughts it evoked. What follows is an edited transcript of our exchanges.

JOHN BROCKMAN: the net, elites and life on the Edge

John Naughton I see you've been variously described as a "cultural impresario" and an "intellectual enzyme". How would you describe yourself?

John Brockman Wallace Stevens wrote in his poem "Man With the Blue Guitar":

Throw away the lights, the definitions,
And say of what you see in the dark
That it is this or that it is that,
But do not use the rotted names.

Any attempt to describe myself would end in awkwardness, confusion and contradiction. Also, I like to keep changing the subject, to surprise myself.

JN What's your intellectual background? From which of the original "Two Cultures" do you come? I'm an engineer, so this two/three cultures stuff really resonates with me.

JB In 1944, at three and a half years old, I was stricken with spinal meningitis and was in a coma for six weeks at Boston's children's hospital. The doctors had given up on me when, unexpectedly, I opened my eyes. I am told the first thing I said was: "I want to go to New York."

I arrived there at age 20 in 1961 for graduate school at Columbia. I was immediately struck by, and impressed with, the argumentative and exciting culture in which conversations were being carried out month after month in the pages of literary magazines such as Commentary, Partisan Review and the UK's Encounter.

For a dollar or two, I was privileged to look over the shoulders of the intelligentsia of the day – Lionel Trilling, Stephen Spender, Hannah Arendt, Alfred Kazin et al – as they went at one another over important issues such as the Eichmann trial and/or more trivial pursuits as to who slept with whom on a particular Bloomsbury weekend or who was still a Stalinist after the purge trials of 1937.

It's interesting to note that while I was ostensibly at Columbia to study economics and finance, my interests and instincts were strictly cultural and I made the most of the resources of a great university and New York City to educate myself in the areas that interested me and also to situate myself in the milieu where the action was taking place.

JN How did you get involved in the arts?

JB I quickly realised, but did not articulate, something the anthropologist Gregory Bateson told me 10 years later: that of all our human inventions, economic man was by far the dullest. A friend suggested I come downtown at night and help out at Theatre Genesis, an off-Broadway theatre in St Mark's in the Bowery, the avant-garde church that also was home to a bustling poetry centre.

So every night I would show up in my three-piece banker's suit and help set up the theatre. Working with me were the 21-year-old Sam Shepard, a young playwright from the midwest, and his room-mate, Charlie Mingus Jr.

One of the artists I got to know was the poet Gerd Stern, who had, on occasion, collaborated with Marshall McLuhan, incorporating live McLuhan lectures into USCO intermedia performances. Gerd, with his unkempt hair and abundant beard, was an odd counterpoint to the buttoned-down classics professor from Toronto, but they got along famously. Through Gerd and other artists, McLuhan's ideas had begun to permeate the art world, though it would be several more years before they hit the mainstream.

Gerd introduced me to the anthropologist Edmund Carpenter, McLuhan's collaborator, who in turn invited me to Fordham University in 1967 to meet McLuhan, Father John Culkin and other members of that charmed circle of communications theorists. The discussion centred on the idea that we had gone beyond Freud's invention of the unconscious and, for the first time, had rendered visible the conscious.

JN OK, so you're deeply immersed in the avant-garde scene and entranced by McLuhan. But how did you get from there to an involvement with science and technology?

JB It was McLuhan who turned me on to The Mathematical Theory of Communication, the book by Bell Labs scientists Claude Shannon and Warren Weaver that began: "The word 'communication' will be used here in a very broad sense to include all of the procedures by which one mind may affect another. This, of course, involves not only written and oral speech, but also music, the pictorial arts, the theatre, the ballet and in fact all human behaviour."

He also pointed me to Oxford zoologist JZ Young's 1950 BBC Reith lectures entitled "Doubt and Certainty in Science". And I recall his quoting one memorable line that has stuck with me and informed my thinking since that day: "We create tools and mould ourselves through our use of them."

John Cage had also picked up on all these ideas. He convened weekly dinners during which he tried them out, as well as his mushroom recipes, on a group of young artists, poets and writers. I was fortunate to have been included at these dinners where we talked about media, communications, art, music, philosophy, the ideas of McLuhan and Norbert Wiener. McLuhan had pointed out that by inventing electric technology, we had externalised our central nervous systems; that is, our minds.

Cage went further to say that we now had to presume that "there's only one mind, the one we all share". He pointed out that we had to go beyond private and personal mindsets and understand how radically things had changed. Mind had become socialised. "We can't change our minds without changing the world," he said. Mind as a manmade extension became our environment, which he characterised as "the collective consciousness", which we could tap into by creating "a global utilities network". In some ways in 1964 and 1965 he was envisioning what would become the internet, long before the tools became available for its implementation.

Inspired also by Buckminster Fuller and others, I began to read avidly in the field of information theory, cybernetics and systems theory. I also seized the opportunity to become the first "McLuhanesque" consultant and producer and soon had a thriving business working with clients that included General Electric, Metromedia, Columbia Pictures, Scott Paper and the White House.

I wrote a synthesis of these ideas in my first book, By the Late John Brockman'(1969), taking information theory – the mathematical theory of communications – as a model for regarding all human experience. A main theme has continued to inform my work over the years: new technologies = new perceptions.

An incident from those years stands out. During an evening at dinner, Cage reached across the table and handed me a copy of Cybernetics by Norbert Wiener. Fast forward two years. Around 1967, I spent two days with Stewart Brand while he was assembling the first edition of the Whole Earth Catalog and we sat and read the book together, underlining as we went along. Central to our interest was the notion of "feedback", the non-linear relationship of input to output. It was apparent that the ideas in cybernetic theory were far more important than the applications for which the mathematical descriptions were designed.

Stewart and I have been in touch regularly since then – a 45-year connection.

JN Was it difficult to come up with Edge's 2010 question, about the internet?

JB Every August, I begin a conversation with three of the original members of Edge – Stewart, Kevin Kelly and George Dyson. Eventually, I came up with the idea of asking how the internet is affecting the scientific work, lives, minds and reality of the contributors. A big consideration of this question is the difference between "we" and "you". When people respond to "we" questions, their words tend to resemble expert papers, public pronouncements or talks delivered from a stage. "You" leads us to share specifics of our lived experience. The challenge then is not to let responses slip into life's more banal details.

JN I was struck by something that one respondent, Evgeny Morozov, said about his fear of a chasm opening "between the disengaged masses and the overengaged elites". The elites, he goes on, "continue thriving in the new environment, exploiting superb online tools for scientific research and collaboration" etc. Actually, it's clear that many – most? – of your respondents are, par excellence, members of those elites. That's not a criticism, but it might mean that a casual reader could come away from the book thinking that public engagement with the internet and its significance is rather more elevated and intelligent than is actually the case.

JB The problem with a discussion that uses the word "elites" is that the word is automatically perceived as a pejorative. But that's not how I feel about it at all. Elites are a problem if they're closed and exclusive. Elites that are open, inclusive and based on merit can be nurturing. Also, members of elites give one another permission to be great. One example is the Beat poets. Another example is the mix of people who created Silicon Valley.

While Edge is a read-only site, the cast of characters contributing to the various projects is ever-changing and inclusion is by recommendation of members of the community. That said, Edge is not for everybody. It helps to know some stuff. But one thing you won't find in the responses is arrogance. The site stands or falls on the quality of the questions it asks.

In terms of this particular question – "Is the internet changing the way you think?" – there's the question of people having skin in the game. The contributors to Edge are what I call third-culture thinkers or intellectuals. Not only are they focused on science-minded pursuits based on evidence and empiricism, they are also public communicators, reaching out to the public by means of their books, lectures, etc. They live by their wits, and doing so in the changing times of the digital age is a challenge. Their concerns are very different than, say, the casual user, who has signed up for a social network and by default becomes the product whose private information is sold to advertisers.

JN In a way, the shadow of Marshall McLuhan looms over the conversation. Two of his aphorisms in particular – "The medium is the message" and "We shape our tools and later they shape us" – seem particularly apposite. The first captured the thought that what's important about a medium is not the content of the messages it carries but what the medium is doing to those who use it. That seemed to me to emerge from lots of the responses. And the meme about our tools shaping us surfaced again and again in the essays.

JB McLuhan is certainly central to this crowd and to the book. This is interesting because for more than a decade his name was barely mentioned. He certainly was an influence on me in terms of my intellectual development and career. In one typical conversation, he recounted his ideas on how psychoanalysis had gone the way of the gods and we were in a new realm where we were looking at the evolution of patterns and information. A lot has been written about the differences between atoms and bits, but the first time I heard it was from Marshall. For anyone who met him during the 60s, his manner and the way in which he presented himself were remarkable and never to be forgotten. Sitting down at lunch, you would be faced with machine gun-like expositions of facts and ideas ranging from medieval classical literature to arcane scientific matters concerning the aural space of the native North American Eskimos, the focus of the work of his collaborator Edmund Carpenter.

It was Carpenter who explained to me what he thought was the secret behind Marshall's brilliance. At the time, McLuhan was hospitalised after being operated on for the removal of a brain tumour. "And all those years we thought about the brilliance and we thought it was just Marshall," Ted said. "But it was the pills he was taking for symptoms of what turned out to be the tumour."

JN I noticed that Martin Rees and Richard Dawkins avoided talking about themselves and wondered if there might be something cultural – ie British – at work here? I'm an Irishman and so can say this!

JB Actually not. In this regard, the major challenge is to get 150 to 200 of the most brilliant people in the world to follow a simple set of guidelines. And one of the pronouncements this year is: "No anecdotes about spouses, significant others, kids, family pets."

The reason for this prohibition is that Edge is a conversation – it's not a magazine written for the public. The audience for the contributors to Edge is the other contributors. The readers have the opportunity to look over the shoulders of some extraordinarily gifted individuals as they go back and forth in the battle of ideas. And since the scientific method is central to our activities, I want to avoid the personal and focus on evidence.

JN I was pleased to see quite a lot about the "collective IQ" of the net – which is something that the mainstream media don't seem to understand at all. A passage in William Calvin's essay where he talks about the net enabling us to "stand on the shoulders of a lot more giants at the same time" reminded me of an older metaphor coined by, I think, Doug Engelbart, who invented the mouse, windowing interfaces and a lot of other seminal computing technology: "Power steering for the mind".

JB One of the concepts that people were talking about in the late 60s was "the collective conscious". McLuhan made specific reference to it on many occasions. Cage used to talk about "the mind we all share". The cultural anthropologist Edward T Hall, who was in that circle, and studied what he called the silent languages of time and space, once pointed out to me that our most significant, most critical inventions were not those ever considered to be inventions, but those that appeared to be innate and natural.

His candidate for the most important invention was not the capture of fire, not the printing press, not the discovery of electricity, not the discovery of the structure of DNA. Our most important invention was… talking. This was something considered innate and natural, or actually something that was probably never even considered, until the first human rendered it visible by saying: "We're talking" – probably an important moment in our evolutionary past.

The internet is such a new invention, a code for the collective conscious or "distributed networked intelligence". The internet is our collective externalised mind. I think of it in terms of the concept of feedback: the infinite oscillation of our collective conscious interacting with itself, adding a fuller, richer dimension to what it means to be human.

It's not about computers. It's not about what music your friends are listening to. It's about human communication. "We're talking."

How is the Internet Changing the Way You Think?, edited by John Brockman, is published by Atlantic Books. John Naughton's new book 'From Gutenberg to Zuckerberg: What You Really Need to Know About the Internet' is published by Quercus Books