In my last post, I offered up a vision of an amateur science renaissance. It was my intention to follow that up directly with some suggestions on how to do that effectively. However, in the wake of my last post, two commenters (Chris Gray and David Zink) made similar excellent points which I felt I should address first. The core of their points as I saw them was this:

It’s all very well to propose a society of amateur scientists who contribute via the web. But how do you maintain any kind of quality in the discussion? 

The answer is, I don’t know. This is a really hard problem. Any kind of society of this sort is going to be plagued by groupthink, factionalism, shouting, self-delusion, and all of the other exciting behaviors we already see on the web.

Other communities have found partial solutions to this problem, though, and I propose that we borrow from them. Here are the partial solutions I think I see.

The Litropolis Solution

A community of amateur scientists will live or die based on the quality, breadth, and precision of the feedback that people receive. To my mind, some of the best critical feedback I’ve ever received on my ideas has come through the medium of writers groups.

Some basic things I learned are these:

• Balance positive input with constructive input, regardless of how strong or weak the work is.
• Give different input from the other people in your feedback group. This means that before you go into the feedback process, you should be thinking of as many different points as you can.
• Always give examples and reasoning with your input, so that it can be understood and successfully applied.
• Use a feature map of the set of attributes that each piece of work has, to help you assess all facets of the work. For fiction, this would include character, plot, pacing, point of view, etc.
• Deliver feedback as you would like to receive it and leave your own emotions out.

 

I learned how to give good feedback at the Gotham Writers’ Workshop in New York. They did an awesome job. My skills were then honed when I had the opportunities to attend Clarion West and Milford. However, the terrific learning curve I went up came to a bumpy end when I found myself dealing directly with science fiction publishers and agents.

Despite the presence of structured writers workshops, classes, and online groups, taking the first steps into print can be a hideous process. This is in significant part because publishers are inundated with work from people who didn’t use these tools, and imagined that they could take a shortcut to literary genius. This process creates a ‘slush pile’ of unedited work. Good work usually gets drowned in it.

My friend Bob Kruger (who I met at Clarion) and I thought about this problem a lot since. We came to the conclusion that what was needed was a structured community where authors could assess each other and gain credibility mutually. This would be much more than an online writers’ group. It would be a reliable pathway to professional-level attention.

To his enormous credit, Bob has gone and built this thing, and is attempting to use his own skills and background as a publisher to lever it into existence. A society for amateur science could learn a lot from how it’s shaped. Credibility in Litropolis is quantized and cumulative, and accretes through respect that comes from more than one direction.

A society for amateur scientists that used something like this would find itself ahead of the state of play in professional science. You don’t have to move far in scientific circles before you encounter frustration at how professional journals are organized. Not only are they usually expensive to publish in, but the mechanisms for peer review and community assessment have barely been updated in decades. There are people at work solving this problem, but they’re doing so from the difficult position of an entrenched set of cultural norms. A society of amateurs could do better by starting from scratch.

The Improv Solution

Nowhere have I seen the principle of embracing failure more effectively manifested than in the improv theater community. Improv requires fearlessness and a separation between a person and their work, and this goal is routinely achieved. This is done by training people to experiment and experience small failures frequently, and to support each other while this is happening.

In so doing, the way that the brain interprets the sensation of failure is physically recoded. People literally become failure-proof. Stage fright falls away. People stop thinking of generated content as owned by an individual creator, and start thinking about it as the product of a group. People start being happier, more secure, funnier, wiser, more open-minded, and better at actually generating ideas. I kid ye not. They just get better. They even stand straighter.

At the core of improv training are some very simple principles:

• Deliberately build on the ideas of others to create shared ownership.
• Refuse nothing that is added to the work, even if you then build on that input selectively.
• Take risks and deliberately extend yourself.
• Trust your collaborators by default.
• Look for ways to make your collaborator look good, rather than yourself.

The results of improv training range from the beneficial to the astonishing. As a result, the World Applied Improv Conference that happens about once a year is the point of closest approach to a cooperative utopia that I have ever seen.

The catch is that improvising is cognitively expensive and so your brain will try to fall out of the habit as soon as you let it. And once you stop doing it, all those habits of fear and entrenched reasoning start coming back. Furthermore, doing improv with the same people every time eventually has a similar effect at the group level. People go past the point where they’re learning from the surprising input they get from fellow players, and start becoming entrenched as a group, while kidding themselves that they’re still getting the full benefit of the process. This, to my mind, is why improvisers haven’t already taken over the world.

Also, improv has limits. It’s way more effective when it happens in person in a room full of people who’re laughing and looking at each other. This suggests to me that our ideal amateur science society would need local chapters and sustaining events to keep the culture functioning as well as online tools for intellectual play.

The network science solution

Some of the research I did over the last year was on collective decision making in people and animals, and how it’s affected by the shape of the social networks we inhabit. As a result of that research I now strongly believe that social media is still in the dark ages compared to what it could eventually be capable of. Because the forums we have now are polluted with self-delusion and infighting does not mean that they will always have to be.

Certainly we will not be able to rewrite the human tendency to jockey for social position or to prefer our own ideas to those of others, but by choosing how we enable people to come into contact with each other, those habits can be put to constructive use.

As starting suggestions for how this could be done, let us consider a hypothetical online science community tool that builds network carefully. Simple things we could try would include:

Capping the number of links or citations that an individual piece of work could receive. Part of what makes science broken is that the citation system has gone from being a roadmap of how to understand the field to a kind of payment system for scientific credibility. Numbers of citations received has become a measure of success. Consequently, everyone loses. Scientific fields become harder to navigate. Creating each bibliography becomes agony. And incentives appear for senior scientists to force more junior ones to quote their work.

Connecting scientists with critics from other parts of the social graph. Groupthink happens when small groups of people create echo-chambers where the can hear their own opinions reinforced. By building a non-local linking system into our service, we could dilute this effect.

Quantifying credibility, and then hiding the credibility scores of individuals who put work into the forum for criticism. One of the most depressing science stories I ever heard was about string theory. It was about a growing tendency for quantum gravity theorists, when trying to assess an idea, to ask ‘what does Ed Witten think?’ before giving an opinion. (Ed Witten is a string theorist so successful that he casts a shadow across the entire field.) Deferring to the opinions of those more prestigious than oneself is a habit that has no place in science. It is a holdover from religious thinking, and one to which people are lamentably prone. By partially anonymizing input, we would allow ideas and feedback to stand for themselves, further strengthening the split between the contributors and their work.

In the end, a sustainable amateur science community, to my mind, will need to involve elements of all of these sources, as well as making up some new tools of its own. So maybe the first job of this new society will be to work out a set of functional ground-rules for its own operation.

Along with that will need to be a shared understanding of what doing amateur science actually means and what it looks like. As I alluded to at the start of this post, I have some opinions on that. Next time, I’ll share them with you.