Monthly Archives: April 2014

From Crackpots to Co-Authors

1 Comment

In my last post, I offered up a vision of an amateur science renaissance. It was my intention to follow that up directly with some suggestions on how to do that effectively. However, in the wake of my last post, two commenters (Chris Gray and David Zink) made similar excellent points which I felt I should address first. The core of their points as I saw them was this:

It’s all very well to propose a society of amateur scientists who contribute via the web. But how do you maintain any kind of quality in the discussion? 

The answer is, I don’t know. This is a really hard problem. Any kind of society of this sort is going to be plagued by groupthink, factionalism, shouting, self-delusion, and all of the other exciting behaviors we already see on the web.

Other communities have found partial solutions to this problem, though, and I propose that we borrow from them. Here are the partial solutions I think I see.

The Litropolis Solution

A community of amateur scientists will live or die based on the quality, breadth, and precision of the feedback that people receive. To my mind, some of the best critical feedback I’ve ever received on my ideas has come through the medium of writers groups.

Some basic things I learned are these:

  • Balance positive input with constructive input, regardless of how strong or weak the work is.
  • Give different input from the other people in your feedback group. This means that before you go into the feedback process, you should be thinking of as many different points as you can.
  • Always give examples and reasoning with your input, so that it can be understood and successfully applied.
  • Use a feature map of the set of attributes that each piece of work has, to help you assess all facets of the work. For fiction, this would include character, plot, pacing, point of view, etc.
  • Deliver feedback as you would like to receive it and leave your own emotions out.

 

I learned how to give good feedback at the Gotham Writers’ Workshop in New York. They did an awesome job. My skills were then honed when I had the opportunities to attend Clarion West and Milford. However, the terrific learning curve I went up came to a bumpy end when I found myself dealing directly with science fiction publishers and agents.

Despite the presence of structured writers workshops, classes, and online groups, taking the first steps into print can be a hideous process. This is in significant part because publishers are inundated with work from people who didn’t use these tools, and imagined that they could take a shortcut to literary genius. This process creates a ‘slush pile’ of unedited work. Good work usually gets drowned in it.

My friend Bob Kruger (who I met at Clarion) and I thought about this problem a lot since. We came to the conclusion that what was needed was a structured community where authors could assess each other and gain credibility mutually. This would be much more than an online writers’ group. It would be a reliable pathway to professional-level attention.

To his enormous credit, Bob has gone and built this thing, and is attempting to use his own skills and background as a publisher to lever it into existence. A society for amateur science could learn a lot from how it’s shaped. Credibility in Litropolis is quantized and cumulative, and accretes through respect that comes from more than one direction.

A society for amateur scientists that used something like this would find itself ahead of the state of play in professional science. You don’t have to move far in scientific circles before you encounter frustration at how professional journals are organized. Not only are they usually expensive to publish in, but the mechanisms for peer review and community assessment have barely been updated in decades. There are people at work solving this problem, but they’re doing so from the difficult position of an entrenched set of cultural norms. A society of amateurs could do better by starting from scratch.

The Improv Solution

Nowhere have I seen the principle of embracing failure more effectively manifested than in the improv theater community. Improv requires fearlessness and a separation between a person and their work, and this goal is routinely achieved. This is done by training people to experiment and experience small failures frequently, and to support each other while this is happening.

In so doing, the way that the brain interprets the sensation of failure is physically recoded. People literally become failure-proof. Stage fright falls away. People stop thinking of generated content as owned by an individual creator, and start thinking about it as the product of a group. People start being happier, more secure, funnier, wiser, more open-minded, and better at actually generating ideas. I kid ye not. They just get better. They even stand straighter.

At the core of improv training are some very simple principles:

  • Deliberately build on the ideas of others to create shared ownership.
  • Refuse nothing that is added to the work, even if you then build on that input selectively.
  • Take risks and deliberately extend yourself.
  • Trust your collaborators by default.
  • Look for ways to make your collaborator look good, rather than yourself.

The results of improv training range from the beneficial to the astonishing. As a result, the World Applied Improv Conference that happens about once a year is the point of closest approach to a cooperative utopia that I have ever seen.

The catch is that improvising is cognitively expensive and so your brain will try to fall out of the habit as soon as you let it. And once you stop doing it, all those habits of fear and entrenched reasoning start coming back. Furthermore, doing improv with the same people every time eventually has a similar effect at the group level. People go past the point where they’re learning from the surprising input they get from fellow players, and start becoming entrenched as a group, while kidding themselves that they’re still getting the full benefit of the process. This, to my mind, is why improvisers haven’t already taken over the world.

Also, improv has limits. It’s way more effective when it happens in person in a room full of people who’re laughing and looking at each other. This suggests to me that our ideal amateur science society would need local chapters and sustaining events to keep the culture functioning as well as online tools for intellectual play.

The network science solution

Some of the research I did over the last year was on collective decision making in people and animals, and how it’s affected by the shape of the social networks we inhabit. As a result of that research I now strongly believe that social media is still in the dark ages compared to what it could eventually be capable of. Because the forums we have now are polluted with self-delusion and infighting does not mean that they will always have to be.

Certainly we will not be able to rewrite the human tendency to jockey for social position or to prefer our own ideas to those of others, but by choosing how we enable people to come into contact with each other, those habits can be put to constructive use.

As starting suggestions for how this could be done, let us consider a hypothetical online science community tool that builds network carefully. Simple things we could try would include:

Capping the number of links or citations that an individual piece of work could receive. Part of what makes science broken is that the citation system has gone from being a roadmap of how to understand the field to a kind of payment system for scientific credibility. Numbers of citations received has become a measure of success. Consequently, everyone loses. Scientific fields become harder to navigate. Creating each bibliography becomes agony. And incentives appear for senior scientists to force more junior ones to quote their work.

Connecting scientists with critics from other parts of the social graph. Groupthink happens when small groups of people create echo-chambers where the can hear their own opinions reinforced. By building a non-local linking system into our service, we could dilute this effect.

Quantifying credibility, and then hiding the credibility scores of individuals who put work into the forum for criticism. One of the most depressing science stories I ever heard was about string theory. It was about a growing tendency for quantum gravity theorists, when trying to assess an idea, to ask ‘what does Ed Witten think?’ before giving an opinion. (Ed Witten is a string theorist so successful that he casts a shadow across the entire field.) Deferring to the opinions of those more prestigious than oneself is a habit that has no place in science. It is a holdover from religious thinking, and one to which people are lamentably prone. By partially anonymizing input, we would allow ideas and feedback to stand for themselves, further strengthening the split between the contributors and their work.

In the end, a sustainable amateur science community, to my mind, will need to involve elements of all of these sources, as well as making up some new tools of its own. So maybe the first job of this new society will be to work out a set of functional ground-rules for its own operation.

Along with that will need to be a shared understanding of what doing amateur science actually means and what it looks like. As I alluded to at the start of this post, I have some opinions on that. Next time, I’ll share them with you.

Amateur Scientists

2 Comments

Yesterday, I started a short series of posts about my conclusions from a year and a half of professional academic research. I outlined what I believe is a growing crisis in science. Today, want to talk about what we can do about it. My point today is this:

The world desperately needs amateur scientists.

Because of what’s happened to science in the mainstream, science that’s outside of it is more important than ever. While doing science professionally, I was astonished at how little time I spent pursuing new ideas compared to what I was used to as an amateur. This wasn’t down to any kind of error in direction or a problem with the mentorship I received. Quite the reverse. The lack of innovation was down to unavoidable elements of the culture of academia. The mentoring I received helped me navigate that culture. Without it, I would have been lost.

What I spent most of my time doing instead of innovating was checking my work to make sure that it was perfect, and that all possible bases were covered in order to make sure the work was of publishable quality. I spent months of my time trying to compensate for a huge number of hypothetical yes-buts that people might employ to try to invalidate my research. Scientists do this because in the world they inhabit, they need to. Anything less than this is a recipe for career-fail. Getting something wrong makes you look bad.

However, in a very important abstract sense, this is the wrong way to do science. That’s because no one can ever adequately check their own work. That’s what the community should be for. But a point has been reached in most areas of science where there are huge career incentives to make it look as if each piece of research that you deliver is correct, even though this goal is unattainable.

As Karl Popper pointed out years ago, science advances through refutation. You can never know what’s true, only what’s false. Thus, the most effective way for science to move forward is for people to make bold, falsifiable conjectures.

What this means in practical terms is that kudos in science should be attached to the act of checking someone else’s ideas, and to the creation of falsifiable conjectures, not their correctness. In short, science would be far better off to adopt a hard-earned lesson from the tech industry: embrace failure.  Wear each failure as a badge of pride. 

However, because the scientific culture we have puts a premium on ownership of an intellectual landscape, and on apparently uncontestable expertise, we deliver science in units of apparent advancement instead.

This is slowly creating a nightmare of confirmation bias in today’s scientific community. With the addition of computer simulations and powerful statistical tools, the point of finding yourself looking at what you want to see has become extremely easy to attain. All the repetitive checking and testing does nothing whatsoever to alleviate this, because the tests that people do are, by and large, those that occur to them after the process of bias has already occurred.

This is not to say that all science done today is wrong. Far from it. Plenty of good work gets done. But that is despite of the system that we have set up, rather than because of it. Science is so hard to do as a career precisely because we’ve gone and made it that way.

There are those who will tell you that science has to look the way that it currently does. They will tell you that, because there is so much research being done, it is the responsibility of scientists to check their own work as thoroughly as possible before publishing it. This stance however, misses the point.

Certainly, keeping up to date on publications (such as Arxiv preprints) in many scientific fields these days can be exhausting. But this is because A: it creates an inferiority complex in young scientists struggling to compete, and B: because the work that’s published is usually so cautious and glutinous in its presentation that reading even one paper can take hours, let alone the dozen that might be posted on a given day. I have yet to meet a scientist of my generation who reads professional science papers for fun.

In a better world, the papers would be written like tutorials, and students would scour Arxiv every day as a goldmine of refutation opportunities. However,  even with all the best will in the world, it will take years, or some exceedingly painful crisis, or both, for the world of professional research to change. To my mind, then, the role of the amateur scientist in today’s world is to do what professionals can seldom afford to do any more: play.

If human knowledge is going to advance fast enough for us to solve the problems we create for ourselves, there need to be people who can try on offbeat ideas without fear. Amateur scientists need to be deliberately operating outside of the mainstream, and ignoring prestige.

This is a key point. Via my work on digital physics, I’ve spoken to a lot of smart, well-intentioned people with interesting ideas who are what people in the physics community often refer to as ‘crackpots’. What this really means is that they’re untrained people who’re giving an idea they care about their best shot. The problem that most of them share is that they’re trying urgently to communicate their reasoning to professionals.

Usually, the crackpot/amateur-scientist narrative goes like this:

I think I’ve found something really important. It looks to me like this could be the basis for a Theory of Everything. Now I just need to talk to a physicist to work out the math and communicate it to the world.

Physicists usually hate this because:

1: They have spent the last n years of their life trying to absorb a highly complex and nuanced way of looking at the world. When someone comes along talking a completely different language, it’s somewhere between annoying and outright painful to then adapt to comprehend someone’s new esoteric frame of reference, particularly when they start off with no proof that there is something concrete behind it.

2: They have so little time to push forward on their own work that they’re desperately keen to pursue their own ideas, not someone else’s. This is because the culture they inhabit tells them that this is what counts, and will either make or break their already fragile career.

3: They’re fighting for every inch of their credibility. The last thing they need is to try attaching themselves to someone with no credibility, and without any formal training in their field. For the professional scientist, it almost never looks like there’s anything to gain.

What this means is that lucky crackpot/amateurs in digital physics receive disdain when they reach out to physicists. The unlucky ones receive a sequence of uncomfortable pseudo-supportive brush-offs that send mixed signals and can last for years, crushing hope and seeding bewilderment.

But look at what causes this problem. These amateurs are chasing what they think science is about: sudden flashes of insight that transform how we think about the world and which reveal to the world the genius of their creator. Amateurs are chasing this dream because they see the professionals do it. The irony is that the professionals should know better.

The answer to this social malaise is simple. Amateur scientists should seek out each other instead and engage in peer review, rather than expecting professionals to help. This is better for everyone. And it makes sense, because, outside of critical input that could equally well be delivered by other amateurs, the best that a professional can offer in support is their credibility and prestige. And frankly, outside of the academic job market, these commodities aren’t that useful. An amateur’s ideas are surely at their strongest when there are enough people already excited about them that professional attention becomes inevitable.

What I’d like to see is a body of amateur scientists helping each other, refuting each other’s work, where possible, and using the internet as their greatest tool and forum. We are in a better position to do this now than at any point in human history. With services like Coursera providing a free scientific and technical education, and the web providing countless platforms to be heard, we are ready for a new amateur science renaissance.

But realistically how, you may ask? Surely scientific literacy takes years. Isn’t the frontier of human knowledge a huge distance away from most people’s understanding of how the world works? Hasn’t all the easy stuff already been exhaustively explored? Shouldn’t we just leave it up to the professionals?

Hell no. The edge of human knowledge is next door and it’s being very inadequately searched. And everyone who can cut code, think critically, and who cares, should be helping to fix that. In the next post, I’ll say more about how.

About Research

1 Comment

It’s been a weird year. I’ve been closer to professional academic research than I have in ages, doing complex systems work at Princeton University on everything from social inequality to abiogenesis. For the most part, I had a great time. However, now that this experience is winding down, there are some important things I feel I should say:

1:Professional science is broken.

2: The world desperately needs amateur scientists, now more than ever.

3: The frontier of human knowledge is an incredibly short walk away, and anyone who cares should be visiting it.  

On the face of it, these remarks might seem somewhat negative and/or counterintuitive, so let me explain, one point at a time.

First: professional science is broken.

While doing work at Princeton I was supported by the most amazing people. All of them were smart. All of them were kind. They were incredibly welcoming to someone who’s been doing science at random in a non-academic context for the last twenty years. I have phenomenal respect for all of them. I cannot say enough positive things about the support I received. Princeton, too, was an surprisingly functional, rational, and inclusive institution. Without exception, every department I encountered was wisely and compassionately run, and managed at a superb level of professionalism. In short, they were awesome. So what’s broken, you may ask?

The system that academics inhabit is broken. Academic career tracks are broken. The funding situation is broken. The compromises that people have to make to stay in research are broken. The way that people have to publish research in order to be considered professional is broken. The workload for junior faculty is broken. The administrative load that senior faculty have to manage is broken. And it is for all these reasons and more that my wife and I are leaving.

I don’t have room here to go into all the subtle details of why I believe there’s a profound crisis taking place in science in the western world. There is plenty on the web that captures parts of this story, for instance, here, here and here. In subsequent posts I’ll try to drill down and explain what I learned. But the main takeaway for this post is as follows:

Much of professional science has reached a point where engaging in the process of risky intellectual exploration is untenable for people who want to retain their job and their credibility.

This is because this career track is now so tenuous, and so over-competitive, that it forces a strategy of extreme caution onto the very people who should be taking the biggest intellectual risks.

Being a scientist is a bit like being in the military. You join up and undergo a rigorous, often demoralizing training process. (Unlike in the military, this process can last anywhere from three to eight years.) At the end of that, you’re effectively posted somewhere. You up and leave your home, and start in your new position, which usually lasts three years (five if you’re incredibly lucky). Then, after that, you’re usually posted somewhere else, also for three years. Then, after that, you’re posted somewhere else again. This time, you get to stay for five years, but you often have to work 60 to 80 hour weeks busting a gut to establish yourself. Then, in many cases, you move again. By now, most people are about 16 years deep into this career track and counting.

God forbid that at any point during this process you become attached to a place, or furniture, or a hobby, or a family member, or a spouse. With each wrenching shift, these people/things either come with you, or you lose them for good. And along the way, if they come, they get a little bit smashed up. That’s because when you land in your new post, there’s room for you in the department, but usually no room for whoever or whatever came with you.

What really differentiates science from the army, though, is that in science, you’re supposed to own this process of transition. What I mean by this is that it’s supposed to be your idea. You’re supposed to feel lucky that you get such an awesome job. You’re supposed to be delighted at your good fortune. The military serviceperson gets to treat their constant relocation as a necessary evil that comes upon them through their duty. The scientist is just supposed to like it. Concerned about the stability of your income stream? You’re not supposed to care. It’s supposed to be beneath you. Want a decent daycare situation for your kids? That’s supposed to be a lower priority.

The number of people for whom this is actually true is vanishingly small. However, you wouldn’t guess it, because most of the people in this line of work have had to convince themselves that they’re one of that tiny minority. Often, they stick with this belief right up until they get sick, or unlucky, or depressed, or just simply forced out in the middle of their lives, and expected to abandon their dreams and retrain.

My wife and I were lucky. We left from a position of strength, with the door wide open for us to do more. But not everywhere in scienceland is as good as Princeton.

My point is that many many scientists are clinging desperately to the mast of their personal ship tossed in a battering career storm that they do not deserve. And these are the people we generally expect to take intellectual risks, to serve as guardians of truth, and to push forward the boundaries of human knowledge. And that’s bullshit.

Next, I’ll cover point two.