NSF “clarifies” Broader Impacts « Gas station without pumps

After reading the report (the body, not the hundreds of pages of appendices), I’m at least as confused as I was before about what the h*** NSF expects for Broader Impacts. (Gas station without pumps, hereafter GSWOP)

What is revealed by the above quote is not simply laziness of mind, but also the greatest potential danger to scientific autonomy in the near future. It is a kind of entitlement attitude that serves to justify an unwillingness to think reflectively about science.

– But I want Broader Impacts to be easy! Give me a list!

If there were a list of “desired societal outcomes” then it might be possible to compare proposals, but without such a list, politics, religion, and random taste prevail. (GSWOP)

Well, if scientists would take the time to read through the rest of the report …

– No way!! Are you crazy?!?

… As I was saying, if scientists would take the time to read through the rest of the report …

– It’s 300 pages!!

… Right. Anyway, those who actually read through the report will see that NSB offered a set of proposed new criteria that included a list, and they got lots of feedback. They actually did an excellent job of incorporating the feedback they got into the ‘new’ criteria, and that was no easy task. Most folks who took the time to write NSB about the proposed new criteria (with the list of “national goals”) were not in favor of the list.

– But how are we supposed to know what to do without a list?!

Hold on a second. Take a look at page 274 of the report, and you’ll find that responses to NSB regarding the ‘criteria with list’ were mixed at best, tending toward negative: 86 negative responses, 71 mixed, 64 positive, and 48 neutral.

– But how could anyone object to a list?! A list would make Broader Impacts so easy!

I’m glad you asked. One problem that many respondents brought up is that some important national goals were left off the list (social welfare, human health, environment, and energy topped the list of ommissions: p.277).

– Well, OK, I see that, I guess. But we could just expand the list, right?

Perhaps, but that wouldn’t really make Broader Impacts any easier to judge.

– Of course it would!

If there were a list of “desired societal outcomes” then it might be possible to compare proposals, but without such a list, politics, religion, and random taste prevail. (GSWOP)

– Right. I could just look at the list and check off whether the proposal meets one or more of the national needs on the list!

One could certainly do that, but how does one judge whether a proposal that meets a national security goal is better than a proposal that meets an economic competitiveness goal? Does a list really alleviate GSWOP’s worry about politics, religion, and random taste?

– One that does both is better than one that meets either by itself. So, the more national goals a proposal meets, the better the proposal in terms of broader impacts.

Perhaps. But wouldn’t trying to meet all the national goals render the proposed activity ineffectual?

– I don’t understand. More is better.

Well, do you think, in general, that it is more effective to pursue as many goals as possible at one time, or to pursue one goal with full attention to attaining it?

– Well, the latter, of course: pursuing one goal rather than many is more likely in general to lead to the attainment of the goal.

So, now you are beginning to see that lists, while they might seem to make things easier, actually only disguise the fact that judgment still must enter into the equation.

– I suppose. But how can we judge Broader Impacts if we don’t know what NSF is looking for?

Yes, GSWOP raises this issue in a slightly more paranoid way:

More likely, there is a secret list of what NSF really wants for broader impacts, which is supposed to be “understood” without ever being stated.  So far as I can tell, NSF has just said “read our minds, we’re going to judge whether you address our goals without ever telling you what those goals are”.

– Well, that’s right, isn’t it? I mean, if NSF doesn’t tell us what they want in terms of Broader Impacts, how are we supposed to know?

Again, I’m really glad you asked that question. Let me respond by asking you a question. Does NSF tell you what they want in terms of Intellectual Merit?

– What? Well, no, not really. But what’s your point?

Well, if NSF doesn’t give you a list of activities that count for Intellectual Merit, how are you able to judge whether one proposal has greater Intellectual Merit than another?

– I don’t understand. That’s easy. You just read the proposals and you know which ones are talking about doing something really good, something really important for the field. I mean, if it’s good science, then it’s obviously good science.

But what makes one proposal for a scientific activity better than another?

– Well, you know, things like what they propose to do, why they want to do that, whether they have a good plan to do what they propose, how they’ll know whether they succeeded or not, and what difference it will make if they do succeed.

But none of that is a list of specific activities — it’s a generic description of what any research proposal (regardless of the specific activity proposed) ought to include.

– Sure, but that’s no problem.

So, it is possible to judge Intellectual Merit without a list of activities that would count for Intellectual Merit?

– Of course. Moreover, it’s better not to have a list!

Really? Why?

– A list would restrict us to doing certain kinds of research. But really, research questions should come from the researchers, not from the government or a government agency.

Really? Why?

– Haven’t you heard of the Haldane Principle?! Well, it doesn’t matter. Researchers should determine what research is important, and a list imposed from the top would just interfere with the progress of science.

I see. But should it not also be possible to judge Broader Impacts without a list of activities that would count for Broader Impact?

– What?! Don’t be absurd! I already told you that it’s not possible to figure out what NSF wants in terms of Broader Impacts! GSWOP is correct, too — I bet NSF has a secret list! They’re just trying to make things difficult for us scientists!

I’m not sure why you’re so upset,  but …

– I’m upset because NSF won’t clarify the Broader Impacts Criterion! What have we been talking about this entire time?!

But if it is possible to judge Intellectual Merit without a list, shouldn’t it also be possible to judge Broader Impacts without a list?

– You are really starting to try my patience!

Indeed. But isn’t it also the case that it should not only be possible to judge Broader Impacts without a list, but also that allowing scientists to propose Broader Impacts activities preserves their scientific autonomy?

– You obviously know nothing about science or scientific autonomy! It’s about being able to do the science for the sake of the science — not about whatever Broader Impacts it might have!

Really? Scientific autonomy is about the intrinsic value of science? I would have thought that scientific autonomy had more to do with science being able to decide on its direction — that is, scientific autonomy ought to be thought of in terms of scientists being empowered to make decisions for science, rather than having those outside science impose their decisions on scientists.

– You just don’t get it! Talking with you philosophers is a total waste of time! You philosophers know as little about science as NSF! I’m done with you!

Well, that’s too bad. Good luck thinking through the Broader Impacts of your proposed research! I’ll be here if you want to continue the discussion.

 

NSF “clarifies” Broader Impacts « Gas station without pumps.