The Personal Paradigm and Peer Review

RenalWeek 2009 in San Diego is underway. I just enjoyed a forum on conflict of interest in medicine. One talk, by Rober M. Califf, MD, of Duke University, discussed bias in clinical trial reporting. Most of the speakers focused on financial conflicts of interest. Dr. Califf described a number of nonmonetary conflicts of interest as well.

One of these nonfinancial conflicts involved scientific hubris. For example, you review a paper or a grant and its (potential) results shoot down your own research and ideas. Do you accept the possibility that you are wrong? This work has the potential to make your prior work irrelevant, as well as decreasing your ability to get funding and promotion. The document is well-written and convincing, at least to the other reviewers. Do you allow it to proceed, or do you demand further work to reconcile it with your current hypothesis? Data that may not be feasible, if your personal paradigm is wrong?

We scientific types supposedly worship at the alter of truth and knowledge. We are human, though, and humans like to have explanations for everything. As nature abhors a vacuum, our brains hate missing information or pieces. In other words, we are biologically predisposed to develop hypotheses and patterns for everything around us. We also seem predisposed to defending views in which we are invested.

This source of conflict is rampant in peer review. Qualified peers must have knowledge of the subject area in question, and they will have pre-formed hypotheses about the material. Most reviewers can accept new experiments that advance their ideas, even if the world moves in a new direction. When those pre-formed ideas quash legitimate questions, then they become a source of conflict of interest.

What can one do? One can exclude reviewers based on probable bias. This practice is commonplace for manuscript submission. If a member of your study section could be trouble, you can request that they not review your grant. You should send your stuff to lots of different funding agencies as well; the same proposal may be funded by one agency and triaged by another, just because peer review is subjective and biased in this way.

Human nature simply will not allow all sources of bias to be removed from the world. I hope we in science can consider our own intellectual prejudices and overcome this conflict of interest.

Photos courtesy of PhotoXpress.

Greetings From San Diego

Most of my administrative tasks have now been completed. I am sitting in a session on conflict of interest, especially the infiltration of pharma into professional societies.
More on this later (when I have my netbook rather than my iPod). Just wanted to let everyone know I was alive!

On The Road Again

Tomorrow I hit the airport en route to Renal Week, the Annual Meeting of the American Society of Nephrology. After some administrative work, my week will be devoted to all things kidney. From the most basic science to the most novel clinical trial, 14,000 devotees of urine (or lack thereof) will be gathered to celebrate their favorite organ.

Attendees and other kidney fans are invited to Tweet the Week. We will be twittering about the meeting and other events surrounding it. I am sure Renal Week will provide fodder for this august venue as well.

I will also try to update everyone on the weather. After all, I will be in San Diego. I get to gloat at those I leave behind in Nebraksa.

A new podcast should also be available soon. I chat with current president Tom Coffman and president-elect Sharon Anderson about the ASN. Look for it and other meeting stuff on the site or in our iTunes area.

For the rest of the day, well, my brain has already flown west…

Correcting My Mistake

Back in August I bought a pair of black shoes. The heels were just under 2 inches on a study wedge. Embossed leather resembled croc. Great meeting and work shoes; dressy, but grounded enough to walk in all day long.

Unfortunately, they eat my feet. Blisters and cuts and pain, Oh My!

Yesterday I acquired the Steve Maddens shown in the photo. They are far more stylish with the patent ruffle. Most importantly, they do not damage my feet.

And I only paid $35.99 at DSW.

Of course, I would wear something totally hot if I were presenting a plenary. I saw these Steve Madden’s in a department store yesterday. I could not figure out a reason to buy them. At least not until they go on sale. You have to admit, they make a statement.

For more shoe wars, point your browser on over to Isis’ place. Hot (and not-so-hot) shoes are driving debate between the bloggers. Don’t miss out!

True Confessions

Over on Isis the Scientist, a perfect storm developed about one of Intel’s recent ads. The ad, depending on one’s viewpoint, might suggest that physical exercise and mental exercise are mutually exclusive and/or that women can’t play chess, the mental work-out.

I am a woman with a high aptitude in math and science. I test at the top of the scale on pattern recognition and manipulation, especially in three dimensions. That test where they show you the paper pattern and you have to figure out what object they would become when folded? Piece of cake. When I did Now, Discover Your Strengths, “Strategic” is number 1.

So why can’t I play chess?

I came of age when Bobby Fischer was kicking Russian ass. Kids my age were chess geeks, and it was not a bad thing at that time. I learned how to play, but I never got good.

Truth? I found chess boring. Incredibly, colossally boring.

Today, I feel the same way about sudoku. I love crossword puzzles and word searches. My new BlackBerry Tour came with Word Mole; it is a blast! But those number patterns? Who the hell cares?

Of course, while we’re at it, I guess I should confess that I lied at my medical school interviews. I told everyone I was interested in a primary care career in a rural area. It was the reason my school existed, and I knew it would be a + in a column for my application. Of course, the one thing I was not considering was a career in pediatrics. Yet, here I am today, a Pediatric Nephrologist.

I feel better now. I still can’t beat you at chess. But then, who the hell cares?

Photo courtesy of PhotoXpress.

Moving On & Moving In

A couple of months ago our cat of 18 years, Denver, died of chronic kidney disease (yes, I know, it is ironic since I am a nephrologist).

My husband never wanted a cat. He tried to convince our almost 4 year old daughter that she wanted a puppy. Her highness was not amused, and we adopted The D from the local animal shelter on her birthday.

My husband, the dog guy, decided last week that we had been feline-free long enough. He missed a buddy with whom he could play, watch TV, curl up with a good book, or nap. So last weekend we found ourselves at a shelter outpost.

And now I am introducing Ozzie:

This 2 month old guy (who would fit in one hand) was found at an intersection in town. He had a cold when we got him, and he didn’t eat last Sunday. After a trip to the vet for fluids, and some higher calorie prescription food, he is the joy and the terror of our lives. He is inhaling canned and dry food. He likes to be chased. He would love to someday catch the laser pointer dot.

He thinks my laptop is a really big cat toy. Writing this post challenged me, both to get it done and to keep him from “writing” his own contribution.

It took several days to name the little beast. Ozzie was selected because it is the name of a Hall of Fame Cardinal shortstop, and short for Osiris of Eyptian lore (hey, they worshipped cats).

He has no pedigree or papers. His origination is “46th and Curtis.” Yet we know he is the greatest cat in the world. Next to Denver.

The Problem With Peer Review

"If this is the best of all possible worlds, what are the others?"

- Voltaire, Candide, Chapter 6

Peer review is a lousy way to score research proposals; it just seems to be the best one we have found to date. NIH is altering its application and review processes, attempting to improve things. Unfortunately, the major problem cannot be fixed.

What major problem? The peer reviewers.

For example (derived from my own experience and that of my colleagues):

A proposal is submitted to study biological process A in disease X. A model of disease X is proposed, and standard techniques will be used to examine the role of process A. On initial submission, the reviewers say that process A appears to be important in disease X, but “enthusiasm would be stronger” if further preliminary data were presented. Furthermore, pathway Y contributes to process A, so perhaps it should be included in the study.

As PI, you consider these suggestions and do further experiments to augment your preliminary data. You review studies of pathway Y and conclude that it should be included in your proposal. You rewrite you grant, and send it back to Bethesda.

This time, you have a novel reviewer. Those who saw the first version are satisfied, but the latest one feels that biological process B is of far more importance than process A, even though you have data and published literature from other labs supporting process A. Perhaps the latest reviewer believes you should study disease Z rather than disease X. Or maybe a different model should be used, even though you have been using this model for all other work in your lab. Or, perhaps, s/he wants all of the above changed.

By the way, the reviewers also criticize your productivity because, while generating the preliminary data requested, you have not continued to publish as many peer-reviewed journal articles.

This wackalunacy must stop.

It is one thing to suggest additional experiments or a better technique. These suggestions may strengthen the work.

Telling the PI to completely change the nature of the study is inappropriate:

• Look at process B even though your preliminary data and published studies support process A in your hypothesis
• Disease Z is more important than disease X, even though you have never studied disease Z

Proposals get toasted with this sort of criticism in each round of reviews. If we could fund the top 25-30% of proposals(which are getting scored in the very good-excellent range), some of these applications would still fly; in the current funding drought, any little glitch (no matter how trivial) can drop the score below the pay line.

I do not have the answer. I beg any reviewers reading this to play fair. Judge what is proposed; no study can examine every biological process! If your favorite pathway is omitted, fear not! Someone else may be asking that question; let this PI examine his/her idea! Occasionally I have seen someone who proposes to examine the role of process A in disease X, but is using a model of disease Z. This is a major flaw, and a change of model or disease is in order. It is NOT in order if you, the reviewer, find disease Z more interesting than disease X. Would you want someone to tell you to study disease X? I thought not…

NIH has shortened its application process and changed the scoring system. Somehow I don’t believe this will change reviewer behavior, but I will find out early in 2010.

In the meantime, I will just keep imagining worse worlds of review. I finally appreciate Candide.

Art courtesy of PhotoXpress.