Journal Club: Strong Inference- Platt

[S.A.M.]

The Platt paper as a starting point for the Journal Club

A very brief summary of the Platt paper:

In the paper, “Strong Inference”, John Platt examines inductive reasoning as an essential tool in the practice of science, the systematic application of which is necessary for an effective scientific approach. This method emphasizes the importance of being “problem-oriented” rather than “method-oriented” and involves exploring all possible alternatives to your hypotheses, testing them and then re-examining the outcomes of these tests using further sub-hypotheses or sequential hypotheses.

This opens up several aspects of the problem for simultaneous examination while refining your original hypotheses and proving (or disproving) its premise. Although inductive reasoning has been around for a long time, it has been constrained by its inherent difficulties. It assumes the ability to discover all possible exclusions in a particular problem and is also obstructed by an inherent tendency in scientists to become attached to a single hypothesis and resisting any criticism of it.

In addition, several research scientists are more concerned with the development and accuracy of the tools used for collecting and analyzing the data rather than the accuracy of the hypothesis to be tested. In conclusion, the paper underlines the importance of scientific integrity and how it is best served by a systematic and intelligent approach to the scientific method.

 

[Exhumed]

Spuriousmonkey should be recruited Anyway...

It was a very compelling article. A few things I'm not sure that I agree on:

The author was advocating the exclusion/strong inference method in general terms, seemingly for all situations. I have trouble imagining this in situations except when trying to figure out fundamental rules of a particular area, which seemed to be what most of his historical examples were.

I have some doubts as to whether I can apply this usefully. Looking at using it in a somewhat specific, random case: the study of blocking Epidermal Growth Factor receptors when studying cancer metastasis, the use of strong inference is very obvious that I wonder if it even counts. Blocking the receptors inhibits cancer movement or has no effect. One of these results falsifies the other. Is there a better way to apply the strong inference method? I have a feeling this is a bad example, not complex enough for a good application ~.~

One other thing, I'm not sure how correct he is in attributing this method to the success in microbiology, but I defer to others as I am not someone to make a judgement on that.

I'd like to hear from the professional biologists here what they think of this question the paper mentioned applied to their work

"But sir, what experiment could disprove your hypothesis?" or on hearing
a scientific experiment described, "But sir, what hypothesis does your experiment disprove?"

I really liked when he said:

We speak piously of taking measurements and making small studies that will "add another brick to the temple of science." Most such bricks just lie around the brickyard

 

[S.A.M.]

In terms of the receptor activity; its not as simple as that is it?

e.g. You could block the receptor and still see activity

Does this mean:

1. the receptor is not important?
2. there is a parallel pathway?
3. the inhibitor does not work


In each case you need to figure out how you will go ahead.

 

[Exhumed]

Yeah, it's not that simple, I was avoiding details : o

In areas of practical research like that one do you think that thinking in terms of the strong inference method helps?

 

[S.A.M.]

In my work, nutrition it helps, because there are no simple answers.

I work in obesity and try to identify how dietary components contribute to or control the different aspects of energy balance (since in very simple terms, obesity is positive energy balance).

Now there are 3 points at which there can be modulation

1. Intake
2. Metabolism
3. Output

The problem is that at every single point along the road, there are many signals. The system is controlled not only for the short term, but also the long term. What is the key? How does it happen? We don't know.


With epidemiological studies, we obtain correlations, but they do not reproduce in the lab; even if they work on animal models, they appear to fail in human studies.


This means that before you design a study, you need to think forward and anticipate the what ifs, so that you don't get stumped at point A.

The idea becomes to cover as many bases as possible, to ensure a more holistic view of what is going on with the manipulations we produce.

Take vitamin D for instance
The same nutrient e.g. vitamin D, can have opposite effects in normal and cancer cells.

A ras oncogene can make the cells resistant to the effects of vitamin D.

We don't even know if the vitamin D made in the skin by UVB radiation is more or less available than the one we are consuming in our diet.

Whats the normal level in the blood? Are the actions different at different levels?

Are we taking too little or too much? Is there a biphasic reaction? a U shaped one?

How do you measure endpoints? What are the endpoints? Are they reflecting the changes we induced?

There are far too many possible variables to cover. Strong inference becomes a must.

 

[Klippymitch]

Good paper and I cant believe I read all of it. I was getting really bored near the end.


If you were a mechanic would you listen to a customer and just fix the part the customer think is wrong. Or do you first make sure the customer's theory was correct by following a method of reasoning?

 

[S.A.M.]

Sorry about the boring paper!

I was trying to start with an easy one, as a trial, I'll pull up more molecular stuff if there is interest.

If I were a mechanic, I would ask the customer what problems he was having. Obviously, as the owner and regular driver of the vehicle, the customer would be in the best position to describe the changes or problems he has experienced. Then I would use my own knowledge of auto mechanics to correlate the symptoms to all the possible problems and by a process of elimination go through them to see if any of my possibilities covered the problem area. If yes, well and good, if not, I would just have to do a regular maintenance check before taking stuff really apart.

But thats okay since I would see $$$ in return

 

[Klippymitch]

Sorry about the boring paper!

I was trying to start with an easy one, as a trial, I'll pull up more molecular stuff if there is interest.

If I were a mechanic, I would ask the customer what problems he was having. Obviously, as the owner and regular driver of the vehicle, the customer would be in the best position to describe the changes or problems he has experienced. Then I would use my own knowledge of auto mechanics to correlate the symptoms to all the possible problems and by a process of elimination go through them to see if any of my possibilities covered the problem area. If yes, well and good, if not, I would just have to do a regular maintenance check before taking stuff really apart.

But thats okay since I would see $$$ in return

No problem. I guess it's good for exercise for the brain and attention span.

Here's some questions for biology.

What is a receptor?
What makes up a receptor?
What makes up the make up of the receptor?
Why does the receptor respond to this chemical?
What attracts that chemical to that receptor?
What is the atomic elements that make up the chemical?
Is their a structural relationship between the chemical and receptor?

 

[S.A.M.]

Going on with our own discussion.

So how do you think in your work? As I said earlier, my work involves a lot of stumbling in the dark. It becomes imperative to clearly define the primary hypothesis, the primary objectives, the specific aims and then delineate the methods that will give the clearest results within the known limitations.

Now as the Platt paper describes,

Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly:
1) Devising alternative hypotheses;
2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses;
3) Carrying out the experiment so as to get a clean result;
1') Recycling the procedure, making subhypotheses or sequential hypotheses
to refine the possibilities that remain;
and so on.


On any new problem, of course, inductive inference is not as simple and
certain as deduction, because it involves reaching out into the unknown. Steps 1
and 2 require intellectual inventions, which must be cleverly chosen so that
hypothesis, experiment, outcome, and exclusion will be related in a rigorous syllogism

There are several assumptions to this paper of course:

1. it assumes a complete knowledge of all the factors that affect the experiment
2. it assumes that one can hypothetically devise all possible hypotheses arising in case of what-ifs
3. these presumptive hypotheses can be actually tested
4. there is only one "right" scientific method.

As I know from my own experience, experiments rarely follow the path you expect they will (especially when its dietary components, with unusual and unknown effects); which means that not only do we have to reevaluate what we need to do next at any stage, we are also frequently stymied as there are too many things we don't know.

In such a case how do you approach the problem?

 

[Klippymitch]

I look at the big picture and break it down into smaller and smaller sections or questions until no more sections or questions can be asked.

 

[S.A.M.]

Could you give an example?

 

[Exhumed]

So how do you think in your work?

I'm a undergraduate student so I don't really experiment in new areas (trying to beg my way into a position as a assistant with someone researching the EGF and metastasis I mentioned). But for the limited things I've done I'm guilty of thinking in the way Platt described, "does not exclude anything. It predicts everything, and therefore does not predict anything." I did the fundamental chemistry experiments Platt mentions at the beginning, yet they did not rub off on me.

In such a case how do you approach the problem?

Are you saying that in such cases you can't/have difficulty making a "logical tree" where you eliminate branches, to put it in the terms of the article?

I don't know of anyway better than strong inference. It is hard to say without getting specific though, not that I would know then either, given how complicated it sounds.

My impression from you is that in some of the advanced aspects of nutrition there is such difficulty and unknowns and problems that there is not anything wrong with the approach, it is just that the answer itself is very complicated/cannot be reduced.

It sounds a bit similar to the conference Platt described. Something that could take 100 years (maybe not that long ) going in really painstakingly small steps. That example was "The problems of how enzymes are induced, of how proteins are synthesized, of how antibodies are formed." Platt implies these were solved by strong inference.

Could the same be done with these nutritional questions? I can't say, but my guess is with a far lower level of success.

 

[Klippymitch]

Could you give an example?

Think of something you want to find out. What you want to find out is the big picture. After you find the big picture see if more questions can be asked?

What is involved?
What is the cause and effect?
What is the involved made of?
Is the make up of the involved made up of something else?
If so what is the pattern?
What is the materials used?
What is the shape?

Once everything is broken down you look for similarities.

Is involved A similar to involved B in Structure?
Is involved A similar to involved B in some other way?
In what way?

You ask a question and once the question is ask you try and see if that question can lead to more questions.

Basically everything results in size,shape, and structure in the atomic level.

The most important thing is to just keep asking yourself questions.

 

[GeoffP]

Platt's position is largely correct: the fast movers are those that remember observation, hypothesis and exclusion. But all science is tempered by the humans that write it. We allow for suboptimal performance and deviation because we are prone to it; me not least. That is humanity.

 

[S.A.M.]

Are you saying that in such cases you can't/have difficulty making a "logical tree" where you eliminate branches, to put it in the terms of the article?

I don't know of anyway better than strong inference. It is hard to say without getting specific though, not that I would know then either, given how complicated it sounds.

My impression from you is that in some of the advanced aspects of nutrition there is such difficulty and unknowns and problems that there is not anything wrong with the approach, it is just that the answer itself is very complicated/cannot be reduced.

It sounds a bit similar to the conference Platt described. Something that could take 100 years (maybe not that long ) going in really painstakingly small steps. That example was "The problems of how enzymes are induced, of how proteins are synthesized, of how antibodies are formed." Platt implies these were solved by strong inference.

Could the same be done with these nutritional questions? I can't say, but my guess is with a far lower level of success.

The approach that I prefer, in lieu of a simple elimination, is to ask one question.

Q: Is there any effect of vitamin D at all on energy metabolism?

Since cell models are by definition, tightly controlled systems not really representative of the internal in vivo milieu, I do a dose and time response effect of vitamin D on obesity in an animal model, which I hope will be easily transposable to humans (since my ultimate aim is not skinny mice)

This means that I will conduct two studies: a short term one and a long term one, using different levels of vitamin D (low, high and optimal, the optimal being the level at which overt deficiency is prevented).

During the course of my two studies I collect information about parameters affected and affecting energy metabolism (data on intake, excretion, and metabolic parameters known to influence energy metabolism).

After the studies are completed, I look at the results and use the results to describe the effects.

This is known as abductive reasoning.

Abduction, or inference to the best explanation, is a method of reasoning in which one chooses the hypothesis that would, if true, best explain the relevant evidence. Abductive reasoning starts from a set of accepted facts and infers to their most likely, or best, explanations. The term abduction is also sometimes used to just mean the generation of hypotheses to explain observations or conclusions, but the former definition is more common both in philosophy and computing.

So what do you think?

 

[S.A.M.]

Think of something you want to find out. What you want to find out is the big picture. After you find the big picture see if more questions can be asked?

What is involved?
What is the cause and effect?
What is the involved made of?
Is the make up of the involved made up of something else?
If so what is the pattern?
What is the materials used?
What is the shape?

Once everything is broken down you look for similarities.

Is involved A similar to involved B in Structure?
Is involved A similar to involved B in some other way?
In what way?

You ask a question and once the question is ask you try and see if that question can lead to more questions.

Basically everything results in size,shape, and structure in the atomic level.

The most important thing is to just keep asking yourself questions.

This sounds similar to the Socratic method of enquiry; ie question based reasoning where you present the question as a statement and ask further questions about it and about related aspects of it.

http://en.wikipedia.org/wiki/Socratic_method

 

[S.A.M.]

We have so far the following methods:

1. simple induction: causal relationship
2. strong inference: problem oriented inferential reasoning
3. abductive reasoning: generating hypotheses from observations or conclusions
4. question based reasoning or Socratic method

Any comments? Any other methods utilised?

Any advantages of one over the other? Any drawbacks?

 

[S.A.M.]

Platt's position is largely correct: the fast movers are those that remember observation, hypothesis and exclusion. But all science is tempered by the humans that write it. We allow for suboptimal performance and deviation because we are prone to it; me not least. That is humanity.

Hmm so how would you temper Platts position on strong inference (with its resident limitations).

Do you perfrom ChIP assays or microarrays? Which method of reasoning is most useful in those kinds of studies?

 

[Exhumed]

The approach that I prefer, in lieu of a simple elimination, is to ask one question.

Q: Is there any effect of vitamin D at all on energy metabolism?

Since cell models are by definition, tightly controlled systems not really representative of the internal in vivo milieu, I do a dose and time response effect of vitamin D on obesity in an animal model, which I hope will be easily transposable to humans (since my ultimate aim is not skinny mice)

This means that I will conduct two studies: a short term one and a long term one, using different levels of vitamin D (low, high and optimal, the optimal being the level at which overt deficiency is prevented).

During the course of my two studies I collect information about parameters affected and affecting energy metabolism (data on intake, excretion, and metabolic parameters known to influence energy metabolism).

After the studies are completed, I look at the results and use the results to describe the effects.

This is known as abductive reasoning.



So what do you think?

I think it's great Too bad your case does not have an easy exclusion tree like some of the examples in the article, allowing for quick progress. Platt seemed to overlook this.

I'm curious, when you do the high dose do you ever do anything about the enzymatic pathway? In humans when you increase your vitamin, i.e., coenzyme, levels with supplementation I believe it has a positive short term effect, but in the long term somewhat blocks the enzymatic pathway.

Might be going on a tangent now, but hopefully it's of interest...I googled "vitamin D enzyme pathway" just now and saw an interesting article about Vitamin D in cancer treatment. It says that they recently discovered how either of it's two enzymatic pathways are controlled, by changing one amino acid in the hydroxylase enzyme. The two pathways breakdown Vitamin D at a different speed. http://www.physorg.com/news104511326.html

 

[S.A.M.]

No problem. I guess it's good for exercise for the brain and attention span.

Here's some questions for biology.

What is a receptor?
What makes up a receptor?
What makes up the make up of the receptor?
Why does the receptor respond to this chemical?
What attracts that chemical to that receptor?
What is the atomic elements that make up the chemical?
Is their a structural relationship between the chemical and receptor?

Oops I missed these; I will answer them in the next two days. Need to figure out how to compress the information.

/thats a whole chapter or two in Lodish!!!!/