Two points I'm hoping you can help clarify:

> Researchers ... found that an increased fMRI signal is associated with reduced brain activity in around 40 percent of cases.

So it's not just that they found it was uncorrelated, they found it was anticorrelated in 40% of cases?

And you are suggesting that conclusion suffers from the same potential issues as these fMRI studies in general?

Like you mention, it seems to me if we wanted to really validate the model, we'd have to run the same experiment with two, three, or maybe even more different modalities (fMRI, PET with different tracers, etc).

Instead of a nothingburger, you could have used your academic prowess to break down the top 1/2 misconceptions with expertise.

You might not have time to respond to all the comments but a couple of clarifications could have helped anyone else who doesn't comment without experience.

Just saying that next time you can be the change you want to see in HN instead of wasting text telling us how ignorant we are.

So here you say quite a mouthful. If you train it on a pattern it'll see that pattern everywhere - think about the early "Deep Dream" trippy-dogs-pictures nonsense that was pervasive about eight or nine years ago.

I repaired a couple of cameras for someone who was working with a large university hospital about 15 years ago, where they were using admittedly 2010s-era "Deep Learning" to analyse biopsy scans for signs of cancer. It worked brilliantly, at least with the training materials, incredible hit rate, not too terrible false positive rate (no biggie, you're just trying to decide if you want to investigate further), really low false negative rate (if there was cancer it would spot it, for sure, and you don't want to miss that).

But in real-world patient data it went completely mental. The sample data was real-world patient data, too, but on "uncontrolled" patients, it was detecting cancer all over the place. It also detected cancer in pictures of the Oncology department lino floor, it detected cancer in a picture of a guy's ID badge, it detected cancer in a closeup of my car tyre, and it detected cancer in a photo of a grey overcast sky.

Aw no. Now what?

Well, that's why I looked at the camera for them. They'd photographed the biopsies with one camera on site, from "real patients", but a lot of the "clear" biopsies were from other sites.

You're ahead of me now, aren't you?

The "Deep Learning" system had in fact trained itself on a speck of shit on the sensor of one of the cameras, the one used for most of the "has cancer" biopsies and most of the "real patient under test" biopsies. If that little blob of about a dozen slightly darker pixels was present, then it must be cancer because that's what the grown-ups told it. The actual picture content was largely irrelevant because the blob was consistent across all of them.

I'm not too keen on AI in healthcare, not as a definitive "go/no-go" test thing.

I think your expertise would be very welcome, but this comment is entirely unhelpful as-is. Saying there are bad comments in this thread and also that there is good literature out there without providing any specifics at all is just noise.

You don't have to respond to every comment you see to contribute to the discussion. At minimum, could you provide a hint for some literature you suggest reading?

Condolences.

[0] https://www.siemens-healthineers.com/en-us/magnetic-resonanc...

The literature is huge, and my bias is that I believe most of the only really good fMRI research is methodological research (i.e. about what fMRI actually means, and how to reliably analyze it). Many of the links I've provided here speak to this.

I don't think there is much reliable fMRI research that tells us anything about people, emotions, or cognition, beyond confirming some likely localization of function to the sensory and motor cortices, and some stuff about the Default Mode Network(s) that is of unclear importance.

A lot of the more reliable stuff involves the Human Connectome Project (HCP) fMRI data, since this was done very carefully with a lot of participants, if you want a place to start for actual human-relevant findings. But the field is still really young.

The BOLD signal, the thing measured by fMRI, is a proxy for actual brain activity. The logic is that neural firing requires a lot of energy and so active neurons will being using more oxygen for their metabolism, and this oxygen comes from the blood. Thus, if you measure local changes in the oxygenation of blood, you'll know something about how active nearby neurons are. However, it's an indirect and complicated relationship. The blood flow to an area can itself change, or cells could extract more or less oxygen from the blood--the system itself is usually not running at its limits.

Direct measurements from animals, where you can measure (and manipulate) brain activity while measuring BOLD, have shown how complicated this is. Nikos Logathetis and Ralph Freeman's groups, among many others did a lot of work on this, especially c. 2000-2010. If you're interested, you could check out this news and views on Logathetis's group's 2001 Nature paper [1]. One of the conclusions of their work is that BOLD is influenced by a lot of things but largely measure the inputs to an area and the synchrony within it, rather than just the average firing rate.

In this paper, the researchers adjust the MRI sequences to compare blood oxygenation, oxygen usage, and blood flow and find that these are not perfectly related. This is a nice demonstration, but not a totally unexpected finding either. The argument in the paper is also not "abandon fMRI" but rather that you need to measure and interpret these things carefully.

In short, the whole area of neurovascular coupling is hard--it includes complicated physics (to make measurements), tricky chemistry, and messy biology, all in a system full of complicated dynamics and feedback.

[1] https://www.nature.com/articles/35084300

Or worse, your whole field can be insitutionally blind to its own failings and randoms outside of it actually DO know more than you!. Chiropractors are literally worthless and being told "oh you don't get it bro" by them is their cope for being scammers, not an example of "their Gell Mann amnesia"

I don't think there would be much clear guidance for them on how to interpret any such fMRI abnormality on its own, but it might still be something useful for further investigations, and this might especially be the case for surgery. It might also have been done as part of research, if you consented to anything like that?

I am NOT an expert on fMRI in medical contexts, but you can surely get a rough idea of the potential value of fMRI with a quick search: https://scholar.google.ca/scholar?q=fMRI+surgery+brain&hl=en...

Direct link to the poster presentation: http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf

Risk of false positives in fMRI of post-mortem Atlantic salmon (2010) [pdf] - https://news.ycombinator.com/item?id=15598429 - Nov 2017 (41 comments)

Scanning dead salmon in fMRI machine (2009) - https://news.ycombinator.com/item?id=831454 - Sept 2009 (1 comment)

Indeed, there's been quite a few studies [1] that find just including any old image of a brain with stuff highlighted will cause a paper to be perceived as more scientifically credible.

[1] https://pubmed.ncbi.nlm.nih.gov/17803985/

Influencers in general are always suspect. The things that get you an audience fast are trolling or tabloid-ish tactics like conspiracism.

There are good ones but you have to be discerning.

Even if neuronal activity is (obviously) faster, the (assumed) neuro-vascular coupling is slower. Typically there are several seconds till you get a BOLD response after a stimulus or task, and this has nothing to do with fMRI sampling rate (fNIRS can have much faster sampling rate, but the BOLD response it measures is equally slow, too). Think of it as that neuronal spiking happens in a range of up to some hundred milliseconds and the body changing the blood flow happens much slower than that.

The issue is that measuring the BOLD response, even in best case scenario, is a very very indirect measure of neuronal activity. This is typically lost when people referring to fMRI studies as discovering "mental representations" in the brain and other non-sense, but here we are. Criticising the validity of the BOLD response itself, though, is certainly interesting.

fMRI's are being used in TBI/Concussion recovery that are study backed and seem to be delivering results.

(I mention this so more people can know the list exists, and hopefully email us more nominations when they see an unusually great and interesting comment.)

p.s. more on the salmon paper in this thread:

https://news.ycombinator.com/item?id=46291600

https://news.ycombinator.com/item?id=46288560

https://news.ycombinator.com/item?id=46288557

Nah, it's not noise. It's a useful reminder not to take any comments too seriously and that this topic is far outside the average commenter's expertise.

It's why I generally only ask questions, or ask for clarification instead of directly challenging something I think might be wrong now in threads that aren't related to something I have deeeeep personal knowledge of. I know when I'm out of my area, and don't want to add to the ignorance.

[0] https://www.youtube.com/watch?v=J-SHgZ1XPXs

https://www.wired.com/2009/09/fmrisalmon/

Reviewing the HN docs, https://news.ycombinator.com/bestcomments?h=168 might also be a good summary link.

I say this as a psychologist who is advising you to ignore all claims to the contrary, because they are misinformed. It is clear from the literature.

This is incorrect, TUM has a PET scanner (site in German): https://nuklearmedizin.mri.tum.de/de/Patienten-Zuweiser/Pet-... Can't comment regarding the other observations.

Also saw this irl with a particular NGS diagnostic. This model was initially 99% accurate, P.I. smelled BS, had the grad student crunch the numbers again, 96% accurate, published it, built a company around this product —-> boom, 2 years later it was retracted because the data was a lot of amplified noise, spurious hits, overfitting.

I don’t know jack compared to the average HN contributor, but even I can smell the BS from a mile away in some of these biomedical AI models. Peer review is broken for highly-interdisciplinary research like this.

You test your DL decoder on held-out data. This is the common practice.

Within-subject effects (this happens when one does A, but not when doing B) can be fine with small sample sizes, especially if you can repeat variations on A and B many times. This is pretty common in task-based fMRI. Indeed, I'm not sure why you need >2 participants expect to show that the principle is relatively generalizable.

Between-subject comparisons (type A people have this feature, type B people don't) are the problem because people differ in lots of ways and each contributes one measurement, so you have no real way to control for all that extra variation.

This all makes sense because fMRI tracks metabolic activity via oxygenation changes, which is much more clearly and plausibly related to tissue health and recovery. In these cases, it is also most likely being used within-subject (i.e. longitudinally) to make comparisons to baselines, rather than in an attempt to make speculative inferences about the mind using groups of people, and likely is a simple comparison to baseline rather than bespoke statistical analyses relying on questionable assumptions about the BOLD response being related to overly-specific kinds of neural activity.

Interesting. Do you happen to have any more information on this topic? I ask because I was under the impression that concussions are a functional/metabolic injury and not a structural injury, therefore, concussions are not visible on any type of fMRI, CT Scan, etc.. Though, I haven't looked into this topic for almost half a decade, so I imagine things have likely progressed.

Being all "PC" and "nice" about stuff that is what it is, or isn't -- THAT adds to ignorance.

You would still in general need many subjects to show the same basic within-subject patterns if you want to claim the pattern is "generalizable", in the sense of "may generalize to most people", but, precisely depending on what you are looking at here, and the strength of the effect, of course you may not need nearly as much participants as in strictly between-subject designs.

With the low test-retest reliability of task fMRI, in general, even in adults, this also means that strictly one-off within-subject designs are also not enough, for certain claims. One sort of has to demonstrate that even the within-subject effect is stable too. This may or may not be plausible for certain things, but it really needs to be considered more regularly and explicitly.

All to say, this application might not fall in the 40%.

I just find articles like these can't help but feel like they have an agenda to undermine something instead of simply acknowledge the kinds of things it is and isn't working for.

There's no doubt these researchers have found something, but the need for sensationalistic headlines is well known in academia as well.

Sometimes it's noticeable where the research is specific in scope, but the findings are more general and broad.

That might be what you're referring to as functional?

Metabolically, or otherwise, if the brain can't operate, other things in the body such as metabolism would be impacted for sure when it can't oversee and run as it normally can?

While I'm not sure if a concussion directly is visible or not (some have sizeable enough brain bleeds that can be visible), concussions to the extent that they are a change in blood circulation changes and issues, can be visualized on fMRI, etc, where it's not regular, those areas suffer in a brain.

Things luckily have progressed and quite exciting.

Out of convenience, I'll share one I know about (no affiliation) that lay out their therapies and the science behind it as well.

Effectively (I hope I'm getting this accurately) it seems the blood vessels in the brain also have signalling from the blood and oxygen that gets affected which affects things downstream from there.

These guys do an fMRI baseline, have you jump on a bike, fMRI again, see what's not getting blood, and then give you exercises and activites for those regions of the brain. It's pretty interesting.

https://www.cognitivefxusa.com/treatment

Some reported patient outcomes: https://www.cognitivefxusa.com/our-patients

Blog links to research: https://www.cognitivefxusa.com/blog

Independently of this I've heard QEEGs can do a similar thing of seeing where brain activity is/isn't baseline.

These can be measured themselves separately (that's exactly what they did here!) and if there's a spatial component, which the figures sort of suggest, you can also look at what a particular spot tends to do. It may also be interesting/important to understand why different parts of the brain seem to use different strategies to meet that demand.

Among other challenges, when we first submitted the poster to the Human Brain Mapping conference we got kicked out of consideration because the committee thought we were trolling. One person on the review committee said we actually had a good point and brought our poster back in for consideration. The salmon poster ended up being on a highlight slide at the closing session of the conference!

EDIT: And kudos to you and your advisor here.

EDIT2: I will also say that a lot of the research on fMRI methods is very solid and often quite reproducible. I.e. papers that pioneer new analytic methods and/or investigate pipelines and such. There is definitely a lot of fMRI research telling us a lot of interesting and likely reliable things about fMRI, but there is very little fMRI research that is telling us anything reliably generalizable about people or cognition.

cog neuro labs need to start organizing their research programs more like giant physics projects. Lots of PIs pooling funding and resources together into one big experiment rather than lots of little underpowered independent labs. But it’s difficult to set up a more institutional structure like this unless there’s a big shift in how we measure career advancement/success.

Task-based fMRI has similar individual variability, but with an added complication: adaptive cognition. Once you've performed a task, your brain responds differently the second time. This happens when studies reuse test questions—which is why psychological research develops parallel forms. But adaptation occurs even with parallel forms (commonly used in fMRI for counterbalancing and repeated assessment) because people learn the task type itself. Adaptation even happens within a single scanning session, where BOLD signal amplitude for the same condition typically decreases over time.

These adaptation effects contaminate ICC test-retest reliability estimates when applied naively, as if the brain weren't an organ designed to dynamically respond to its environment. Therefore, some apparent "unreliability" may not reflect the measurement instrument (fMRI) at all, but rather highlights the failures in how we analyze and conceptualize task responses over time.

It's not at all clear to me that teenagers' brains OR behaviours should be stable across years, especially when it involves decision-making or emotions. Their Figure 3 shows that sensory experiments are a lot more consistent, which seems reasonable.

The technical challenges (registration, motion, etc) like things that will improve and there are some practical suggestions as well (counterbalancing items, etc).

Funny, because these exact measures [0] were brought up in response to a similar claim I made over a year ago [1] about the resolution of our instrumentation.

There would appear to be a worrying trend of faith in scientism, or the belief that we already have all the answers squirreled away in a journal somewhere.

[0] https://news.ycombinator.com/item?id=41834346

[1] https://news.ycombinator.com/item?id=41807867

But I don't think we are really disagreeing on anything major here. I do think there is likely some useful potential locked away in carefully designed resting-state fMRI studies, probably especially for certain chronic and/or persistent systemic cognitive things like e.g. ADHD, autism, or, perhaps more fruitfully, it might just help with more basic understanding of things like sleep. But, I also won't be holding my breath for anything major coming out of fMRI anytime soon.

And yeah, part of why we need more within-subject and longitudinal designs is to get at precisely the things you mention. There is no way to know if the low ICCs we see now are in fact adaptation to the task or task generalities, if they reflect learning that isn't necessarily task-relevant adaptation (e.g. the subject is in a different mood on a later test, and this just leads to a different strategy), if the brain just changes far more than we might expect, or all sorts of other possibilities. I suspect if we ever want fMRI to yield practical or even just really useful theoretical insights, we definitely need to suss out within-subject effects that have high test-retest reliability, regardless of all these possible confounds. Likely finding such effects will involve more than just changes to analysis, but also far more rigorous experimental designs (both in terms of multi-modal data and tighter protocols, etc).

FWIW, we've also noticed a lot of magic can happen too when you suddenly have proper longitudinal data that lets you control things at the individual level.

I'm not sure what you mean by "experimental psychology" though. There are areas like psychophysics that are arguably experimental and have robust findings, and there are some decent-ish studies in clinical psychology too. Here the group sizes are probably actually mostly not too bad.

Areas like social psychology have serious sample size problems, so might benefit, but this field also has serious measurement and reproducibility problems, weak experimental designs, and particularly strong ideological bias among the researchers. I'm not sure larger sample sizes would fix much of the research here.

I can believe it; but a change doesn't have to be sufficient to be ncessary.

Obviously we hope what we learn from e.g. psychology and fMRI will help us explain more things about the mind, and surely most researchers in psychology hope their research will help us get some answers on things related to qualia as well. And almost certainly most good / consistent reductionist researchers must believe that qualia arise from the brain, at least in significant part.

However, precisely by this reductionist logic, and since it is immediately and phenomenally clear that the rate of change of qualia in the mind (or the "amount" of different qualia, i.e. images or sounds that one can process or generate in the mind in under a second) is incredibly fast, it follows immediately and logically without any need for an experiment that fMRI cannot have the temporal resolution needed for a rich understanding of the mind, simply based on knowing the TR (temporal sampling resolution) is so poor. And yet, I also find a lot of people in scientific brain research go oddly silent or seem to refuse to accept this argument unless some strange sort of published, quantificationist operationalization can be pointed to (hence my pre-emptive mentioning information transmission in neurons in under 100ms).

I'm not sure I'd call this scientism, exactly, I tend to see it as "selective quantificationism", i.e. that certain truths can only be proven as true if you introduce some kind of numerical measurement procedure and metrical abstraction. Like, no one demands a study with Scoville units to prove that e.g. a ghost pepper is at least an order of magnitude hotter than candied ginger, even though this is as blazingly obvious as the fact that the mind moves too fast for something that can barely capture images of the brain at a rate of two per second.

I think I broadly agree with you though that credulousness to (statistically and methodologically weak) scientific / technological claims mostly comes down to vibes and desires / needs, and not statistical significance, logical rigor, evidence, or etc.

Where needs / desires are high, vibes will (often) win over rationality, and vice-versa. It is easier for people to be objective about science that doesn't really clearly matter in any obvious direction, or at all. fMRI is "the mind", and thus consciousness, and so unfortunately reduces rational evaluation in much the same way speculation about AI and "consciousness" and etc does. *Shrug*