Random thought: NLP in 2010 and NLP in 2020 already felt like two different worlds. The jump was huge.
Now its growing even faster.
So Iam curious how do you think NLP will look in 2030?
What big shifts do you expect? Will it still be mostly scaling transformers or will something completely new take over?
Preparing for NLP Research Scientist Intern roles and overwhelmed by how fast the field moves.
Any advice from people who landed or hire for these roles? What do people waste time on?
Thanks
Trying for AI Research Scientist Intern roles and overwhelmed by how fast everything changes.
What actually matters most right now for interviews/hiring?
Any concrete advice from people who landed or hire for these roles? What do people waste time on?
Thanks.
Now we have RAG and agentic AI (tools + reasoning). Are making good datasets and fine-tuning old methods? Or are they still better for making AI smarter in special areas?
Link: https://arxiv.org/abs/2606.31608
Summary:
Large language models ace medical exams but struggle with real clinical reasoning. This new paper introduces CLExEval using progressive information masking on rare cases + 5,600 physician annotations.
Key findings:
- Verbosity Bias: GPT-4o-mini accuracy drops from 95% to 32.5% with less info
- Hidden Knowledge Paradox in specialist models
- High Reasoning-Output Mismatch (~69%)
- LLM judges approve a shocking % of clinically wrong outputs
Why it matters: Highlights the evaluation illusion where fluent text masks real failures in high-stakes domains.
What do you think? Is human-in-the-loop evaluation the way forward for clinical AI, or are there better approaches?
(Genuinely interested in discussion)
Link: https://arxiv.org/abs/2606.31608
Summary:
Large language models ace medical exams but struggle with real clinical reasoning. This new paper introduces CLExEval using progressive information masking on rare cases + 5,600 physician annotations.
Key findings:
- Verbosity Bias: GPT-4o-mini accuracy drops from 95% to 32.5% with less info
- Hidden Knowledge Paradox in specialist models
- High Reasoning-Output Mismatch (~69%)
- LLM judges approve a shocking % of clinically wrong outputs
Why it matters: Highlights the "evaluation illusion" where fluent text masks real failures in high-stakes domains.
What do you think? Is human-in-the-loop evaluation the way forward for clinical AI, or are there better approaches?
(Genuinely interested in discussion)
