u/NeighborhoodFatCat

I feel like I am unable to digest most machine learning research these days. The primary reason is because I feel like they have gotten tremendously wordy. I find it not uncommon to find research papers consisting of 20+, 30+, 40+ pages of pure texts. An example is https://arxiv.org/pdf/2302.13971 I understand this is in the context of LLM but I've also noticed this trend outside of LLM, even in things like optimization.

Also because the papers seem to have gotten wordy, it is possible to come across some undefined (suspected) new terminology due to word usage. An example (and not a research paper per-se) is https://transformer-circuits.pub/2021/framework/index.html Ok, if you just scroll down you do find that blog is interspersed with mathematical formulas, but the writing just feels off. The authors mixes colloquial conversational structure into a scientific publication "a lot of ...", "we don't feel....", "it feels like...". This makes thing unreadable in my honest opinion.

At the same time, I feel mathematical explanation are becoming rarer in some research areas. There seems to be an over-reliance on "block diagrams", except diagrams misses a lot of details and virtually impossible to translate into math/code without a bunch of guess works.

Looks like game over for liberal art colleges. Thoughts?

I think some of math has taught me, such as

• understanding/questioning definition deeply,
• checking assumptions carefully,
• checking for logical gaps,
• finding optimal solution,
• probabilistic reasoning

has made it more difficult for me to navigate through society, especially social situations.

I guess because society is just very arbitrary and not "logically tight" if that makes sense. There is a lot of "looseness" in conventional thinking. However, I found people do not actually care to resolve those logical looseness. In fact, logical looseness, information obfuscation, lack of optimality seem to be crucial in how society functions.

And if you ask for clarification or suggest optimal solutions or even try to identify a logical looseness, you are really seen as a weirdo.

For example, just today my new job wants me to get a non-criminal record so I went down to the police station, filled out a form. It doesn't say what time period (start and end date) the record is for, so I asked the staff and they gave me a rude response: "of course the end date is today! how can we check your non-criminal record for tomorrow, which hasn't happened yet?" The back of my mind I'm just thinking: but it all depends on when you start to check, isn't it? There is no rule that says you must check the record up to today....

Wonder if anyone else has had similar experiences.

I feel that reproducibility is now a lost cause in machine learning research for three reasons:

1. Many research is moving towards the physical AI territory, where you need expensive hardwares or even entire laboratories with high-speed cameras, in order to perform an experiment. You truly have no idea if the experiment can be reproduced and have to trust the demo. But demos are not perfectly reliable. Plus people are incentivized to only show the part of the demo that works. The entire system can fall apart the moment the recording stops.

2. You have big AI companies releasing various tools, which they claim to solve a host of problems with certain amount of accuracy or efficiency. Unless you work at those companies there is really no proof of that and you will have to take their words on it. They have strong financial incentive to blow-up those figures. There is no solid way to check it either because the problem that they solve are so vague and subjective.

3. We need to address the elephant in the room which is that people are incentivized to produce non-reproducible work to prevent their lunch being eaten by their competitors or looking bad. That's why some of us will probably never get a reply when we email the authors for their code.

So what now? Maybe everything will be OK because we can contrast it with scientific progress in earlier parts of history, e.g., building the atomic bomb or sending people to the moon. These projects had low "outside reproducibility" but high "internal reproducibility". Plus all these work were mathematical in nature and carefully checked. But I don't think many areas of machine learning research is like that. What do you think? Should reproducibility be abandoned? If not how is it best implemented going forward?

From my experience, the first moment of interaction between two humans is "exploitation".

The person looks at you up and down (and you can actually see them doing this), starts checking if you are an in-group or an out-group, decides how to react to you all the while formulating ideas on how they can use you for their own gains.

• If you are an in-group: how to use you (briefly) to climb the social ladder, extract information, to show off, or to feel better about one's self
• If you are an out-group: how to extract money out of you, use you for a brief moment of pleasure

All interactions are lingered with fakeness, as if both parties are engaging in some kind of theatrical production. We mask our wants and desires through this fake language, fake gesture. Almost archaic in some ways. Have you noticed how we tend to use archaic language in formal situations? Who says "greetings" and "best regards" in real life?

The only true interactions you get are interactions which short and self-terminating. Engage with them for a brief moment in a safe-space and never see them again. A small talk with a stranger on the bus. A brief glimpse. An exchanged smile. That's it.

Every once in a while, I encounter a new tech trend. For example:

1. Nano
2. Internet of Things
3. Edge computing
4. Fog computing
5. Ubiquitous computing
6. Digital twin
7. Zero trust architecture
8. Federated learning
9. World model
10. Industry 5.0
11. 6G mobile

...

What I find is that there will be many researchers working on it. There does not seem to be a consensus as to exactly what they are working on, and proposed solution can be wildly different from each other. There is also sometimes a lack of agreement what something actually is, "what is a world model?".

Another feeling I get is that some of the technologies are rebranding older technologies or doing something in a silly way, such as "digital twin" introducing huge amount of potential compounding error, latency and additional points-of-failure through a computerized model of the actual system.

I've also noticed that industry adoption is quite varied and sometimes drastically different from the success that is portrayed by academia. I remember there was an article on "differential privacy" and it was reported the industry adoption is near 0, despite thousands of research paper a year on this topic.

How do you assess if a technology is genuine and not simply a rebranded hype usually by academics to publish papers? I mean ultimately we all want to advance engineering and create cool tech, and not simply just beating around the bush.

1. Wanting to get tenure. A tenure is used to protect you from getting fired due to controversial ideas, which could make sense if you are a philosophy professor studying neo-colonialism. But most of the professor nowadays are conformists or fully ideologically aligned with the 1% elites. It is not worth the mental struggle push years worth of junk so to voice completely uncontroversial opinions.

2. Directly competing with big tech companies, military or government agencies. Half of the mission statement you see nowadays in STEM is "Unlocking General Artificial Intelligence", "Making Physical AI Learn by Themselves!" But that's exactly what EVERYONE and their contractors are doing, and they have way more funding than some academic at no name college with 2 graduate students and a Windows XP computer lab. What happened to doing something that wasn't completely and fully aligned with industry?

3. Publishing in trash journals and not getting called out on it. If there is anything academia is good at, then it has to be being polite at people who are obviously publishing one junk paper after another, or assembling an entire cohort of people publishing junk together. There is a professor at U of Chicago who is/has published over 200+ pieces of AI-generated junks in 2026 and nobody even bats an eye. When did the idea that "any conference/journal is reputable" become the dominant train of thought in academia?

4. Being a willing slave to industry. A lot of so-called "academic-industry collaboration" is just industry outsourcing some of their specialize work to highly-trained people in academia at below market value and somehow academia thinks its a glorious thing to have on one's CV. "Yippee, I underpaid a young person!"

5. Recruiting new students with the aim of getting them to work in academia. The typical way that academia creates a job vacancy is when someone gets arrested or dies (honestly, what else?). Where are these new PhD students/post-docs suppose to go? If industry, then why not train them for industry? How is that teaching assistantship or lecturer position supposed to help the student to land in industry? All the new PhD students are in this limbo of cannot go onto academia because there are no spots, and couldn't get into industry either because they have no non-academic experience.

So there are these alcohol-based sanitary wipes that claim to kill 99% of the bacteria. It is 99% because some bacteria always survives and scientist even said that's how we get superbugs.

Ok, so use the same logic. Send a bunch of bacteria to the moon or some other planet. Let the bacteria get killed by all that space magic. But that 1% that survives? It's gonna pee and poo and produce useful gases.

It's going to make small progress but humans can send food up there to feed them and let them grow.

BAM! Now you have a working atmosphere from all the bacterias. You've just successfully terrformed a planet.

Feng Tian, an assistant professor of neurology at HMS and head of a lab at BIDMC, was arrested by the Brookline Police Department two weeks ago and faces two counts each of assault with a dangerous weapon and assault and battery on a pregnant victim, among other charges, in connection with reported domestic violence in 2023 and 2026.

Tian has continued to give research-related instructions members of his lab, which studies neurons susceptible to neurodegenerative diseases including Alzheimer’s disease.

Tian continued sending research-related instructions in the WeChat group as recently as Monday morning, when he asked lab members to join a Slack channel.

Publishing while perishing.

Google AI says YES, but I feel like you'd pretty much be sun-burned to death before diarrhea can happen.