I am in academia and worked in NLP although I would describe myself as NLP adjacent.

I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app.

That said traditional NLP has its uses.

Using the VADER model for sentiment analysis while flawed is vastly cheaper than LLMs to get a general idea. Traditional NLP is suitable for many tasks people are now spending a lot of money asking GPT to do just because they know GPT.

I recently did an analysis on a large corpus and VADER was essentially free while the cloud costs to run a Llama based sentiment model was about $1000. I ran both because VADER costs nothing but minimal CPU time.

NLP can be wrong but it can’t be jailbroken and it won’t make stuff up.

As an NLP professor, yes, I think we're mostly screwed - saying LLMs are a dead end or not a big deal, like some of the interviewed say, is just wishful thinking. A lot of NLP tasks that were subject of active research for decades have just been wiped out.

Ironically, the tasks that still aren't solved well by LLMs and can still have a few years of life in them are the most low-level ones, that had become unfashionable in the last 15 years or so - part-of-speech tagging, syntactic parsing, NER. Of course, they have lost a lot of importance as well: you no longer need them for user-oriented downstream tasks. But they may still get some use: for example NER for its own sake is used in biomedical domains, and parsing can be useful for scientific studies of language (say, language universals related to syntax, language evolution, etc.). Which is more than you can say about translation or summarization, which have been pretty much obsoleted by LLMs. Still, even with these tasks, NLP will go from broad applicability to niche.

I'm not too worried for my livelihood at the moment (partly because I have tenure, and partly because the academic system works in such a way that zombie fields keep walking for quite long - there are still journals and conferences on the semantic web, which has been a zombie for who knows how long). But it's a pity: I got into this because it was fun and made an impact and now it seems most of my work is going to be irrelevant, like those semantic web researchers I used to look down at. I guess my consolation is that computers that really understand human language was the dream that got me into this in the first place, and it has been realized early. I can play with it and enjoy it while I sink into irrelevant research, I guess :/ Or try to pivot into discrete math or something.

Unless we intend to surrender everything about human symbolic manipulations (all math, all proving, all computations, all programming) to llm in the nearest future, we still need some formal representations for engineering.

The major part of tradidional NLP was about formal representations. We are still to see the efficient mining techniques to extract the formal representations and analyses back from LLM.

How would we solve the traditional NLP problems, such as, for example, formalization of law corpus of a given country with LLM?

As an approximation we can look at non-natural language processing, e.g. compiler technologies. How do we write an optimizing compiler on LLM technologies? How do we ensure stability, correctness and price?

In a sence, the traditional NLP field has just doubled, not died. In addition to humans as language capable entities, who can not really explain how they use the language, we now also have LLM as another kind of language capable entities. Who in fact also can not explain anything. The only benefit is that it is cheaper to ask LLM the same question a million of times that a human.

As someone deeply involved in NLP, I’ve observed the field’s evolution: from decades of word counting and statistical methods to a decade of deep learning enabling “word arithmetic.” Now, with Generative AI, we’ve reached a new milestone, a universal NLP engine.

IMHO, the path to scalability often involves using GPT models for prototyping and cold starts. They are incredible at generating synthetic data, which is invaluable for bootstrapping datasets and also data labelling of a given dataset. Once a sufficient dataset is available, training a transformer model becomes feasible for high-intensity data applications where the cost of using GPT would be prohibitive.

GPT’s capabilities in data extraction and labeling are to me the killer applications, making it accessible for downstream tasks.

This shift signifies that NLP is transitioning from a data science problem to an engineering one, focusing on building robust, scalable systems.

CNNs were outperforming traditional methods on some tasks before 2017.

Problem was that all of the low level tasks , like part of speech tagging, parsing, named entity recognition , etc. never resulted in a good summarizing system or translating system.

Probabilistic graphical models worked a bit but not much.

Transformers were a leap, where none of the low level tasks had to be done for high level ones.

Pretty sure that equivalent leap happened in computer vision a bit before.

People were fiddling with low level pattern matching and filters and then it was all obliterated with an end to end cnn .

If Chomsky was writing papers in 2020 his paper would’ve been “language is all you need.”

That is clearly not true and as the article points out wide scale very large forecasting models beat that hypothesis that you need an actual foundational structure for language in order to demonstrate intelligence when in fact is exactly the opposite.

I’ve never been convinced by that hypothesis if for no other reason that we can demonstrate in the real world that intelligence is possible without linguistic structure.

As we’re finding: solving the markov process iteratively is the foundation of intelligence

out of that process emerges novel state transition processes - in some cases that’s novel communication methods that have structured mapping to state encoding inside the actor

communications happen across species to various levels of fidelity but it is not the underlying mechanism of intelligence, it is an emerging behavior that allows for shared mental mapping and storage

Great seeing Ray Mooney (who I took a graduate class with) and Emily Bender (a colleague of many at the UT Linguistics Dept., and a regular visitor) sharing their honest reservations with AI and LLMs.

I try to stay as far away from this stuff as possible because when the bottom falls out, it's going to have devastating effects for everyone involved. As a former computational linguist and someone who built similar tools at reasonable scale for largeish social media organizations in the teens, I learned the hard way not to trust the efficacy of these models or their ability to get the sort of reliability that a naive user would expect from them in practical application.

Lots of great quotes in this piece but this one stuck out for me:

> TAL LINZEN: It’s sometimes confusing when we pretend that there’s a scientific conversation happening, but some of the people in the conversation have a stake in a company that’s potentially worth $50 billion.

I think the AI hype (much of which is justified) detracts from the fact that we have Actually Good NLP at last. I've worked on NL2SQL in both the before and after times, and it's still not a solved problem, but it's frustrating to talk to AI startup people who have never really thought deeply about named entity recognition, disambiguation etc. The tools are much, much better. The challenges and pitfalls remain much the same.

As someone who dropped out of NLP during the chaos, all this stuff honestly feels way too familiar - progress is cool but watching your work become pointless overnight stings hard.

I’m curious how have large language models impacted linguistics and particularly the idea of a universal grammar?

Has there been an LLM that reliably does not ignore the word "not"?

Because I'm pretty sure that's a regression compared to most prior NLP.

"It helps to have tenure when something like this happens."

I was contrasting FiNER, GliNER, and Smolagents in a recent blog post on my substack and while the first two are fast and provide somewhat good results, running a LLM locally is 10x better easily.

For me as a lay-person, the article is disjointed and kinda hard to follow. It's fascinating that all the quotes are emotional responses or about academic politics. Even now, they are suspicious of transformers and are bitter that they were wrong. No one seems happy that their field of research has been on an astonishing rocketship of progress in the last decade.

Have we already forgotten what AlexNet did to Computer Vision as a research domain?

The field is natural language processing.

My view is that "traditional" NLP will get re-incorporated into LLMs (or their successors) over time. We just didn't get to it yet. Appropriate inductive biases will only make LLMs better, faster and cheaper.

There will always be trouble in LLM "paradise" and desire to take it to the next level. Use raw-accessed (highest performing) LLM, intensely, for coding and you will rack up $10-$20/hr bill. China is not supposed to have adequate GPUs at their disposal, - they will come up with smaller and more efficient models. Etc, etc, etc...