Preparing for EMNLP 2026
Preparing for EMNLP 2026
Photo: Jakub Hałun / Wikimedia, via the EMNLP 2026 website.
My colleague Khaled and I are going to EMNLP 2026. We will present at the FinNLP workshop, and I have started working through a small selection of papers before the trip. As usual, the reading list has grown faster than the time available to read it.
The reason for attending goes back to our research on evasiveness in earnings-call Q&A. An analyst asks a question; the executive gives an answer. Sometimes the answer is informative. Sometimes it addresses a nearby question, gives a reassuring description of the business, or provides plenty of detail while leaving the original question unresolved. We have been investigating how to identify these behaviours from transcripts.
One of this year’s FinEval tasks, Japanese Implicit Communication Recognition, was close enough to these interests to catch our attention. We participated in all three tasks and did rather well: first in Greek Financial NER, tied first in Japanese Implicit Communication Recognition, and second in Multilingual Financial Q&A. The results are available here, and our system paper is on the FinNLP programme. I am looking forward to discussing the tasks with the other participants.
With another colleague, Hamdan, we have been pursuing the audio side of the problem. Does the way an executive answers change what we can infer from the words? We have spent quite a bit of time on this, with some results and plenty of unresolved questions. I have described parts of that work in the recent earnings-call audio series, as well as in our preprint on residual speech embeddings for tone classification and two earlier posts: Disentangling Speech Embeddings and Serious, Sassy, or Sad?. These explore how to reduce the influence of linguistic content in speech embeddings and make vocal tone easier to analyze. This research is part of Hamdan’s PhD thesis, which I co-supervise. We still have work to do before I would be comfortable drawing strong conclusions and submitting to a Q1 journal.
Since we are attending the conference, I went through the paper listings and selected twenty papers related to financial NLP, or methods that might be useful for it. Topic modeling is one example. I have worked through seven so far; the others are still on the reading list. Below are some first impressions and the questions I would like to discuss with their authors.
Starting with the earnings calls
A Shaky Voice Is Not Always a Dodge is particularly close to our work. The authors introduce DualEvasion, with 505 question-answer pairs from 60 earnings calls. Each answer receives two labels: direct or evasive from the text, and confident or unconfident from the audio. An executive can sound very confident while avoiding a question. Someone else may hesitate while giving a perfectly adequate answer. The difficult part is deciding what vocal confidence measures, and how much it tells us about evasiveness.
The authors find that audio models struggle with the confidence task and suggest that they rely too much on absolute vocal characteristics. A slow speaker may be classified as uncertain even when that is their usual delivery. Supplying other recordings from the same speaker helps only modestly. Is that a limitation of the audio models themselves, or a shortcoming in how they are used?
This leaves a fairly concrete experiment to try. What happens if we supply mainly “delta” features, calculated as deviations from a speaker’s usual delivery, instead of asking a multimodal model to infer the baseline from a few clips? And would the result hold for executives the model has not seen during training? I would also like to understand how much depends on the annotation instructions and the choice of reference clips.
The dataset is rather small, and I don’t think we can draw firm conclusions from it. But I empathize with the authors: I know how tedious it is to label these answers one by one. Five hundred examples sounds small until you have spent time listening, replaying, checking speaker identity and resolving ambiguous cases. It takes a lot of time and attention, and it is not an easy task to outsource if the goal is high-quality annotation. Ideally, I would want experienced finance professionals involved.
QUORUM, which I have yet to read closely, studies how to allocate annotation between humans and LLMs under a budget. That could be useful for extending this type of dataset.
What changed in the business, and what changed in the presentation?
From Text to Alpha tracks changes in the performance metrics companies emphasize. The LLM extracts contextualized metric descriptions, and embeddings help match descriptions that use different wording across periods.
If a company stops discussing sales growth and starts emphasizing cost savings, is management responding to a change in what matters for the business, or directing attention away from a deteriorating metric? Both seem plausible. I would like to test them against subsequent fundamentals, and also examine whether analysts bring the omitted metric back into the Q&A.
CAME takes a more direct route to fundamentals by forecasting next-quarter revenue. This is an understandable target, and a potentially useful one if it can be handled across a broad universe. The evaluation covers twelve companies, so scaling and generalization remain questions for me.
My first comparison would be with analyst expectations. Does the model improve on the forecasts already available to a professional analyst? Where it disagrees with consensus, is it more often right? A lower revenue-forecast error is useful evidence, but it does not tell us how much information the forecast adds to what the market already expects.
Two papers further down the list approach forecasting prospectively: Can LLMs Take the Pulse of the Economy? introduces a live macroeconomic nowcasting evaluation, while FinDeepForecast evaluates research agents on financial forecasting tasks. I want to look at their information cutoffs, comparison forecasts and scoring rules before deciding what to make of the results.
A library of things an analyst knows how to do
I was initially unsure why I should read AnalysisBank. It is a paper about financial report generation. If an LLM can read the original documents and answer questions about them, why produce another report? For systematic research, I would often prefer a structured finding, its supporting evidence and a number.
The part that changed my interest was the library behind the generation system. The authors extract reusable reasoning patterns from expert reports. An entry describes a condition in the data, an analytical operation to apply, and the original passage illustrating it.
One example is a revenue line that falls sharply, but less than expected and less than comparable peers. The analytical operation is to quantify those comparisons and investigate relative resilience or possible share gains. That is a much more useful object to store than a generic label such as “negative revenue sentiment.”
Over many reports, that might become a catalogue of research ideas: what to measure, what to compare it with, and under which conditions the comparison could matter.
This also brought me back to Bayesian networks, a topic I have explored on this blog before: first through Alexander Denev’s book on probabilistic graphical models in finance, then with illustrative credit-default models for Matalan and Boparan, and later with a simpler business example, modeling the profit and loss of a cafe in Hong Kong.
A report might provide evidence about input costs, pricing ability or product mix, which would update a distribution over future margins. “Margins fall because input costs rise” and “margins rise despite input-cost pressure” concern the same topic, but call for different updates. The second might tell us something about an offsetting mechanism. Extracting those arguments would be a start; estimating the probabilities and checking the mechanisms would still be an important and difficult task.
LEAP looks relevant here. Its authors propose interpreting evidence items separately and aggregating their implications into a probability distribution. I have not worked through the method yet. I particularly want to see what happens when several sources repeat the same information.
There is also an older paper I have bookmarked, Argument-Based Sentiment Analysis on Forward-Looking Statements, which extracts claims, premises and scenarios from equity research. That seems worth reading alongside AnalysisBank.
Representations I would like to try
I have used topic modeling on financial text before, including Top2Vec on 10-K business descriptions. Reading Model Directions, Not Words made me realize I have some catching up to do.
The paper builds topic models from sparse-autoencoder features extracted from a language model’s internal activations. Some examples are impressive. I would like to run it on filings and compare the results with more familiar methods: which topics are easier to interpret, and which help with a downstream task such as retrieval? Looking at a few appealing examples would only get us so far.
CrossAlpha interests me for a different representation: networks of companies derived from annual reports across markets. I found Figure 1 considerably easier to understand than the abstract. The general idea of identifying economically related firms across countries is worthwhile. The historical construction matters, though: the paper’s long backtest uses a fixed FY2024 disclosure graph. The authors distinguish this from a point-in-time trading simulation. Reconstructing the graph from what was actually available at each date would be a substantial part of any follow-up.
I have also set aside papers on inferring events from numerical time series, cross-lingual aspect-based sentiment, and SAEs for causal confounding adjustment. I haven’t had time to read these yet.
What information was available, and what survived the pipeline?
Point-in-time validity keeps coming up in this reading. Point-in-Time Financial RAG contains an idea I would like to explore: adapt retrieval according to which source families have helped earlier predictions. Textual relevance alone does not tell us whether a passage will improve a forecast.
Its implementation is quite coarse, tracking news versus SEC filings by event type and horizon. Still, the idea could be extended to more document types and different targets. I would be interested in learning which sources help forecast a particular KPI or a surprise relative to consensus.
The word “frozen” initially led me to ask when the LLM had been frozen. In this paper, it means the reader’s weights are unchanged during the experiment. The reader is Llama 3.1, while the test period is 2021–2023; Meta gives a December 2023 pretraining cutoff and a July 2024 release. Restricting retrieved documents to their historical availability does not remove possible hindsight from the model itself. The paper’s retrieval idea remains interesting, but that chronology limits what I can infer from the backtest. Model card
Several unread papers address related problems directly. Temporal Leakage in Financial News NLP examines evaluation splits. Summoning the Oracle to Slay It and All Leaks Count, Some Count More propose ways to examine or mitigate knowledge of future outcomes. I want to understand what their controls can establish, and what uncertainty remains afterwards.
There are less obvious ways to change the information reaching a model. TEMPS addresses temporal relevance in retrieval: a passage can concern the right subject and the wrong period. When Summaries Distort Decisions asks what happens when compression changes the judgment made from the source. This connects directly to my reservations about generating another layer of reports. Which caveats disappear on the way to the final score?
Finally, Human Agreement and Return Association Are Not Interchangeable Criteria is on the list because of the distinction in its title. Agreement with a human annotation, forecasting a business outcome, and producing useful investment information are different evaluations.
The reading list
For convenience, here are the twenty papers together. “Read” means I have worked through the paper and made notes; it does not mean I have reproduced the results (yet). “To read” means it remains a candidate for a closer look. Links point to the preprints I collected.
I will probably arrive in Budapest with some of this list still unread. Having specific questions should at least make the poster sessions and conversations more fruitful.