Preparing for EMNLP 2026
Preparing for EMNLP 2026
Photo: Jakub Hałun / Wikimedia, via the EMNLP 2026 website.
My colleague Khaled and I are going to EMNLP 2026. We will present at the FinNLP workshop, and I have started working through a small selection of papers before the trip. As usual, the reading list has grown faster than the time available to read it.
The reason for attending goes back to our research on evasiveness in earnings-call Q&A. An analyst asks a question; the executive gives an answer. Sometimes the answer is informative. Sometimes it addresses a nearby question, gives a reassuring description of the business, or provides plenty of detail while leaving the original question unresolved. We have been investigating how to identify these behaviours from transcripts.
One of this year’s FinEval tasks, Japanese Implicit Communication Recognition, was close enough to these interests to catch our attention. We participated in all three tasks and did rather well: first in Greek Financial NER, tied first in Japanese Implicit Communication Recognition, and second in Multilingual Financial Q&A. The results are available here, and our system paper is on the FinNLP programme. I am looking forward to discussing the tasks with the other participants.
With another colleague, Hamdan, we have been pursuing the audio side of the problem. Does the way an executive answers change what we can infer from the words? We have spent quite a bit of time on this, with some results and plenty of unresolved questions. I have described parts of that work in the recent earnings-call audio series, as well as in our preprint on residual speech embeddings for tone classification and two earlier posts: Disentangling Speech Embeddings and Serious, Sassy, or Sad?. These explore how to reduce the influence of linguistic content in speech embeddings and make vocal tone easier to analyze. This research is part of Hamdan’s PhD thesis, which I co-supervise. We still have work to do before we would be comfortable drawing strong conclusions and submitting to a Q1 journal.
Since we are attending the conference, I went through the paper listings and selected twenty papers related to financial NLP, or methods that might be useful for it. Topic modeling is one example. I have now read most of them, to varying depths. Below are some first impressions and the questions I would like to discuss with their authors.
Starting with the earnings calls
A Shaky Voice Is Not Always a Dodge is particularly close to our work. The authors introduce DualEvasion, with 505 question-answer pairs from 60 earnings calls. Each answer receives two labels: direct or evasive from the text, and confident or unconfident from the audio. An executive can sound very confident while avoiding a question. Someone else may hesitate while giving a perfectly adequate answer. The difficult part is deciding what vocal confidence measures, and how much it tells us about evasiveness.
The authors find that audio models struggle with the confidence task and suggest that they rely too much on absolute vocal characteristics. A slow speaker may be classified as uncertain even when that is their usual delivery. Supplying other recordings from the same speaker helps only modestly. Is that a limitation of the audio models themselves, or a shortcoming in how they are used?
This leaves a fairly concrete experiment to try. What happens if we supply mainly “delta” features, calculated as deviations from a speaker’s usual delivery, instead of asking a multimodal model to infer the baseline from a few clips? And would the result hold for executives the model has not seen during training? I would also like to understand how much depends on the annotation instructions and the choice of reference clips.
The dataset is rather small, and I don’t think we can draw firm conclusions from it. But I empathize with the authors: I know how tedious it is to label these answers one by one. Five hundred examples sounds small until you have spent time listening, replaying, checking speaker identity and resolving ambiguous cases. It takes a lot of time and attention, and it is not an easy task to outsource if the goal is high-quality annotation. Ideally, I would want experienced finance professionals involved.
QUORUM, which I have yet to read closely, studies how to allocate annotation between humans and LLMs under a budget. That could be useful for extending this type of dataset.
What changed in the business, and what changed in the presentation?
From Text to Alpha tracks changes in the performance metrics companies emphasize. The LLM extracts contextualized metric descriptions, and embeddings help match descriptions that use different wording across periods.
If a company stops discussing sales growth and starts emphasizing cost savings, is management responding to a change in what matters for the business, or directing attention away from a deteriorating metric? Both seem plausible. I would like to test them against subsequent fundamentals, and also examine whether analysts bring the omitted metric back into the Q&A.
CAME takes a more direct route to fundamentals by forecasting next-quarter revenue. This is an understandable target, and a potentially useful one if it can be handled across a broad universe. The evaluation covers twelve companies, so scaling and generalization remain questions for me.
My first comparison would be with analyst expectations. Does the model improve on the forecasts already available to a professional analyst? Where it disagrees with consensus, is it more often right? A lower revenue-forecast error is useful evidence, but it does not tell us how much information the forecast adds to what the market already expects.
Forecasts recorded before the answer exists
Can LLMs Take the Pulse of the Economy? has an appealing experimental premise. Historical GDP and inflation releases are widely reported and may have been memorized during pretraining. LiveMacroEval instead records hourly nowcasts for sixteen U.S. indicators before their official releases. Bloomberg consensus and specialist Federal Reserve nowcasts provide substantive comparisons; a univariate ARIMA baseline is less demanding. The market-linked scores should not be confused with ordinary forecast errors or realized portfolio returns.
My main statistical question is how many distinct releases support the conclusions. Hundreds of predictions about one release still resolve against one outcome. I would like uncertainty estimates over release events and comparisons on the same events and at the same decision times. Access to existing public forecasts also makes simple forecast combinations an important baseline.
The live dashboard makes it possible to follow the work beyond the preprint. When I inspected its October 2 update on October 19, the picture was mixed: newer models had positive aggregate LiveMacro scores, but the original models did not uniformly improve, and quarterly results differed from the longer-period results. Coverage and evaluation settings also changed, so this was not a clean experiment isolating better models. These are dated observations, not a claim about the dashboard’s latest state.
The released code makes a continuing prospective evaluation worth considering. I would like to compare model releases, retrieval setups, information sources and conventional forecast combinations under a stable protocol. That seems a useful research capability. FinDeepForecast extends prospective evaluation to company financial metrics and events as well as macro releases. Its initial ten-week experiment compares research agents on numerical forecasts and binary event questions. The corporate targets make it particularly interesting to me, although I would want comparisons with analyst consensus, simple seasonal forecasts and event base rates. The pass/fail tolerances for numerical answers also make continuous forecast errors an important complement to the headline score.
The public resources are worth bookmarking: the forecasting leaderboard, OpenFinArena GitHub repository, and Hugging Face dataset. On October 21, 2026, I checked the rendered leaderboard directly: all 13 overall entries still carried December 27, 2025 submission dates, and the completed forecast window ended December 28, 2025. The public leaderboard therefore appears stale, despite the live-benchmark framing; continued operation is unverified. I would like to revisit it as results and tools evolve, and potentially reproduce a small part of the evaluation to compare forecasting approaches. There should be useful lessons in the task definitions, evidence collection and scoring, even before considering investment applications.
A library of things an analyst knows how to do
I was initially unsure why I should read AnalysisBank. It is a paper about financial report generation. If an LLM can read the original documents and answer questions about them, why produce another report? For systematic research, I would often prefer a structured finding, its supporting evidence and a number.
The part that changed my interest was the library behind the generation system. The authors extract reusable reasoning patterns from expert reports. An entry describes a condition in the data, an analytical operation to apply, and the original passage illustrating it.
One example is a revenue line that falls sharply, but less than expected and less than comparable peers. The analytical operation is to quantify those comparisons and investigate relative resilience or possible share gains. That is a much more useful object to store than a generic label such as “negative revenue sentiment.”
Over many reports, that might become a catalogue of research ideas: what to measure, what to compare it with, and under which conditions the comparison could matter.
This also brought me back to Bayesian networks, a topic I have explored on this blog before: first through Alexander Denev’s book on probabilistic graphical models in finance, then with illustrative credit-default models for Matalan and Boparan, and later with a simpler business example, modeling the profit and loss of a cafe in Hong Kong.
A report might provide evidence about input costs, pricing ability or product mix, which would update a distribution over future margins. “Margins fall because input costs rise” and “margins rise despite input-cost pressure” concern the same topic, but call for different updates. The second might tell us something about an offsetting mechanism. Extracting those arguments would be a start; estimating the probabilities and checking the mechanisms would still be an important and difficult task.
LEAP offers a concrete way to separate interpreting evidence from calculating a forecast distribution. The LLM interprets evidence items separately; fixed mappings and a deterministic probabilistic update combine their implications with a prior. Its appendix works through an AAPL daily-high forecast, including evidence removal to show how individual items affect the result. This makes the calculation inspectable, although a leave-one-out contribution is not a causal attribution.
The question I kept returning to was calibration. The authors describe constants fixed before final evaluation, rather than a documented rolling procedure that learns evidence reliability from realized outcomes. A precisely calculated posterior can still be overconfident if its likelihood inputs are poorly calibrated. I would ask how those constants were chosen, how sensitive the results are to them, and how dependence between repeated evidence and information already in the prior is handled.
This connects to the Bayesian examples above, but it is not itself a causal model of a company. For a follow-up, I would first want a clearly defined forecast target, chronological calibration and comparisons with simple combinations using the same evidence. The code is available.
There is also an older paper I have bookmarked, Argument-Based Sentiment Analysis on Forward-Looking Statements, which extracts claims, premises and scenarios from equity research. That seems worth reading alongside AnalysisBank.
Representations I would like to try
I have used topic modeling on financial text before, including Top2Vec on 10-K business descriptions. Reading Model Directions, Not Words made me realize I have some catching up to do.
The paper builds topic models from sparse-autoencoder features extracted from a language model’s internal activations. Some examples are impressive. I would like to run it on filings and compare the results with more familiar methods: which topics are easier to interpret, and which help with a downstream task such as retrieval? Looking at a few appealing examples would only get us so far.
CrossAlpha interests me for a different representation: networks of companies derived from annual reports across markets. I found Figure 1 considerably easier to understand than the abstract. The general idea of identifying economically related firms across countries is worthwhile. The historical construction matters, though: the paper’s long backtest uses a fixed FY2024 disclosure graph. The authors distinguish this from a point-in-time trading simulation. Reconstructing the graph from what was actually available at each date would be a substantial part of any follow-up.
What information was available, and what survived the pipeline?
Point-in-time validity keeps coming up in this reading. Point-in-Time Financial RAG contains an idea I would like to explore: adapt retrieval according to which source families have helped earlier predictions. Textual relevance alone does not tell us whether a passage will improve a forecast.
Its implementation is quite coarse, tracking news versus SEC filings by event type and horizon. Still, the idea could be extended to more document types and different targets. I would be interested in learning which sources help forecast a particular KPI or a surprise relative to consensus.
The word “frozen” initially led me to ask when the LLM had been frozen. In this paper, it means the reader’s weights are unchanged during the experiment. The reader is Llama 3.1, while the test period is 2021–2023; Meta gives a December 2023 pretraining cutoff and a July 2024 release. Restricting retrieved documents to their historical availability does not remove possible hindsight from the model itself. The paper’s retrieval idea remains interesting, but that chronology limits what I can infer from the backtest. Model card
This raises the question of what the language model already knows, even when the supplied documents respect historical availability.
Summoning the Oracle to Slay It proposes an interesting intervention for this problem. Its FinCAD method first elicits historical-outcome memory using a company, a date and an optimized prompt. During generation, it contrasts this memory-eliciting branch with the normal analysis of the supplied financial context, subtracting the former’s token scores with a strength that varies by company and date. The model’s weights remain unchanged. The authors report lower historical backtest performance, relatively preserved post-cutoff performance for the larger tested models, and closer alignment between historical and post-cutoff model rankings. Code
I find the idea worth investigating, but the interpretation of the correction remains uncertain. Does it selectively suppress knowledge of future outcomes, or also remove useful prior knowledge and alter the model’s reasoning? Confidence is an imperfect proxy for memorization, and a lower historical return is not, by itself, evidence that a backtest has become unbiased. My question for the authors is what independent controls can distinguish successful removal of hindsight from other changes in behaviour. I would treat FinCAD as a sensitivity test, alongside prospective evaluation, rather than as certification that a historical backtest is clean.
All Leaks Count, Some Count More approaches the problem through claims and their influence. Its TimeSPEC pipeline checks whether individual claims in a generated explanation were knowable at the historical decision date, then regenerates the prediction using validated evidence. Its evaluation uses Shapley values to estimate how much each claim changes predictions across different subsets of claims. A temporally invalid claim receives more weight when it has greater measured influence on the decision.
This connects directly to our research with Saeed AlMarri and colleagues: Interpreting LLMs as Credit Risk Classifiers and Measuring What LLMs Think They Do. We compared LLM-generated explanations of feature importance with independently computed SHAP attributions and found discrepancies. TimeSPEC applies a related attribution idea to textual claims rather than tabular features, then connects that influence to temporal validity.
Our findings also suggest a question for the authors. If the original explanation omits information that influenced the prediction, attributing importance across its claims cannot expose that missing influence. How do we establish that the measured contributions capture contamination in the original decision, rather than sensitivity to claims supplied during a subsequent reconstruction? I like the move from counting leaked claims to assessing their influence, while retaining this distinction between an auditable explanation and a faithful account of the original prediction.
TEMPS addresses another aspect of evidence selection: a passage can concern the right subject and the wrong period. It combines ordinary semantic similarity with a separate temporal compatibility score, using an anchor date to ground expressions such as “next quarter” in calendar time. Semantic matching supplies topic continuity; the temporal branch adds alignment with the period of interest.
This could complement tracking changes in KPI emphasis by helping distinguish what is discussed from when it applies. At least two clocks matter: when a statement is made and which period it describes. When the information becomes available can be a third. My question for the authors is where learned temporal representations improve on explicitly extracting the referenced period and using it as retrieval metadata. The idea is worth investigating, but better temporal retrieval does not by itself establish more useful financial signals.
When Summaries Distort Decisions examines what happens when compression changes the judgment made from the source. There are substantive experiments behind the question: repeated full-source evaluations, different evaluators, compression budgets, and tests that restore omitted context versus placebo facts. The judgments being compared are LLM judgments, not human investment decisions. Preserving the full-source judgment is therefore a measure of consistency, not proof of correctness. The return-prediction case study also leaves me wanting absolute performance and uncertainty before drawing investment conclusions.
The takeaway I would keep is simple: a summarizer is part of the signal model. It selects and reweights information, so it should be versioned and tested like any other feature transformation. Changing its model, prompt or compression budget can change the evidence that reaches the next stage. I would evaluate those changes through downstream scores and decisions, rather than fluency alone. This gives a practical reason to preserve source passages and their qualifications alongside compressed outputs.
The reading list
For convenience, here are the twenty papers together. “Read and discussed” means I have examined the paper and made notes, with depth varying by paper; it does not mean I have reproduced the results. Seventeen papers have reached this stage. “To read” means it remains a candidate for a closer look. Links point to the preprints I collected.
I will probably arrive in Budapest with some of this list still unread. Having specific questions should at least make the poster sessions and conversations more fruitful.