How We Built an LLM Pipeline That Detects Buying Intent in Telegram Chats

People say out loud what they need. Someone posts "moving to Antalya in September, any agent who actually answers?" in a city chat, gets three replies from friends, and never opens a search engine. The demand existed for about four hours and then disappeared.

Catching that is a classification problem before it is a lead-generation problem. This article is about the pipeline we built to solve it — what breaks, what it costs, and how we measure whether it works.

The actual problem: "I need this" vs "I do this"

The naive framing is "find messages about real estate." That framing produces a useless product, because in any chat about a topic, most people talking about the topic sell it.

Four messages, all containing the same vocabulary:

A keyword filter scores all four identically. It has no representation for the difference between wanting a thing and offering it, because that difference lives in the intent of the sentence, not in its nouns. This is the single reason keyword-alert bots feel like a firehose of garbage: they are answering a different question than the user asked.

So the job is not search. The job is classification under a per-niche definition of what a request is — and that definition is the product.

Why a pre-filter is not an optimization

Take a few hundred active public sources. That is tens of thousands of messages a day, and it grows linearly with every source you add.

Sending all of it to a language model is where most projects in this category quietly die. The arithmetic is unforgiving: your cost scales with total chat volume, while your revenue scales with delivered leads — and delivered leads are a tiny fraction of total volume. Adding sources to improve coverage makes the economics worse, not better. You are paying to have a model read "gm" ten thousand times a day.

So the first stage is a cheap, deterministic filter that drops 90–95% of traffic before any model call: length and structure checks, lexical similarity against the niche vocabulary, obvious-junk patterns (stickers, single emoji, greetings, forwarded ads), and cheap author-side signals. No embeddings, no model, no network call.

Two properties matter more than the exact rules:

1. It must be tuned for recall, not precision. The pre-filter's only job is to not throw away real requests. Letting borderline junk through costs a fraction of a cent; dropping a real request costs the whole product, and you will never know it happened.
2. It must be measurable independently. Log what it drops and sample the drops by hand, regularly. A pre-filter that silently starts eating a message format nobody anticipated — a new slang term, a chat that switches to voice-note transcripts — is invisible in every downstream metric. Your lead count goes down and every dashboard says the system is healthy.

The 5–10% that survives is what the model actually reads.

The classifier: the niche is a config, not a prompt

The surviving messages go to a language model with a niche definition: the vocabulary, the explicit statement of what counts as a request in this business, and counter-examples of what looks close but is not.

The important design decision is that this definition is a config file, not code and not a bespoke prompt written per customer. A new market — a new city, a new service, a new language — is a new config: vocabulary, lead definition, source list. Nothing is recompiled and nothing is redeployed.

That constraint is worth defending even when a special case would be faster to handle in code, for two reasons. Commercially, it is the honest answer to "does this work for my business?" — it works if we can write the definition, and we can tell you within a day whether we can. Operationally, it means the thing you tune when quality drops is a text file you can diff, not a branch of logic buried in a classifier.

The model returns a verdict plus a reason. Storing the reason is not decoration: when someone reports a bad lead, the reason line tells you in seconds whether the definition was wrong, the vocabulary was too broad, or the model simply misread the message. Those three failures have three different fixes.

Deduplication, edits, and the boring parts that decide quality

Between "the model said yes" and "a human sees it" there is a layer that is unglamorous and does most of the damage when it is wrong.

Deduplicate on the message, not the text. A unique index on (chat_id, message_id) is the floor. People also cross-post the same request into five chats — that is one person, and delivering them five times makes a short feed feel like spam.

Edits are an exception to deduplication. A very common pattern: someone posts "looking for a villa", then twenty seconds later edits in the budget and the dates. If edits are blocked by your dedup key, you deliver the useless first version and never see the good one. Messages edited within a short window get re-classified, and the result updates the existing verdict instead of creating a second one.

Latency is mostly not the model. The under-a-minute target is spent on the read connection, the queue, the model call, and the delivery — and when it slips, it is usually the queue backing up behind a burst in one large chat, not inference. Measure the stages separately or you will optimize the wrong one.

How we measure it: precision, recall, and the feedback loop

Two numbers, and they trade against each other.

Precision — of the leads delivered, how many were real requests. Our target is 80%. We say that number out loud, including what it implies: roughly one in five will miss. A vendor claiming 99% either has a very narrow niche or is not counting the same way you are.

Recall — of the real requests that appeared in the sources, how many we delivered. Our target is 70%, and it is by far the harder number to measure, because it requires knowing about the requests you missed. The only honest method is manual: take a chat, read a day of it by hand, and compare against what the pipeline delivered. It does not scale, which is exactly why it has to be scheduled rather than left to good intentions.

The feedback loop is the cheap part, and it is the one worth building first. Every delivered lead carries a thumbs-down, and every rejection is a labeled example — the highest-quality training and evaluation data available, produced by the person who knows the niche better than we do. Those rejections drive vocabulary changes, lead-definition edits, and source removals.

That last one matters more than people expect: a source that produces zero confirmed leads for two weeks is not neutral, it is spending capacity for nothing. Sources need retirement criteria as much as they need discovery.

What the pipeline does not do

Three limits, stated first rather than discovered later:

• It does not promise clients, sales, or revenue. It delivers requests. What happens after — whether you answer within the hour, and how you write that first message — is where the deal is actually won or lost, and no classifier touches it.
• It reads only public supergroups — the same thing any of the thousands of other members sees. Closed chats that would not let a person in are not read at all.
• It never writes. The accounts are read-only: no blasts, no auto-replies, nothing posted in anyone's name. What is stored is the request and a link to it, not a person's chat history.

The privacy question is the most common objection in this category, and it deserves the direct answer rather than a paragraph of hedging: a public supergroup is public, being an anonymous reader in one is what every other member is doing, and the correct scope for storage is the request — not the person.

This runs in production

The pipeline above is Sonar: it watches public Telegram chats for your niche and pings you within a minute of someone asking for what you sell. Read-only, 30-day free trial, and a plain-language answer up front on whether your niche has enough volume to be worth it.

If you are building something similar, the compressed version of everything above: put a cheap filter in front of the model, make the niche a config, measure recall by hand even though it hurts, and say your precision number out loud.