u/Playful_Air_7174

Trying to figure this out and could use some pointers.

I'm feeding sales call transcripts into Gemini and asking it to pull out patterns that correlate with whether the rep booked a meeting. What I get back is stuff like "asks follow-up questions" or "uses social proof". Technically correct but useless because every rep does these to some degree.

What I actually want is patterns like "asks about urgency right after a price objection" or "names a competitor only after the lead mentions budget". Specific moves in specific spots. The LLM seems to default to category labels even when I ask for verbatim quotes and context.

Two things I think are going on:

The model groups things during extraction. Even when I tell it to keep the exact phrasing it still slaps a generic label on top, and when I aggregate across calls the specifics get lost behind the label.

I don't think my prompting is forcing the specificity hard enough. Saying "be specific" doesn't really work. I've tried giving examples of good vs bad outputs and it helps a little but not enough.

Things I'm thinking about trying:

Skip the LLM label entirely. Just keep the verbatim quote plus some context (what phase of the call, what came right before). Then embed all the quotes and cluster them, and let the clusters be the patterns instead of the LLM-assigned labels.

Two-pass extraction. First pass pulls candidate quotes. Second pass takes a batch of similar quotes and writes a tight description of what they have in common.

Use a stronger model just for the labeling step and see if the specificity changes.

Has anyone done something like this? Particularly interested if you've found a prompt pattern that reliably gets phrase-level output and not category-level. Also curious if there's a name for this problem in the literature, feels like it should have been studied but I haven't found the right keywords.

I'm trying to extract useful behavioral patterns from sales call transcripts and I'm stuck on the abstraction level. Hoping someone here has thought about this.

Setup: Danish-language sales calls, around 5 min each, transcribed and speaker-labeled. About 15k calls a month from a team of 15 reps. Binary outcome per call: did the rep book a meeting or not. I want to figure out which conversational moves actually work, so the manager can coach the team on real stuff instead of vibes.

Right now I run transcripts through Gemini Flash and ask it to pull out behavioral patterns with verbatim quotes. Then I aggregate across calls and check if a pattern shows up more often in booked calls vs lost ones. Threshold to call something validated is n>=20, lift >=3pp booking rate, p<0.05.

Problem is the patterns that come out are too generic to actually use. Stuff like "asks follow-up questions" or "mentions price". Technically true, useless as coaching. What the manager actually needs is something like "asks about urgency right after a price objection", a specific move in a specific spot.

I think there are a few things going wrong but I'm not sure which one to fix first:

The LLM produces category-level labels because that's what it's trained to do. Even when I ask for verbatim quotes it still ends up grouping them under a generic label, and the aggregation step throws away the specifics.

The sample size is small once you slice by phase and behavior. 20 to 50 observations per candidate. P-values at that size with no multiple comparisons correction probably means I'm just catching noise.

I'm treating it as a hypothesis test when it should probably be a ranking problem. I don't actually need "this is statistically true". I need "this move is more likely to precede a good outcome than this other move".

Stuff I've considered: tightening the prompt to demand phrase-level output with context (helps a bit, doesn't fix aggregation). Clustering phrase embeddings before aggregating instead of using the LLM label as the unit. Comparing top vs bottom performers within the same team directly instead of trying to make population-level claims. Reframing the whole thing as next-move prediction conditioned on call state.

What I'd love input on: has anyone done conversational success prediction at this kind of low-n where you want phrase-level moves and not category labels? Any prompting tricks for forcing the LLM to keep specifics through aggregation? Any pointers to the dialog acts literature that's actually useful for this vs theoretical?

Happy to share examples if it helps.

Hey venner!

Jeg er netop sprunget ud som iværksætter😀

Min vision er at løse det psykologiske og retoriske møde i salg.

Det gør jeg ved at datasegmentere, forudsige adfærds- og salgsmønstre på tværs af kunder og sælgere samt skabe læringsmateriale og aktiv feedback ud fra det.

Jeg kommer til at lave LinkedIn-content to gange om ugen hver tirsdag og torsdag (startede i sidste uge), så hvis i har lyst til at linke op, følge med, sparre osv., så får i den her:)

https://www.linkedin.com/in/thomas-axelsen-211500263?utm_source=share_via&utm_content=profile&utm_medium=member_ios