The LLM talks, the optimizer decides
A WhatsApp bot for receivables advance: the AI runs the conversation and a pruned beam search picks the invoices.
Summary
Context
Advancing receivables means selecting which invoices to advance, which is a combinatorial choice. Doing that inside a WhatsApp conversation calls for a near-real-time response.
Action
I split the responsibilities. The LLM (Vertex AI) runs the conversation; invoice selection belongs to a deterministic beam search algorithm with pruning, which keeps response time acceptable. The API receives messages and documents via webhook, runs a daily job, and has a simulation mode that works without credentials.
Result
The prototype was complete and working in simulation mode, but it never ran with customers.
Lesson
Do not ask the LLM for what an algorithm does better: use the model for language and code for the decision.
Architecture
Long version
How it works
- The message arrives through the WhatsApp webhook (Z-API).
- The LLM runs the conversation with the customer.
- When the customer asks for an advance, the beam search builds combinations of invoices, keeps only the best ones at each step and prunes the rest. That way the answer arrives at conversation speed.
- The result goes back to the customer in the conversation.
The project started with security in mind: no versioned credentials, all configuration through the environment, and a simulation mode for developing without touching the real WhatsApp.
Lessons learned
- Division of labor. The LLM interprets and converses; the invoice choice is deterministic and testable.
- Search with a budget. Beam search with pruning trades a guaranteed optimum for a good answer within conversation time.