A Microsoft Teams agent uses OCR to turn handwritten accident reports into structured claims, completed in under 10 minutes.
A public transport operator manages several hundred buses. When a bus is involved in an incident, the driver and their manager must open a claim file by completing a bus-specific accident statement (if a third party is involved) or an incident report (for incidents with no third party). These standardized paper documents require a lot of manual input and manual review of extracted information.
Every year, hundreds of reports are processed; the accumulation of documents and back-and-forth between drivers and claims teams increases the time needed to open files.
Digitize the first incident notification while preserving the richness of the paper forms. Drivers must be able to report a claim from the field, validate the bus license plate, submit the correct form type, and provide all required information.
A key challenge is ensuring reliable OCR of handwritten documents: statements and reports include free-text areas where drivers and involved parties describe the incident. The agent must extract these data and validate them with the user.
Deployment of a conversational agent available via Microsoft Teams for operations managers. The process runs in three steps.
The user enters the bus license plate; the agent checks it against the fleet database and retrieves the route, subsidiary, and driver identifier, then asks whether a third party is involved to select the right form.
A vision/OCR model extracts form fields — date, time, location, injuries, witnesses, vehicle and driver details. When OCR is uncertain, the agent asks questions to confirm or correct the value.
The agent fills missing fields and generates a structured report. A summary email is sent to the claims department, and data is injected directly into the claims system.
Used when a third party is involved — the standard bus-specific accident statement form.
Used when no third party is involved — a simpler, detailed incident report.
The driver or manager opens the bot in Microsoft Teams and enters the bus license plate. The agent verifies the plate and automatically retrieves fleet data — route, depot, driver. The bot then guides the user to upload the right document.
The vision model extracts information from the form: date, time, location, description, driver contact details, license plates, insurers, accident sketch, signatures. The agent also collects additional information not present on the form, such as injuries, witnesses, or weather conditions.
The data template developed with the client defines around fifty mandatory or optional fields.
Each field is labeled mandatory or optional and maps directly to the claims system fields. The agent checks consistency — such as date after the reporting day, or correct license-plate format — and requests corrections when needed.
At the end of the journey, the agent sends a summary email to the claims teams, including the main fields and the attached statement or report. A confirmation message is also sent to the user via Teams — structured data is automatically injected into the processing system, with no re-keying.
Early feedback shows that drivers and managers appreciate the simplicity of the process: the bot handles and validates documents, while claims teams no longer have to retype information. The main challenge remains recognition of handwritten areas and ongoing tuning of vision models; the agent relies on human validation to correct uncertainties.
With this foundation, the client plans to extend the solution to other services, such as light service vehicles, and integrate additional channels such as WhatsApp or email to reach drivers in the field.
This use case demonstrates that automating incident reporting for a bus fleet can drastically reduce turnaround times, improve file quality, and provide actionable data to analyze accident hotspots. The combination of OCR on standardized documents, interactive user validation, and direct integration into the claims system is a powerful lever to industrialize claims handling.