Why does an Agent Builder agent give inconsistent data extraction and fail to match names in different formats when processing long PDF documents?
Summary:
An Agent Builder agent (Automation Cloud) processing long PDF files (50–60 pages) alongside input variables may shows below issues:
- Name format mismatch — the agent cannot consistently match an input name variable against the name in the document when formats differ (e.g., "Hanson Michael D." vs. "Michael Hanson", "Jones D Marcia" vs. "Jones Marcia"). It is expected to report whether the input name is present and to extract a related field (e.g., signature date).
- Extraction inconsistency — the same PDF produces different results across runs (e.g., "Date of Service" or member names extracted in one run, missing in the next), even though the data is present. No errors are thrown.
Resolution:
Some Best Practice to consider
- Explicit name normalization in the system prompt: decompose both the input name and the document name into Last/First/Middle, compare Last and First independently (ignore middle initials and ordering), use case-insensitive comparison, and report MATCH / PARTIAL MATCH / NO MATCH with document context.
- Add few-shot examples covering the actual format variations encountered (e.g., "Jones D Marcia" vs. "Jones Marcia", "Hanson Michael D" vs. "Michael Hanson") to reinforce normalization.
- Reduce extraction inconsistency: increase the relevance threshold from 0.3 to 0.5 to reduce retrieval noise, add anchor phrases for target fields (Date of Service, DOS, Service Date, Signature Date), and optionally split extraction into one focused prompt per field on long documents.
- Model selection: choose the model based on both extraction accuracy and processing-time constraints for the document size. Keep the best-performing model as primary under the revised prompt, and when evaluating an alternative model, retest it with a tighter output-format instruction rather than reusing the existing prompt as-is.