This directed study presents a comprehensive overview of Visual Document Understanding (VDU)—a rapidly evolving field within Document AI that enables machines to read, interpret, and reason over visually rich documents such as invoices, forms, receipts, medical records, and academic papers.

The report explains that traditional OCR-based and rule-driven systems are insufficient for real-world documents because they fail to capture layout, structure, and semantic relationships. To address this, modern VDU systems integrate Computer Vision (CV), Natural Language Processing (NLP), and multimodal deep learning, allowing models to jointly reason over text, visual appearance, and spatial layout.

The study traces the evolution of document understanding, starting from early OCR systems, moving through rule-based and machine-learning approaches, and culminating in transformer-based multimodal architectures such as LayoutLM, LayoutLMv3, DocFormer, Donut, and Pix2Struct. It distinguishes between OCR-based and OCR-free pipelines and explains their trade-offs.

The report also surveys core VDU tasks (e.g., key-value extraction, document classification, layout analysis, table understanding, document VQA), benchmark datasets, evaluation metrics, and real-world applications in finance, healthcare, legal systems, education, and retail. Finally, it highlights key challenges—such as multilinguality, handwriting, noisy scans, layout variability, and low-resource data—and outlines future research directions, including large multimodal models and retrieval-augmented document understanding.

Overall, the work positions VDU as the multimodal core of Document AI, essential for transforming unstructured document images into structured, machine-interpretable knowledge.