Interpreting a hand-drawn floor plan requires combining computer vision, symbolic reasoning, and real-world motion planning so a robot can move reliably in human environments. Human experts in computer vision and robotics demonstrate the core components that enable this capability, building on work by Fei-Fei Li at Stanford University in visual recognition and by Dieter Fox at University of Washington in probabilistic robotics and localization.
Perception and symbolic extraction
First the robot converts the sketch into a machine-readable map. This uses image preprocessing to enhance lines and remove artifacts, followed by semantic segmentation with convolutional neural networks trained on architectural symbols to label walls, doors, windows, and room types. Visual recognition research led by Fei-Fei Li Stanford University and scene understanding work by Jitendra Malik University of California Berkeley provide foundational methods for identifying ambiguous strokes and contextual cues. Next comes symbol recognition and vectorization that turns freehand marks into geometric primitives and a connectivity graph that represents passable space and obstacles. Scale and intent are often missing from sketches, so inference from common object sizes and textual labels helps estimate distances.
From map to motion
Once a topological and metric representation is available, classic robotics methods convert that map into actions. Simultaneous Localization and Mapping algorithms allow a robot to align its sensor observations with the inferred map, a field advanced by Dieter Fox University of Washington and Sebastian Thrun Stanford University. The robot builds an occupancy grid from the vector map and uses path planning algorithms such as A star search for route selection, coupled with local reactive control to avoid unforeseen obstacles. Continuous re-planning and sensor fusion from lidar or cameras keep navigation robust in changing conditions.
Relevance, causes and consequences
Accurate interpretation of hand-drawn maps expands access where digital blueprints are absent, aiding search and rescue in disaster zones and enabling assistive robots in older homes with no CAD records. Causes of failure include ambiguous symbols, inconsistent scales, and culturally specific drawing conventions that vary by region and socioeconomic context. Consequences of misinterpretation range from inefficient task performance to safety hazards, highlighting the need for human-in-the-loop verification and culturally aware training data. Environmental factors such as cluttered interiors and degraded infrastructure further complicate reliable deployment. Integrating trustworthy vision and robotics research with community-centered design improves both performance and social acceptability.