Strokes2Surface: Recovering Curve Networks From 4D Architectural Design Sketches

We present Strokes2Surface, an offline geometry-reconstruction pipeline built upon a 4D Sketching Interface, MR.Sketch, targeted at architectural design. The pipeline recovers a curve network from designer-drawn strokes, thus bridging between concept design and digital modeling stages in architectural design. The input to our pipeline consists of 3D strokes' polyline vertices and their corresponding timestamps (as of the fourth dimension), along with additional geometric and stylus-related recorded properties. Inspired by sketch consolidation and sketch-based modeling methods, our pipeline leverages such data and combines three Machine Learning (ML) models; a classifier and two clustering models. In particular, based on observations of practices designers typically employ in architectural design sketches, we solve a binary classification problem to recognize whether a stroke depicts a boundary and edge or is used to fill in the enclosing areas and faces of the intended architectural object. Followed by the two clustering models, strokes of each type are further parsed into groups, each representing either a single edge or a single face. Next, groups representing edges are approximated with B-spline curves, followed by a topology-recovering process identifying and fixing desired connectivities between the curves forming a well-connected curve network. Next, groups representing the faces are employed to detect the cycles bounding patches in the curve network, resulting in the final surface mesh geometry of the architectural object. We confirm the usability of Strokes2Surface via a user study and further validate and compare our results against a range of reconstructions computed using alternative methods. We also introduce our manually labeled dataset of 4D architectural design sketches for further use in the community.