Creating 3D structural frames from 2D drawings — Using data-analysis techniques and Reinforcement Learning

We propose an algorithm that (1) extracts information from 2D CAD drawings, (2) learns patterns from the information and set of design policies in constant interactions with human experts (3) automatically generates feasible new design and 3D models and objects. The outputs are in DWGs or other formats that are supported by Autodesk products.

Unlike most recent data-extraction methods, our algorithm neither employs OCR to extract and analyze data from 2D drawings nor uses Deep-Learning approaches to reconstruct 3D models. The algorithm is originally a data-analytic approach that is equipped with a Reinforcement Learning architecture. The algorithm has been developed based on analytical geometry and conventional techniques in Reinforcement Learning.

The algorithm detects structural components or other desired objects through pattern analysis of geometries in the drawings. The algorithm learns the relationship between structural components as well as constraints being trained by human experts. The algorithm’s accuracy for data extraction and pattern recognition is extremely high; no matter how complex the drawings are, while the performance of the Reinforcement Learning module improves as it is being used.

We have successfully applied our algorithm for two business cases (Unfortunately I am unable to share the clients’ details)

General Business Challenges:

1. Can we extract desired structural components from complex engineering drawings? (e.g. objects and their locations such as “columns”, beams, rafters, girders, “doors”)
2. Can we reconstruct a 3D feature of structural objects (columns, beams, girders, and rafters) from information in 2D drawings?
3. Can we develop a smart structural frame designer (an AI agent) to locate columns, beams, girders, and rafters into a given architectural design plot?

Ideation
Imagine we were given a bunch of drawings such as the one below. We employed a Meta-Data approach to solving the first two challenges above. For the last one, we carried out experiments by using Reinforcement Learning. I briefly explain the approach below.

To meet the first two business challenges above, we extracted information of walls, windows, doors, and a location of a random column from architectural plots. The algorithm recognized the existing network shown in yellow in the image below, whose nodes are where the columns are most likely located. Based on architectural plots and the knowledge of columns-distance, the algorithm learned how to extract the size of columns and how to place them on the most likely relevant nodes of the network where satisfied the structural engineering technical specifications as well as architectural design constraints.

The algorithm extracted information of the structural components plus geometrical features from associated 2D drawings of one project included Plot-Views, Cross-Section-Views, Connection-Views. The algorithm recognized the logical interconnections and patterns between 2D data. It incorporated some learned structural-analysis constraints to build 3D images.

Here are some examples for the algorithm’s outputs (the suggested locations for placing columns)

The 3D images are saved in any format supported by Autodesk.

Here are some short demos.