Data-driven Spatial Design
Computation Design Lab
Course: MIT 4.570 Learning from Design Heritage - Research Workshop on Data-driven Methods
Instructor: Takehiko Nagakura, Daniel Tsai
Date: Feb.-May., 2022
This class investigates recent technologies that helps studying “design heritage”, spatial designs that surround our lives. Design heritage broadly includes architecture, city and landscape; the built, demolished, and planned; and culturally important as well as the banal ones. We will look at various data- driven methods relevant to learn them, such as photogrammetry, image/video feature detection, machine learning, physiological sensors, natural language processing, augmented and virtual reality, and gamification.
By examining how to collect data, how to process the raw data into forms useful for evaluation, and how to interpret and apply the findings, the students build a foundation for research projects bettering our understanding of the design
heritage around us.
KEYWORDS:
Data-driven Design, VR/AR, Data Collection, Data Processing, Data Interpretation
OBJECTIVES
References
​
- Mobile application: augmented reality on site or in museum for visualization of demolished, incomplete or hidden state of the building
- Construction and interface design of multi-disciplinary database of text, artifact, texture, drawing, photography, and video
- Use of photogrammetry/RGBd tools on-site for digitally capturing built forms, texture, as well as movement of people
- Interactive educational tool that uses game engine and helps museum visitors to learn the history and architecture of the heritage
- 3D panoramic/stereographic narrative of immersive, VR experience in built/un-built spatial environment
- Mining big data for analysis and visualization of the heritage as it exists in the mind of people
01 Using Image Data
Scraping SNS, Online crowd-sourcing Impression of Heritage Places, Mechanical Turk, LAMP pipeline
Images of places are one of the most fundamental data source that informs the spatial designs such as form, material, ethnography and other social/physical/historical context. In this exercise, we will first test a feature detection tool, which is an example of technologies for image analysis. Then speculate opportunities for the research projects through the use of similar image analysis tools. Also, search for reference projects or precedents that use image analysis tools as part of the process of studying architecture, landscape, or urban designs.
Feature Analysis of Photos of Kyoto
Reference
- Google Vision AI AutoML
- Azure Cognitive Services
- Research Paper
Data mining, inference, and predictive analytics for the built environment with images, text,
and WiFi data
​Mapping urban perception : how do we know where we are?
Measuring the immeasurable : an experiment for a machine to map low-level features to
high-level semantic representation of architectural space using a single view photo
02 Using Video Data
Content segmentation, Panoramic projection, HMD
ethnography of heritage places, Pedestrian simulation
Scientists in the Computer Vision domain suggest that we now have been able to recognize not only static objects, but also dynamic events unfolding within short videos such as walking, falling, opening, jumping, and grabbing (Monfort, 2019). Advances in deep convolutional networks have been successful in predicting human interactions with objects, animals, and the environment within video context. In this exercise, we will first test a feature detection tool, which is an example of technologies for video analysis. Then speculate opportunities for the research projects about architecture, landscape, or urban design. Also, search for reference projects or precedents that implement video recognition techniques.
Reference
- Moments in Time Dataset (OpenCV)
- NRC VAD Lexicon
- MoSculp: Interactive Visualization of Shape and Time
- Research Paper
AI Visitor: Tracking and simulating pedestrian trajectories
in Machu Picchu
Machine mediated human perception
Panoramic Video: Recording and Representing the Digital Heritage Experience
03 Representations and UI
Photogrammetric Modeling and Virtual Gallery
Reference
- Research Paper
Digital Heritage Visualizations of the Impossible
Photogrammetric Models of Villa Foscari and Villa Pisani at Bagnolo
Capturing History Bit by Bit
Architectural Database of Photogrammetric Model and Panoramic Video
04 Measuring Human Response to Spatial Designs
Measuring Psychological/Physiological Responses in Spatial design using simulated VR environment - Physiological sensor, eye-tracking
Studying architectural experience has traditionally been difficult because the test needs to be done onsite and controlling/reproducing it is costly and sometimes impossible. A recent alternative is to make a replicated environment in virtual space, and conduct the test by putting the human subjects with HMD in the simulated space. This week, we will first participate in such a test prepared as an example, and then are asked to design a test of our own that uses a simulated environment for studying architectural experience. Architectural spaces evoking the aesthetic emotion of awe have recently received increased interest in the domains of cognitive science and neuroscience. The test example prepared is to study how specific architectural features (sound, light and geometry) can promote the feeling of awe in religious space.
Reference
- Research Paper
A neurocognitive study of the emotional impact of
geometrical criteria of architectural space
Effects of Changes to Architectural Elements on Human
Relaxation-Arousal Responses: Based on VR and EEG
Immersive virtual environments versus physical built environments: A
benchmarking study for building design and user-built
environment explorations
05 Natural Language Processing
Exploring Language and Design via NLP
A. Filtering design-related comments from social media: Messages and images for cultural heritage sites abound. Only some data has interesting comments related to cultural heritage and design issues (comments that might be useful for a designer to improve access to a site, for example).
B. Design in real estate: do people read listings or just look at the pictures? What is the language of real estate listings?
C. General, to be applied to an area of your choosing: Can you detect design language in any text? e.g. descriptions of space and structure, design statements, comments about a space.
Reference
- NLTK Software (Word2Vec)
- OpenGPT-2 : unsupervised, web-text, language model
