Big data and the built environment: challenges and applications

Handling the data flood

22 September 2017

Dr Thomas Whiffen assesses the challenges and increasing applications of big data for the built environment

The human eye is a complex visual sensor, connected to a data processor capable of making split-second decisions. Societally, we need similarly efficient ways to harness the rising tides of built environment big data.

“Big data” refers to extremely large data streams that require advanced computational analysis to reveal patterns, trends and associations, particularly relating to human behaviour and interactions. Innovation in big data and the “internet of things” (IoT) – the internet capability and connectivity of everyday objects – is now commonplace. This has led to a growing trend among tech-giants to move investment into home hardware, laying the foundations for the connected home.

Smart homes

There has never been so much information generated about our buildings and their occupants. This growing surge of data – brought about by increased connectivity – suggests that the smart home is no longer a distant dream but a modern-day reality.

The flow of building condition data is also increasing exponentially. It is generated and exploited by installed sensors, smart thermostats, smart meters, IoT devices such as TVs and fridges, and hand-held devices including phones and tablets and then available to researchers, system and service providers.

Out of context, this flood of localised, personalised data provides little more than a swamped server and terabytes of unintelligible alphanumeric garbage. However, a rich data resource can be produced by sifting and then layering these data feeds with contextual information, such as building typology and occupant behaviour.

There are further benefits when historic open national data sets are added, and relationships refined through machine learning and optimised statistical analysis. The National Energy Foundation’s integrated approach to asset information performance is a good example. It has coupled these layered data sets with organisational objectives and key performance indicators to produce strategic investment plans for portfolio managers.

At a domestic level, big data has the potential to improve the comfort and control we have in our homes. Using historic data, cloud analytics, and a plethora of localised sensors and IoT nodes, smart homes create personalised environments. They may also change behaviour to drive down energy bills, improve indoor air quality and reduce carbon emissions.

Domestic solar generation meter


Domestic solar generation meter in situ, with attached in-house monitor sensor


Big data in the built environment has definite benefits, but before these can be fully realised, widespread social engagement is required through an increase in the following qualities.

  • Awareness: surveyors and decision-makers need to be aware of how big data could benefit their environments.
  • Trust: systems must be trusted by the building occupants, so that users understand what to do to achieve the conditions they desire.
  • Simplification: any interaction between humans and machines must be simple, intrinsically obvious and satisfying.
  • Empowerment: systems must offer a sense of user empowerment. Too much automated control too soon will undermine users’ trust and curtail their engagement, prompting them to resort to more familiar control methods.
  • Security: trust and engagement must be supported by watertight cyber security, with appropriate legal and privacy policies in place to provide a safe framework for progress.

Looking ahead

The ever-rising seas of big data in the built environment provide the potential for a wealth of benefits. For everyone involved, there is a need to understand both the promise and ramifications of big data, and then work hard to bring about the benefits, prioritising wellbeing.

There have already been legal cases in which building data has been used to resolve claims of poor performance in new-builds. With this in mind, residents and occupants should be encouraged to monitor their energy consumption and internal environment – to hold developers to account and highlight the reality of the building performance gap.

Dr Thomas Whiffen is a Senior Energy Specialist at the National Energy Foundation

Further information