Machine & Deep Learning for housing associations: no more manual measurements
A vision of the future or today’s reality? Imagine a housing association never having to visit a property just to take measurements or count window frames, windows, and doors. Brink, together with CQM, developed an image recognition app that makes this prospect sound like music to the ears! Where a tape measure used to be constantly pulled from a pocket, now just a few photos of a house—plus a dash of Machine Learning and a pinch of Deep Learning—are all it takes to learn everything about the property. Brink is the beating heart of the construction, infrastructure, and real estate sectors. Combine that with mathematics, and our hearts start racing... Read on to discover how we—together with Joost van der Werf and Judith van Rijswick from Brink—turned this image recognition project into a success. And the fact that Judith once worked at CQM makes this project even more special!
Smart use of technology
“Brink provides services including consultancy, management, and software support for real estate challenges faced by housing associations. The challenge lies in managing a financially healthy portfolio while keeping homes affordable. There is still significant room for improvement in the real estate performance of these associations through the use of tools and sharp insights—for instance, by making smart use of technology and data. That is why we—with CQM’s help—developed an app that enables users to collect data on a home’s exterior based on photos: a process that is automated, accurate, and fully traceable,” explains Judith.
Do you have a question for Matthijs?
A drone does the job
Joost adds: “I thought: we’ll just knock this out quickly. We fly a drone around a house and—voilà—we have everything we want and need. Unfortunately, it turns out things work a bit differently in practice. That’s where CQM’s help came in handy. Knowing exactly how many windows, frames, doors, and the like a house has requires a lot of information—let alone knowing the dimensions of those elements. Before we could even do anything with image recognition, we had to collect a vast amount of data. And I mean a *lot*.”
Help from all quarters
Judith: “To gather as much information as possible about a property, we developed two approaches at Brink. One is the ‘geoscript’ route. This relies on public data—such as the BAG, BGT, and AHN. The latter is essentially a point cloud dataset covering all elevation levels in the Netherlands. Using geoscripts, we map out the property’s 3D contours, allowing us to determine metrics like roof and façade surface areas. However, if you want more detailed information about the building’s appearance—such as window frame dimensions—you need to work with imagery. That brings us to our second route: image recognition. Consequently, Step 2 involved gathering as much imagery as possible to combine with the geoscript data. To achieve this, we together with CQM enlisted the help of Cyclomedia. On an annual basis, this organization currently captures 300,000 kilometers of 360° panoramic imagery worldwide. That amounts to 30 million images per year—in other words, usable data for generating valuable insights!
The similarity between machine learning and a toddler
“The next step was training a neural network with the aim of recognizing elements of houses. This is where CQM’s expertise came in handy. You really have to help a neural network learn. Deep learning only works once it has encountered many situations and actual recognition takes place. That’s why we asked students to analyze 5,000 photos from Cyclomedia. They meticulously marked the location of the roof, door, window, and so on, on every photo of a house. It was incredibly tedious work; afterwards, even the students said, ‘Working at a supermarket for less money actually sounds pretty fun, haha.’”
“Training a neural network involves a lot of repetition much like raising a toddler. A child only recognizes a car as a car after seeing one several times, and only then attaches a word to it. A neural network works the same way. The big difference between a neural network and a toddler is that the former actually listens,” Joost says with a laugh.
Judith adds: “CQM provided the computing power, the software, the hardware everything needed to ensure the neural network could actually be trained. It’s a massive undertaking involving heavy-duty computation. But we pulled it off.”
Photos do the work instead of a tape measure.
Judith: “Ultimately, we gathered a vast number of elements to feed into the neural network: around 20,000 glass panes, 10,000 window frames, and so on (see the table opposite)—totaling more than 70,000 housing elements. This enabled the neural network to recognize a great deal. The only downside is that the photos collected by Cyclomedia vary in quality. Sometimes, for instance, a van or a large tree blocks the view, obscuring half the house. This means not all the data is usable. Moreover, Cyclomedia doesn’t collect photos of the rear of the house—even though the back, of course, also needs to be measured and maintained.
That’s why we developed our own app that allows housing associations to drive up to a property and take photos of the front, back, and sides. That’s all there is to it. They then upload the photo into the app, which handles the rest. A geoprocess is applied, and the photo is processed by CQM’s neural network. You then learn everything about the property: how many window frames it has, their dimensions, and so on. Without having to take any manual measurements, all the data is traceable and available—height, width, length, surface area, whether or not there is glass, and even the position on the façade. We can even specify how tall a ladder needs to be for painting...”
The initial test results
Joost: “Naturally, we are continuing to develop this project, but we are already supplying customers as well. A project involving machine learning and deep learning requires a great deal of time and patience. As I mentioned earlier, at the start of the process I imagined we would simply fly a drone around a house and instantly know everything—the condition of the house, the quality of the paintwork, and so on. In practice, things turned out a bit differently. Still, I am incredibly proud of what we have achieved. This image recognition app is going to save so much time in the future.
The information collected becomes traceable, verifiable, and far more accurate. Currently, you might have a painter measuring everything by hand and a real estate agent making an estimate based on experience—one person might be a stickler for detail, while another might have slept poorly and makes frequent calculation errors. Then there’s our ‘math whiz’ of a system that knows the exact figures—and delivers consistent results every time. Ultimately, we aim to minimize the margin of error, and we are confident we will succeed. For now, it saves a lot of time and represents a huge leap forward in quality. And, of course, I hope my ‘drone dream’ will one day become a reality.”
An adventure powered by computing power
“The best thing about working with CQM is that we embarked on a kind of adventure together without knowing exactly where it would lead. It was a challenging project, yet we created something truly impressive. It involves constant fine-tuning and leveraging each other’s talents. That scope for collaboration exists, and we made the most of it during our joint brainstorming sessions. Working with CQM really feels like a partnership; it happens quite naturally,” says Joost.
Making texts/images public and/or reproducing them is permitted only with the express consent of CQM.
Photo credits: Pexels, Pixabay, and Brink Groep.
Want to apply Machine Learning and/or Deep Learning in your organization as well?
Then please contact Matthijs.
If you would also like to stay up to date with the latest news from CQM, follow us on LinkedIn or sign up for our digital newsletter, Quant.