Pipe dream or reality? Imagine never again as a housing cooperation having to visit a property to measure everything, and check how many windows, frames or doors it has? Together with CQM, Brink has developed an image-recognition app that will sound like music to the ears of housing corporations. Where previously a tape measure had to be repeatedly whipped out, now just a few photos of a house – along with a dash of machine learning and a pinch of deep learning – is enough to find out everything you need to know about a property. Brink is at the heart of the construction, infrastructure and real estate industries. Throw in a bit of maths and you’ll always get our hearts at CQM beating a little faster... Read how, together with Joost van der Werf and Judith van Rijswick from Brink, we made this image-recognition project a success. And the fact that Judith once worked at CQM only made this project all the more fun!
Making smart use of technology
“Brink provides housing corporations with advice, management support and software support around real estate issues,” explains Judith. “The challenge is to maintain a financially healthy portfolio while keeping homes affordable. With tools and sharp insights, there are still many improvements that can be made in the real estate performance of housing corporations. For example, by making smart use of technology and information. Which is why, with the help of CQM, we’ve developed an app that makes it possible, on the basis of photos, to collect data from outside a property that is automated, accurate and always traceable.”
A drone does the work
“Initially,” says Joost. “I thought: Ok, let's just fly a drone round the outside of a house and voilà: we’ll have everything we want and need. Unfortunately, it turned out slightly differently in practice! So CQM's help was really useful. To find out exactly how many windows, frames, doors, etc. a property has, you need a lot of information. Let alone if you also want to know the dimensions of all those items. Before we could do anything with image recognition, we needed to collect a lot of data. And I mean a lot.”
Help from various sources
Judith: “To find out as much as possible about a given home, within Brink we developed two routes. First, the geoscript route. This uses public information. For example from the BAG, BGT and AHN (the Register of Addresses and Buildings, the Register of Topographical Information, and the Dutch Digital Elevation Model). The latter is a kind of database of all elevation heights throughout the Netherlands. Using geoscripts, we map the 3D contours of the house. This tells us, for example, the surface areas of the roof and facade. To know more details about the façade, for example the dimensions of the window frames, you have to work with images. And this is our second route: image recognition.
Step 2 was therefore to collect as much visual material as possible and then combine it with the geoscript. For this, alongside CQM, we brought in Cyclomedia. Cyclomedia currently records 300,000 kilometres of 360° panoramic images across the globe annually. That’s 30 million images per year. Incredibly useful data that can provide invaluable insights.”
The similarities between machine learning and a toddler
“The next step was to train a neural network to recognise housing elements. This is where the knowledge and expertise of CQM was needed. A neural network should help you learn. Deep learning only works when a large number of examples have been observed and actual recognition occurs. So we asked secondary school students to analyse 5,000 photos from Cyclomedia. They accurately indicated on each photo of a house the location of the roof, door, windows, etc. Work so mind-numbing that the kids joked afterwards that, in comparison, they found working in a supermarket for less money good fun.”
“Training a neural network means lots of repetition, like raising a toddler. Only after several repetitions does a child recognize that a car is a car and attribute a word to it. A neural network works the same way. The big difference between a neural network and a toddler,” laughs Joost, “is that a neural network listens”.
“CQM provided the processing power, the software, the machine - everything to ensure the neural network could actually be trained,” explains Judith. “That was quite a job, involving some serious calculations; but we managed it."
Photos do the work instead of tape measures
Judith continues: “In the end, we collected a huge number of elements to feed the neural network: some 20,000 glass items, 10,000 window frames, and so on (see figure 1). In total over 70,000 property elements. As a result, the neural network already recognized a lot. The only downside was that while many photos from Cyclomedia are good quality, some aren’t. Sometimes, for example, a van or large tree stands in the way, so you can’t see half the house. This means not all the data is usable. Moreover, Cyclomedia doesn’t collect photos of the rears of houses, while obviously these, too, need to be measured and maintained.
“For this reason, we’ve developed an app ourselves, whereby a housing corporation can drive to a house to take pictures of the front, back and sides of the house. After that, all they have to do is load the photos up to the app. The geo-process then does its thing and the photo is passed through CQM’s neural network. As a result, you learn all you need to know about a property: how many frames it has, their dimensions, and so on. Without having to measure anything yourself, all the measurements are traceable and accessible: height, width, length, surface area, glazed or not, but also where it is on the facade. So we can even say how long a ladder needs to be in order to paint a given frame..."
Figure 1.
First test results
“Obviously we’re continuing to refine this project,” says Joost, “but we’re already supporting customers. A project with Machine & Deep Learning takes a lot of time and patience. As I said, initially I thought we'd fly a drone round a house and immediately know everything about it: the condition it’s in, quality of the paintwork, and so on. In practice, it’s not like that. So I’m extremely proud of what we’ve achieved. This image-recognition app will save so much time in the future."
He continues: “Collected information will be traceable, imitable and far more accurate. You can have a painter who measures everything manually, and an estate agent who makes a valuation based on experience, maybe very accurately or maybe they’ve slept badly and make lots of errors. Or you can have our miraculous calculator that gets everything exactly right, time after time. The margin of error ultimately needs to become minimal, and we’re convinced that will happen. But already, it saves a lot of time and has delivered a huge quality improvement. And, of course, I still hope my drone dream will one day come true.”
An adventure full of processing power
“The best thing about working with CQM”, says Joost, “is that together we embarked on a kind of adventure and didn't know where we’d end up. It was a difficult project and yet we have achieved something really worthwhile. It’s a question of continual refinement and making best use of each other's talents. There’s the space to do that, for example we’ve run joint brainstorming sessions. With CQM, work really feels like a collaboration that occurs very naturally.”
Publication and/or duplication of texts/images is only permitted following explicit permission from CQM.
Photo credits: Pexels, Pixabay and Brink Group.
If you want to stay up to date with the latest news from CQM, you can follow us on LinkedIn or sign up for our (Dutch) digital newsletter.
