NS (Dutch Railways) prioritizes passenger comfort during train journeys, one key indicator being the likelihood that a passenger can sit during their trip. This 'Seat Availability KPI' is a government-mandated performance metric for NS. When NS wishes to adopt a new model for calculating passenger counts, it must demonstrate that the performance evaluations remain unaffected. CQM has prepared NS for a critical audit.
Over the years, CQM has developed numerous quantitative models across various fields for many clients. Increasingly, we are also asked to review models created by clients or their contractors. NS had been using the ROCKT model for calculating passenger numbers for years, developed internally by their data scientists. With different models created at various organizational levels, NS considered this situation undesirable, and the most-used model, ROCKT, needed replacing.
Calibrations
According to Sven Barmentloo, Business Consultant at NS Network Development & Design, "ROCKT worked well until recently but became outdated for modern needs. It struggled with the latest public transport card developments and could not account for specific travel subscriptions, contactless payments, or fare evasion. We had to rely on field surveys, with researchers scanning passenger tickets on trains. This method allowed us to estimate passenger numbers across all trains." However, the 2020 pandemic halted these surveys, prompting the development of a new model independent of onboard measurements.
FLUX
NS data scientists have developed FLUX, a comprehensive model adaptable to future developments and travel products. The model primarily uses all public transport card transactions, supplemented by ticket sales and conductor scans to enhance accuracy. FLUX can process all travel products and predict incomplete journeys, thus eliminating the need for recalibrations. Sven explains, "Travel is not always straightforward. Some passengers may not check out properly, while others might stop for coffee at the station. FLUX can accurately predict complete journeys based on past travel patterns, even determining preferences for train types. This information allows us to properly estimate passenger numbers and adjust train services accordingly, ensuring we meet the Seat Availability KPI."
Audit
The question arises: if FLUX is so effective, why was CQM needed? According to Sven, "The Seat Availability KPI is a regulated metric with set minimum and target values. It is crucial for NS to report accurately and correctly on this metric. The Ministry of Infrastructure and Water Management needed assurance that the KPI would remain consistent with the use of FLUX. Initially, we planned a comprehensive internal audit with CQM due to their familiarity with NS and their involvement in developing FLUX. However, due to scheduling constraints, CQM proposed a self-executable step-by-step plan that allowed us to prepare thoroughly for the audit."
(text continues below the photo)
Step-by-step plan
NS and CQM held joint sessions to develop this plan. Heleen van Beek, CQM’s project leader, stated, "First, we fully understood how the Seat Availability KPI was calculated to assess the impact of the new model. It was acceptable for some calculations to change as long as the final KPI evaluation remained consistent." In four intensive months, NS and CQM developed a statistical method-based plan, which NS executed just before the ministry's audit. The successful completion of the plan led to ministry approval in early December, and since January 2024, NS has been reporting the KPI based on FLUX—a significant achievement.
New: Model Review
The advantages of FLUX for NS are substantial, according to Sven: "FLUX is future-proof. As we add more travel products and methods, such as mobile check-ins, FLUX can incorporate them seamlessly. It’s an AI model that adapts to changes in travel patterns over time." Given the success of the FLUX project, CQM has launched a new service named Model Review, aimed at evaluating and refining client models. It’s ideal for organizations needing to assess if their model still fits their needs or requires improvements. Interested? Contact CQM, and we can create a tailored plan that perfectly suits your model and organization, just as we enthusiastically did with NS.
Want to know more about Model Reviews?
Would you like to review and possibly improve an existing model as well?
Contact Principal Consultant Lieneke, she would be happy to help!
Additionally, if you want to stay updated with the latest news from CQM, follow us on LinkedIn or subscribe to our (Dutch) digital newsletter.
