Controlling railway crime with passenger counting and facial recognition technology

Posted: 12 March 2020 | , | No comments yet

Passenger counting and facial recognition technology is being tested in South Africa in a bid to combat rising vandalism and theft to the country’s rail assets. Peter Blaine and Pieter Conradie from the University of Stellenbosch give more details about the project, which is soon to enter live field trials.

Controlling railway crime with passenger counting and facial recognition technology

The Passenger Rail Agency of South Africa (PRASA) is mandated by government to provide passenger rail services across South Africa. In recent history, it has been brought under a huge amount of pressure, due to sabotage and arson. Since 2016, over 140 carriages have been destroyed by arson, with only a small number of vandals being prosecuted. To make matters worse, and as a result, it is common practice for carriages to be overloaded with commuters using rail to travel to and from work. This causes damage to train carriages’ structures and suspensions. There is also strong evidence of general vandalism and theft.

PRASA has funded a Research Chair in the Department of Industrial Engineering at the University of Stellenbosch. The Chair, in collaboration with the Department of Mechanical and Mechatronic Engineering, was initially tasked with developing a system which would count the number of passengers entering and leaving a carriage, so that railway staff could control numbers and check tickets. The data collected could be used by the planners at PRASA to optimise train schedules and minimise overloading caused by too few trains. With the increase in crime on rail lines, PRASA requested that the project be expanded to include facial recognition services to be run in conjunction with the South African Police Service (SAPS) to identify, and then apprehend, known criminals.

The project has been running for the past four years using final-year engineering students and has achieved notable success as a pilot project. Both passenger counting and facial recognition have proven to be successful under controlled conditions and are due to be tested in live field trials during 2020.

The initial project concept used door-facing cameras to count faces and, from this, the idea of including facial recognition developed. Faces would be recorded at the ticket office and then checked on entry to the carriage, thereby providing an opportunity to identify freeloaders. The sheer size of the data that needed to be stored soon showed that this concept was impractical. However, it was suggested that facial recognition could be used to compare passengers with a stored database of people wanted by law enforcement. A warning would be transmitted to a central control room and the authorities would be waiting at the next station to arrest the wanted person.

Because of the environment in which the project was developed, the budget was minimal, and the use of freeware and shareware was essential1.

Passenger counting

The use of cameras was fundamental from the beginning. Initial use was made of single lens cameras, but situating the cameras facing the doors proved impractical because of the huge range of light intensity the cameras would have to deal with. It soon became apparent that vertical stereo cameras placed above the door would give the best solution. The problem then became one of identifying individual heads as they entered and exited the doors, and so the development of a suitable algorithm was needed to make the required count.

facial recognition

Depth Map (Neser, 2019)

After much research and experimentation, the Depth Map system, using the Waterfill algorithm developed by Zhang and others2, was used. The Depth Map created from a 3D camera, a landscape with peaks and troughs where simulated rainfall could accumulate, where the algorithm identifies the peaks as representative of a passenger’s head. Thus, it can count multiple heads of different heights simultaneously. The software was further made capable of identifying in which direction the heads were moving. This allowed counts to be made monitoring movements in both directions, resulting in the net flow of passengers. The information was stored locally for onward transmission to a central website.

The software was developed, originally, using MatLab. It was later converted and improved using Python. This was done following a policy decision to use as much freeware as possible. Furthermore, Python could process the data more efficiently and integrate easily with the facial recognition code. The system was tested under poor lighting conditions and correctly counted with an accuracy of 70 per cent.

The process steps were:

  • Initialise the device
  • Generate the Depth Map stream
  • Process Depth Maps
  • Track objects
  • Increase or decrease the passenger count
  • Store the count locally on the train
  • When suitable, transmit the data to a central database.

Facial recognition

Controlling railway crime with passenger counting and facial recognition technology

Facial Recognition (Neser, 2019)

As a result of the first concept, which proposed the use of camera facing the doors, a call was made to add facial recognition. It was quickly apparent that the amount of data storage that would be required would be excessive if every face was to be captured. This would be further complicated by image processing if it was intended to compare facial images with a stored database of sought people. A facial processing algorithm was developed which reduced the size of the data that needed to be compared with similar process data supplied by SAPS. The captured face was reduced to essential biometric data in both cases. The deep learning algorithm software developed was able to identify faces even when they were turned sideways to the camera.

The process steps were:

  • Initialise the device
  • Generate the RGB stream
  • Process RGB frames
  • Detect faces
  • Identify faces
  • Store recognition results locally on the train
  • When suitable, transmit the data to a central database.


It was found that trying to compute and correlate the data captured resulted in delay times, which affected the accuracy of the passenger count. The facial recognition facet required too much of the available resources using a single processor. As a result, the passenger count system was delegated to a stand-alone Raspberry Pi, which allowed the systems to have an accuracy of 90 per cent for facial recognition and more than 75 per cent passenger counting. The other handicap was that the project was severely financially restricted, and the results would be significantly improved with better cameras. For instance, the camera which counted passengers had a minimum focal length of 400mm, while the door height was 2.1m, meaning anyone over 1.7m tall would not be counted.

Ongoing project

The project is not yet complete and will be worked on during 2020 to improve and finalise the system. It is expected that more suitable cameras will be sourced, which should increase the accuracy of passenger counting. The system must still be tested under field conditions.


  1. Zhang, X., Yan, J., Feng, S., Lei, Z., Yi, D., & Li, S.Z. 2012. Water filling: Unsupervised people counting via vertical Kinect sensor, 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS). 18-21 September, Beijing, China.
  2. Neser, J.E.  2019. Development of a passenger counting system incorporating facial recognition for the passenger rail environment, Department of Mechanical and Mechatronic Engineering, University of Stellenbosch.

Peter BlainePeter Blaine graduated with a BScEng in Mechanical Engineering from Natal University in 1970, and a Master of Engineering degree and a Diploma in Advanced Engineering in 1974 from the University of Stellenbosch. He also holds a PMD from the University of Cape Town. Peter is a registered Professional Engineer, Fellow of SAIMechE and a member of ASTM and Life Member ASME. After his BScEng degree, Peter joined the CSIR as a research engineer while studying for his Masters. He left to join the National Accelerator Centre where he was Chief Mechanical Engineer and involved in the design of the magnets for the 200MeV open sector cyclotron. He later joined Sintered Metal Components as production engineer. He then formed his own powder metallurgy company, Powdermet (Pty) Ltd making parts for the international automotive industry responsible for design, quality assurance and IT. He led the company to qualify the quality management system to comply with ISO TS 16949. He left Powdermet after 28 years to join the Pebble Bed Modular Reactor design and research team at the University of Stellenbosch in order to manage various research projects. With the demise of the PBMR project, he continued to work at the university and is presently responsible for supervising final year students with their projects.

Pieter ConradiePieter Conradie was appointed in May 2011 as Senior Research Engineer to assist the Chair with appropriate research in solving PRASA’s problems. He graduated with a BEng in Mechanical Engineering in 1994, Post Graduate Diploma in Industrial Engineering in 1995 and Master of Engineering degree in Engineering Management in 2015, all from the University of Stellenbosch. He is enrolled for his PhD, is a registered Professional Engineer and Mentor with the Engineering Council of South Africa (ECSA), Fellow Member of SAIIE and a member of the Institute of Asset Management. After obtaining his BEng degree, Pieter started at Spoornet as candidate engineer in the Engineering Department. His main tasks included engineering support, project management, and failure investigations. A special achievement during this period was the development of a patented rail maintenance tool, which received several awards. After leaving Spoornet in 2000, he worked as Production Managers for various companies; John Daniel Containers in Mossel Bay for four years, Agri-Phyto in Albertinia for one year; and Vulcan Steel in Cape Town for six years. During these periods, he was responsible for production, maintenance, quality assurance, design modifications, personnel and was often the only qualified engineer providing engineering input. His research interests include Physical Asset Management, Reliability and Maintenance Management. He is the author of four international conference papers, and the author of four peer-reviewed journal papers.

Send this to a friend