Every year thousands of lives are lost across the globe in emergency situations, a large chunk
of which is lost due to stampede as a consequence of the panic in the crowd. The emergency
situation does not last long, but its repercussions stay with us forever. An emergency situation
becomes difficult to handle because the operational forces lack real time data of the scene to
take appropriate decisions. Before the rescue team can render any help, people run around
anxiously to save their lives and they end up doing just the opposite of harming and
sometimes killing people. There is a pressing need to address this problem as a social issue
and save as many lives possible.
In the last ten years a lot of surveillance cameras have been installed in areas of huge crowd
movement for safety of people. It is almost impossible to track the images of all the installed
cameras and it would be so much simpler if we could use the artificial intelligence of
computers to detect any unusual activity by monitoring the crowd density in a place. With
image recognition and crowd behaviour analysis in crowded areas the operations teams
always have access to the congestion estimation.
Crowd Density Monitoring using Crowd steering
Each surveillance camera has a fixed area that it can view at a time and a fixed angle at which
it can rotate thereby increasing the total area it can analyse. The technology of image
recognition counts the number of faces in the frame and gives us the density. Entire landscape
to be scanned is divided into a grid and each CCTV camera gives its response to the gird
following a colour code to highlight any abnormal activity. The clusters in the grid need not
necessarily be disjoint for reporting, there may be overlapping portions amongst two or three
camera recordings.
In an ideal situation, as
shown in figure, the
impact of any
abnormality reaches
beyond the originator.
In a situation of a
person falling to the
ground the immediate
reaction of the crowd is
to stop and build a
crowd around the
injured person which builds which slowly builds crowd and hinder the free movement of
people. We plan to use Artificial Intelligence here that will check the crowd density from the
2
2
neighbouring CCTV cameras to evaluate where to divert people. The mode of
communication of the system with people will be via audio and video cues on the TVs and
speakers installed in the area. This is based on the understanding that in times of frenzy
people do not stop to check to access the situation but if a computer does the same for them
and guides them via audio they are likely to follow the instructions.
A large amount of data will be collected from the crowd this data of time based crowd
density is fed as data to pattern recognition, Machine Learning can also assist us in giving an
admonition of when and where crowd can go out of control. We can prepare a report on
crowd behaviour based on the trend analysis for and share it with psychologists to know of
concrete plans to steer crowd.
In difficult times of a disaster, the rescue teams can provide a smart and reliable support crisis
management. Using the heat map and density grid they can know at a glance where crowd
density levels are increasing and police and can dispatched to the scene without any delay. It
is quite possible that by the time police reaches the site the crowd has already been dispersed
by the suggested solutions of AI and ML.
Handling a social issue always involves multiple stakeholders and this system adds values to
the functioning of each one those stakeholders.
‘ Decreased time in collecting and analysing data
‘ Almost negligible human effort involved in assessing emergency data to come up
with contingency plans. As no humans are involved the entire process takes place in
milliseconds
‘ Reduced scope of error with introduction of reliable and fast systems
‘ Efficient utilisation and management of the space to distribute crowd
‘ All the rescue contact persons are informed to be alert as an when the crowd density
increases
How it adds value to the precious lives
‘ Civilians feel safer and valuable with computers constantly monitoring their activity
‘ The system never obstructs the activity of any of the stakeholders, it only suggests
solutions from videos and images
‘ Rescue team can now be better prepared with a plan and equipment to save
‘ Government and public authorities who are answerable to the people for their safety,
now feel confident that they have a system constantly helping people
‘ Data collected is only used for crowd behaviour analysis
Architecture of our Visual Crowd Simulation technology
Conventional instances of mass gatherings like the Kumbh Mela in Allahabad, the Hajj at
Mecca have been managed successfully using crowd steering technologies. The Kumbh mela
crowd of 2015 made it into the news for having been steered and managed effectively, having
3
3
used the cell tower pings from the all-pervasive technology of today- the smartphone. In this
project however, our focus would be more on a technological solution which would assess
crowd dynamics via Crowd counting through visual analytics & machine vision.
Below is shown an image of process and information flow:
The architecture of our visual crowd simulation & steering models would have Video
surveillance cameras (both fixed and moving field of vision) coupled to a central Systems
Manager, which would enable operators to adjust and control the visual feed from different
areas, and also control which data to receive and analyse for potential situations. Video feed
from the cameras would be passed on to an Acquisition module, which would essentially
convert standard definition raw video feed into Grayscale low resolution (320×160) for
analysis and comparison with results provided by crowd simulation algorithms running in
parallel in the backend.
In view of a low-cost solution requiring less processing power, we propose the use of a crowd
counting algorithm to process the visual feed on a quasi-real-time basis. Discrete measurements on
frames provided on a 30 second basis is good enough for assessing crowd build-up and dynamics in
most environments of India.
Analysis of a crowd starts with an estimate of crowd numbers. As of today, this has multiple goals,
some of which maybe detection, tracking or analysing individuals for security or resource
management. Some technologies rely on explicit tracking of individuals (to the level of complete
facial identification) to do so, however in this case estimates of crowd level would be done on a global
level. The property being exploited here would be that a crowd can often be modelled holistically- i.e.
Crowd behaviour as a whole which can be checked for abnormal deviations if any (as a whole or on
Analyse Information Visualise Acquire Information Information
4
4
the basis of few individuals). Use of Dynamic Textures, a method of abstracting complex
global level structures (such as scenes involving vegetation) into a simple spatio-temporal
model can be fitted to our purpose. Analysis of Video Pixels, without investing processing
power in individual facial tracking enables a holistic modelling of the prevailing crowd; with
standard algorithmic procedures running in the background specifying distances between
dynamic textures for specific crowd behaviours.
Having modelled the crowd into a dynamic texture based software model, a crowd count can
be achieved from ‘Global Low-Level Features’, whereby the crowd is segmented into
specific areas of likewise clusters, such as groups of people moving in the same direction, or
inanimate objects in the surroundings and mapping them into a ‘Crowd Count per area’
estimate. Specialised statistical methods such as Gaussian Process Regression can directly
map such features into counts. Methods such as these have been shown to be robust and
accurate even in cases of inhomogeneous and large crowds.
Role of Machine Learning:
To prevent unmanageable situations involving crowds in busy areas, crowd counting by itself
is of no use unless and until reactive control systems assessing crowd behaviour is present. In
order to do so, the role of Machine Learning becomes indispensable, as ‘Anomaly Detection’
stochastic models for automatic crowd management are handled solely by machine learning
based programs. Here we propose use of Machine learning algorithms to observe real
populations and fit day to day populations into it for checking normal behaviour scenarios. A
Shadow based simulation which at first in its learning phase creates a ‘Shadow’ agents for
crowd behaviour as a whole, and then calculate divergence in between the two in order to
flag anomalous behaviour can be used to indicate signs of an impending chaotic situation.
Fig.: The algorithm at work
At each time ‘t’, the shadow agent a (representative of the crowd) has two positions in real
time, a real world tracked position pa & a simulated position ma . Whether the difference in
these positions amount to an anomaly or not is specified in some difference kernel K.
Machine learning algorithms running iteratively can over time get better at detecting
anomalies and indicate potential break-outs of stampedes or mobs.
Role of Artificial Intelligence:
Real time assessment of abnormalities in crowd behaviour can be further processed forward
to actuate lifesaving crowd control measures powered by artificial intelligence. Upon
5
5
indication of impending chaotic situations, Artificial intelligence can immediately route
people away from congestion areas by way of the following measures:
‘ In case of congestions in confined spaces (example: Two-way passages or
overbridges in Subway or railway stations), AI can trigger events such as automated
announcements to divert people to other routes, or shift oncoming trains to other
platforms to reduce influx of more passengers into populated areas. [Route mapping
may require advanced route seeking techniques such as Djikstra’s Algorithm]
‘ In case of Fire or other emergencies, Crowd anomalous behaviour can be used to
trigger Audio & Visual cues (such as Emergency directional lights and
announcements) in order to streamline flow and avoid progressive crowd collapse.
6
6
Implementation
The implementation and deployment process will be as follows:
To generate a system as discussed above we will first start with the requirement gathering. A
starting point can be an existing system which we can modify to suit our need. We will
integrate it with constraint propagation technique to ensure that there is no gap in the
requirements and code. The benefit of this technique would make our code more reusable and
deployable.
Requirement Specification: Our own analysis of the situation is not enough to start making
the solution. We need to gain more insights from people who have been caught, injured in a
stampede. We can also collect responses from people who have witnessed such situations
from a distance and they have felt helpless that they couldn’t assist ailing people. This will
help us identify the key issues or reasons that lead to a chaotic crowd.
Design: From the insights gained we can evaluate which of our solutions would or would not
work in resolving the problem. For AI and ML to work together we will have to allow some
time for the system to see analyse and then generate its suggestive solution. For this purpose
we must use Artificial Intelligence with memory of its own so that it can retain and process
the data.
The system will be platform independent and object oriented to reduce requirement to code
cycle time.
Error Detection: In case of AI and ML error detection can be in two stages. The first one
can be in code testing due to inconsistent data or incorrect logic. In this case the expected
result does not match the actual result. The other form of error detection can be in the form of
the system being unable to give a result. This is common when the system is taking too long
to filter the data or it is running out of memory to process the request. In either case an
immediate rectification is needed to bring the system up and working.