Challenge Statement

The Department of National Defence (DND) is looking for solutions that will assist analysts in monitoring and interpreting the high volume of Full Motion Video (FMV) feeds. FMV analysis supports the detection, identification and tracking of events, people and objects of interest.

Background and Context

The Canadian Armed Forces (CAF) has acquired new airborne intelligence, surveillance and reconnaissance (ISR) platforms while preparing for the next generation multi-mission aircraft (CP-140 Aurora maritime patrol aircraft replacement). These platforms will enhance the capacity of CAF to provide critical, near real-time, surveillance and operational support to military organizations. These airborne platforms will be equipped with state-of-the-art sensor suites to enable the collection of still images and FMV feeds. While the ability to operate these platforms has advanced to a high level of sophistication, the process of monitoring and interpreting video feeds continues to make a significant demand on the operational community. The CAF needs access to new tools to assist the analysts in monitoring multiple video feeds, identifying activities of interest, and alerting the operators.

With access to such tools, it will be possible for an operator to simultaneously monitor multiple feeds, thereby achieving enhanced efficiency and reduced workload. The task of monitoring video feeds is well-suited to emerging technologies in order to identify items of interest within the field of view against a moving background and in the presence of other activities. Beyond recognizing items of interest, a further challenge is to determine if the system is observing an activity which could be of interest to the operator. This includes basic counting of objects of interest, analysing patterns and, eventually, identifying indicators of anomalous activity.

Outcomes and Considerations

The desired outcome is the development of tools for automatic monitoring and interpretation of multiple live/near-real time image/video feeds or FMV feeds, collected with various camera technologies (e.g. high resolution, electro-optics, infra-red and Synthetic Aperture Radar). One should note that these sensors often image areas for which limited a-priori information is available.

The proposed solution should address one or more of the following:

• Generate meta-data to assist in the rapid retrieval of images or video clips;
• Automatically recognize and track specific objects within the field of view (for example, buildings, vehicles, people, etc.);
• Track mobile objects as they travel through the field of view;
• Automatically compare patterns from the same area at different times, potentially using data from different sensor platforms and develop a model of normal patterns of life in a region;
• Detect anomalous behavior of objects (for example, vehicles travelling against normal traffic flow) with a reliable degree of confidence, thus alerting the operators to items of interest.