Challenge Statement

The Department of National Defence and security intelligence communities are challenged with making sense of ever increasing volume, variety, and velocity of social media data to produce actionable intelligence in support of decision making. We are looking for novel approaches, processes, technologies, and methods to assist intelligence analysts in the analysis of social media to extract relevant information for improved situational awareness and prediction of potential threats.

To improve the intelligence capability, we are particularly interested in developments (with varying levels of automation) related to:

• Content analysis and extraction;
• Data fusion;
• Social science approaches for inferring intent;
• Processing of multiple languages and cultural use of languages (e.g. particular semantics);
• Validation and assessment of credibility (source reliability and inference);
• Display of results (e.g. visual analytics, reporting);
• Data searching, filtering and alignment, and
• Alerting and notification (e.g. cross-cueing)

Background and Context

With the increasing complexity and fluctuating veracity of data in the open source domain, it is not possible to filter, identify and make sense of all information which could be relevant to defence and security. Data is often in multiple formats, multiple languages, unstructured and highly dynamic originating from hundreds of different social media platforms. Analysts are increasingly faced with cognitive overload and fatigue, while trying to make sense of this complex data. Growth in the number of analysts to manually process and interpret the data is not sustainable in the long term. In their aggregation with other similar topics, which are potentially based on different time periods or audiences, interpretation and contextual errors easily occur.

Increasing computing power has improved the ability to identify trends and relevant patterns in huge data sets, which would usually remain hidden, in part due to the recent advances in artificial intelligence with novel methods of text and data mining, including statistical and machine learning techniques.

Outcomes and Considerations

The expected outcome is the development of more effective and efficient processes for content analysis capable of understanding and predicting human behavior based on online activities and communications. Specific outcomes of this effort include:

• Achieving strategic and tactical advantage;
• Process efficiencies for analysts;
• Informing protection of assets, and
• Predictive analytical power.

Concepts and technologies proposed may include, but are not limited to the application of:

• Natural language processing;
• Artificial intelligence, text analytics and pattern recognition;
• Detecting deception;
• Behavioral modelling based on social sciences;
• Sentiment analysis;
• Smart data tagging, and
• Data aggregation and visualization

For this particular call for proposals, we are not interested in data warehousing, information technology associated with hardware for computation, compression techniques for bandwidth management, analysis and extraction of full motion video (FMV), other intelligence gathering mechanisms, or policy considerations which will be assessed at later stages of development.

We are interested in novel capabilities and concepts, but also practical solutions for operators which could be tested, fielded, and implemented quickly in the next few years.