Big Data in Defence and Security :From the User's Perspective
As the hype continues around Big Data, its ambiguous nature continues to obscure the potential applications of the technology. Clarity is beginning to emerge within the A&D community about how Big Data can be leveraged for 'intelligence amplification' and companies are beginning to respond. This research aims to define these emerging applications and how intelligence amplification can be achieved in a defense and security environment. The research service has considered Open Source Intelligence, Activity-based Intelligence, and Deep Learning.
An Indefinite Definition: What is Big Data?
Defining Big Data is problematic. While faster, more, and cheaper are readily tangible concepts, understanding precisely how this data is leveraged to meet operational requirements can remain vague. This is no accident. The term is not employed to solely describe technologies such as remote sensing or distributed file systems; nor is its application exclusive to either the retail sector or law enforcement. Rather, Big Data encompasses a diverse architecture of technologies that can be used in unison to provide ‘actionable insight’. For a single term to encompass all the products that have enabled the unprecedented study of diverse, cheap, exploitable digital information across different verticals, a level of ambiguity is understandably necessary.
This ambiguity, however, also frames a backlash targeted towards the concept’s proponents. There are concerns that the very properties that make data ‘big’ (such as volume, variety, and velocity) also make data disordered and biased to the conventions established by a given architecture.1 It is argued that obtaining an authoritative data set on any subject is unlikely. The ‘fire hose’ quality of modern data collection, coupled with the inherent subjectivity of interpretation, brings into question the fidelity that any data set has to reality. Facts in the data are often assumed rather than proven and it is this critique that suggests that the much sought-after ‘actionable insights’ within data stores are not easily extractable. It is at this intersection where a purposeful definition for emerging Big Data technologies can be established: between the digital capability of mass collection/storage and the human ability to critique/hypothesise.
For the purposes of this paper, Big Data can be perceived as a concept that straddles this gulf; a collection of systems that empowers a human user to ask relevant questions of data sets in order to extract some ‘actionable insights’. The following sections aim to outline some of the concepts that have driven the adoption of Big Data in the defence and security sector, before going on to detailing existing use cases that are attempting to provide actionable insights.
Table Of Contents
Big Data in Defence and SecurityÂ An Indefinite Definition: What is Big Data? 3 Intelligence Amplification: What has Driven Big Data? 4 Open Source Intelligence 4 Activity-based Intelligence 6 Deep Learning 7 Big Data in the Context of AandD 8 Defence 8 Law Enforcement 10 Challenges and the Future 12 Legal Disclaimer 14 The Frost and Sullivan Story 15