What is Big Data?

Big Data is not a technology, but rather a phenomenon resulting from the vast amount of raw information generated across society, and collected by commercial and government organisations. Put simply, it is the management and exploitation of large or complex data sets.

• The increased use of big data : Total Installed base of data storage capacity in global datasphere 2020-2025. The installed base of global data storage capacity is expected to increase from 6.7 zettabytes (ZB) in 2020 to around 16 zettabytes in 2025.

• The increased use of data in defense sector : The Increased Market Size of Unmanned Aerial Vehicle. One MQ9 Reaper Sortie collects the data equivalent to 20 laptops

• Defense Advanced Research Projects Agency (DARPA) : collects over 40 gigabytes of information per second.

• The estimated volume of data currently collected by British UAS platforms in Afghanistan equates to between 1 and 5 per cent of the Google index, causing some commentators to observe that ‘the MoD doesn’t have Big Data. Google and Amazon do’

• Big Data also has applications in cyber-security, providing network managers with the means to process millions of daily attacks and identify the more serious attacks dubbed ‘advanced persistent threats’. This has led to increased employment of data analyst in the United States army

Areas where Data analyst have been used: It has already been applied in a broader context as part of the ‘human-terrain mapping’ exercise in Afghanistan, importing traditional databases, biometric data and imagery to extract information that would improve ISAF’s understanding of the local population and key relationships within it.

Comparative Data Requirement

The director of DARPA, a US agency responsible for developing new military technology, illustrated the case for automated analysis to a Congress hearing by comparing the intelligence collected for counter-insurgency operations with that for air defence against strategic bombing...

Signal Intelligence (SIGINT)

Communication Interception: Intelligence agencies collect and analyze data from intercepted communications, including emails, phone calls, and other forms of electronic communication, to gather information about potential threats, activities, and plans.

Geospatial Intelligence (GEOINT):

Satellite Imagery and Mapping: Data from satellites and other geospatial sources are used to monitor and analyze physical locations. This helps in tracking movements, identifying infrastructure, and understanding the geographical context of intelligence information.

Route Optimization

Data analytics helps in analyzing historical traffic patterns, weather conditions, and other relevant data to optimize delivery routes. This minimizes travel time, fuel consumption, and transportation costs

Predictive Maintenance

Analyzing sensor data from vehicles and equipment allows for predictive maintenance. By identifying potential issues before they cause a breakdown, logistics companies can reduce downtime, lower maintenance costs, and enhance fleet reliability.

Inventory Management

Data analytics can be used to analyze demand patterns, historical sales data, and other factors to optimize inventory levels. This helps in reducing carrying costs while ensuring products are available when needed.

Finance: It will help the defence forces to assess where they need to spend their money and from where the expenses could be reduced.

Future: In the UK at least, shrinking budgets have led to similarly shrinking armed forces. Future operations are likely to be conducted with fewer forces. There is likely to be an ever-greater dependence on battle-winning intelligence in order to underpin the command and control that the Vice Chief of the Defence Staff recently reflected had refined the application of force to a fine art.2

This will drive even greater reliance on bandwidth and the network that increasingly sits at the heart of the business of defence. Modern, capable infrastructure will therefore be key. New, more agile commercial relationships will also be needed in order to reduce the cost of operational maintenance and permit more investment in information-enablers matched to the expenditure in platforms and sensors
For this exact reason, the Australian Defence Imagery and Geospatial Organisation has adopted Big Data techniques to help analysts sift through satellite imagery and monitor changes in areas of interest around the world.
Separately, the NATO Allied Rapid Reaction Corps HQ is working with academics to find methods of applying Big Data analytics to open-source intelligence (OSINT) as a means of predicting likely targets for radicalisation (both communities and, increasingly, specific individuals) as well as to ‘collapse the distance between policy, planning and operations’ by capturing large datasets and seeking to identify patterns.