Apache Spark is a popular emerging technology for real-time Big Data analytics. This demo uses a cluster of HP® Moonshot ProLiant® m400 64-bit ARM® Powered server cartridges (hosted within the HP Discovery Lab) – running Red Hat‘s ARM Development Preview Operating System release – to perform live streaming analysis of public tweets. Each inbound tweet is filtered to find tweets containing associated geographic information, then currently trending hashtags in various geographic regions of the world are identified and plotted onto a navigable map:
Scrolling around the map will allow you to discover regions of the world with trending hashtags (popular discussion topics) over the past hour. Representative locations of trending hashtags are shown as balloons. Please note that the cluster is not directly connected to the external world (due to HP lab security policy), and so the data is periodically synchronized onto the cloud based public system you are using to access this website.
About the Spark Cluster
The Apache Spark cluster performing this real-time analysis is located in the award winning HP Discovery Lab, running on HP Moonshot ProLiant m400 64-bit ARM server cartridges. Each cartridge contains an 8 core AppliedMicro X-Gene Server-on-a-Chip™ operating at up to 2.4GHz and featuring 64GB of memory, multiple Mellanox (PCIe based) 10Gbit Ethernet network links, and up to 400GB of local storage on each cartridge.
The m400 cartridges are running the latest HP firmware supporting ARM industry standards (including UEFI, SMBIOS, and ACPI – collectively part of the ARM Server Base System Architecture and Base Boot Requirements) that have been created with strong collaboration from both Red Hat and HP. These allow for a seamless customer experience from system provisioning through to datacenter management, supported by all of the standard system management technologies users have come to expect from today’s Enterprise.
You can learn more about support for the ARM Architecture within Red Hat technologies by visiting http://www.redhat.com/arm/