Raspberry Pi Based Hadoop Cluster

Recently one of our student researchers Saimul Hoque (Department of CSE) at our CCSE labs has put together a Raspberry Pi based Hadoop Cluster. If you are unaware of “Apache Hadoop” then just know that this is a Database cluster which is a special type of computational cluster designed specifically for storing and analyzing huge amounts of unstructured data in a distributed computing environment.

Using 4 Raspberry Pi 2 , Dedicated Power Rail, Temperature Monitoring with Cooling and a wifi router, a portable package has been created that can operate at a very low cost and power over conventional computers. Such devices are incredibly scalable as it can be done simply by adding more clusters at a fraction of the price of an entire computer, both storage and computation can be affordable. Raspberry Pi being solid state or memory based is also more stable at performing small data reads and writes over conventional hard drives. Maintainability and easy swap features make this a formidable device for usage in remote data collection such as surveys (which we are planning to deploy at IUB’s LFE course) or can be used to disseminate information or provide services to locations previously not possible due to cost. Currently, We are working to add a custom built higher gain wifi antenna and also perform rigorous stress test analysis on performance and stability.

The project has been initiated by Dr. Ali Shihab Sabbir (SPM of CCSE).