Deploying Hadoop Architecture Using Ansible and Terraform

Management of excessive data in the Big Data world is difficult. Software frameworks and clusters help in the management of the data. Although the setting up of the cluster consumes greater time and effort to implement for the distributed data storage and processing. This problem can be automated with the usage of various Development Operation tools. Terraform is a powerful tool for Provisioning and Ansible is used for Configuration management and Application deployment tool. The proposed work is about using this tool to completely automate the process of setting up the Hadoop cluster with no human intervention. Ansible provides Infrastructure as a Code. There are many pre-created modules available in Ansible to make our task simpler and easier. The main feature of these modules is that they are idempotent. Making efficient use of these modules in building the code is foundation of this work. Ansible is preferred over the other tools because it is agentless and it uses SSH protocol to connect to Linux systems and Win RM protocol to connect to Windows systems. The entire architecture can be deployed with in no time using the automation of the Ansible and Terraform over the cloud servers.