Swap Memory Usage While vm.swappiness=1

Long story short: Even if memory is not overcommitted and vm.swappiness is set to 1, if you run disk IO heavy workloads on Hadoop, also increase vm.vfs_cache_pressure (to 200, say) and vm.min_free_kbytes (to 5242880 kB […]

Cloudera Hadoop Cluster Deployment Automation Options

If you need transient/ephemeral Cloudera Hadoop (CDH) clusters for testing or training purposes, or you just simply would like to re-spawn the cluster using the same profile whenever necessary, you can use a wide range […]

SPNEGO Protected Hadoop Web Consoles Behind Proxy (ex. NameNode web console)

This post gives a quick overview on how Kerberos (SPNEGO) protected Hadoop web consoles (like NameNode, ResourceManager, JobHistoryServer, Hive, Oozie web interfaces) work and how they should be configured when a proxy is installed in […]