The Mapreduce framework is too small for realtime analytic query, so we need to change engine of Hive from “mr” to “spark” (link):
1. set environment for spark:

export SPARK_HOME=/home/my/spark/

2. copy configuration xml file for Hive:

cp /home/my/hive/conf/hive-default.xml.template /home/my/hive/conf/hive-site.xml

and change these configuration items:


  hive.execution.engine
  spark


  spark.executor.memory
  4g


  spark.serializer
  org.apache.spark.serializer.KryoSerializer

Notice: remember to replace all “${system:java.io.tmpdir}/${system:user.name}” in hive-site.xml to “/tmp/my/” (link)

Related Posts