scala

Importance of function’s return Type in Scala

The Scala code is below:

abstract class Human {
        def call
}
class Student(val name: String) extends Human {
        override def call = "\"" + name + "\""
}
object MyApp extends App {
        var s:Student = new Student("robin")
        println(s.call)
}

But the output of this section of code is:

()

Nothing, just a pair of parenthesis. What happen? Why doesn’t Scala use the ‘call’ function in subclass ‘Student’?
The answer is: we forget to define the return Type of function ‘call’, so its default return Type is ‘Unit’. The value of ‘Unit’ is only ‘()’. Hence no matter what value we give to ‘call’, it only return ‘()’.
We just to add return Type:

abstract class Human {
        def call: String
}
......

It print “robin” now.

Books I read in year 2016

Here comes the last day of 2016 year. And it is also the time for me to review my harvest about knowledge, or books.
Frankly speaking, the book “All hard thing about hard things” literally frighten me, and cause me to give up any idea about joining a startup company in China. Maybe this is the best consequence, for many startup companies failed in this end of year and I fortunately avoid this tempest.
Diving more deeper into the ocean of “Hadoop Ecosystem”, or “Big Data”, I find out Spark is really a convenient and powerful framework (compare to MapReduce) which could implement complicated algorithm or data-flow with a few lines of code. Surely, Scala is also a key element for Spark’s efficiency and concision.
Today, even normal person could imagine a sci-fi story about how modern people will fight with Alien invaders. But, what will happen if Aliens attacked the earth in the ancient time? What about Medieval age? Then comes the funny and bold sci-fi novel “The High Crusade”. A group of Medieval army defeat the invader of Alien， and did even more: occupied a frontline planet of a gigantic Alien Empire. It is really out of my imagination 🙂

Use type of “Any” carefully in Scala

Think about the code below:

object My extends App {
        def increment(key: Any) {
                println(key);
        }
        increment("hello");
        increment("world", 6, 7);
}

I intent to see the compiler error for second ‘increment’ int the first place. But it don’t, the compiler report ok and the output of the program is:

hello
(world,6,7)

The compiler recognize the three arguments “world”, “6”, “7” as a tuple of (“world”, 6, 7). So the correct type of arugment for function ‘increment’ should be ‘String’:

def increment(key: String) {
....

Import data to Redis from JSON format

Redis use rdb file to make data persistent. Yesterday, I used the redis-rdb-tools for dumping data from rdb file to JSON format. After that, I write scala code to read data from JSON file and put it into Redis.
Firstly, I found out that the JSON file is almost 50% bigger than rdb file. After checking the whole JSON file, I make sure that the root cause is not the redundant symbols in JSON such as braces and brackets but the “Unicode transformation” in JSON format, especially the “\u0001” for ASCII of “0x01”. Therefore I write code to replace it:

val key: String = line.stripPrefix("\"").stripSuffix("\"").replaceAll("\\\\u0001", "^A")

Then the size of JSON file became normal.
There was still another problem. To read the JSON file line by line, I use code from http://naildrivin5.com/blog/2010/01/26/reading-a-file-in-scala-ruby-java.html:

import scala.io._
object ReadFile extends Application {
  val s = Source.fromFile("some_file.txt")
  s.getLines.foreach( (line) => {
    println(line.trim.toUpperCase)
  })
}

But this code will load all data from file and then run “foreach”. As my file is bigger than 100GB, it will cost too much time and RAM ….
The best way is to use IOStream in java:

val dataFile = new File("my.txt")
var reader = new BufferedReader(new FileReader(dataFile))
var line: String = reader.readLine()
while (line != null) {
    // do something
    line = reader.readLine()
}

This is exactly read file “line by line”.

From scala Array[String] / Seq[String] to java varargs

While testing performance of redis these days, I need to use mset() interface of jedis (a java version redis client). But the prototype of mset() in jedis is:

@Override
  public String mset(final String... keysvalues) {

Firstly I write my scala code like:

var array = Array[String]()
array = array:+key1:+value1
array = array:+key2:+value2
jedis.mset(array)

But it report compiling errors:

[error] xxx: overloaded method value mset with alternatives:
[error]   (x$1: String*)String 
[error]   (x$1: Array[Byte]*)String
[error]  cannot be applied to (Array[String])
[error]                                 jedis.mset(array)
[error]                                       ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Jan 8, 2016 11:32:47 AM

After searching many documents about scala/java on google, I finally find the answer: http://docs.scala-lang.org/style/types.html. So, let’s write code this way:

jedis.mset(array:_*)

Then Array[String] of scala changes to varargs in java now. It also viable for Seq[String].

Robin on Linux

scala