Somebody has finished the work about exporting YOLOv5 models to tflite model. To use it, we only need to:

git clone --single-branch --branch tf-export
cd yolov5
# it will download all pytorch models
sh -x weights/
# export a tflite model from yolov5l
PYTHONPATH=. python3 models/ --weights --cfg models/yolov5l.yaml --img 640
# there will be a tflite model file
ls yolov5l-fp16.tflite

The model file yolov5l-fp16.tflite is 91MB, which is a little too big but still could be put into a mobile phone.