shell编写streaming

前端之家收集整理的这篇文章主要介绍了shell编写streaming前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。
##mapper.sh
#! /bin/bash
while read LINE; do
  for word in $LINE
  do
    echo "$word 1"
  done
done



##reducer.sh
#! /bin/bash
count=0
started=0
word=""
while read LINE;do
  newword=`echo $LINE | cut -d ' '  -f 1`
  if [ "$word" != "$newword" ];then
    [ $started -ne 0 ] && echo -e "$word \t $count"
    word=$newword
    count=1
    started=1
  else
    count=$(( $count + 1 ))
  fi
done
echo -e "$word \t $count"

猜你在找的Bash相关文章