【Scala入门学习】集合常用方法和函数操作

news2024/11/24 1:51:01

1. foreach循环遍历

foreach 方法的原型：

// f 返回的类型是Unit， foreach 返回的类型是Unit
def foreach[U](f: Elem => U)

该方法接受一个函数 f 作为参数， 函数 f 的类型为Elem => U，即 f 接受一个参数，参数的类型为容器元素的类型Elem，f 返回结果类型为 U。foreach 遍历集合的每个元素，并将f 应用到每个元素上。

res176: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7)

scala> arr.foreach(t=>{
     | if(t%2==0)
     | println(t)
     | else
     | println(t*2)
     | })
2
2
6
4
10
6

scala> val arr = Array(Array(11,22,33,44),Array(1,2,3,4))
arr: Array[Array[Int]] = Array(Array(11, 22, 33, 44), Array(1, 2, 3, 4))

scala> arr.foreach(arr=> arr.foreach(t=>{
     | if(t%2==0)
     | println(t)
     | else
     | println(t*2)
     | }))

2. map

map操作是针对集合的典型变换操作，它将某个函数应用到集合中的每个元素，并产生一个结果集合；

map方法返回一个与原集合类型大小都相同的新集合，只不过元素的类型可能不同。

map带有返回值，集合类型和元类型一样，元素的个数不会发生变化，元素本身会变化

scala> arr
res193: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.map(t=>t*2)
res194: Array[Int] = Array(2, 4, 6, 8, 10, 12, 14, 16, 18)

scala> arr.foreach(t=>println(t*2))
2
4
6
8
10
12
14
16
18

scala> arr.map(t=> if(t<5) t+1 else t )
res196: Array[Int] = Array(2, 3, 4, 5, 5, 6, 7, 8, 9)

scala> arr.map(t=> if(t<5) t+1 else t ).map(t=> if(t%2==0)t else t*2)
res197: Array[Int] = Array(2, 6, 4, 10, 10, 6, 14, 8, 18)

scala> arr
res198: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.map(t=> if(t%2==0) (t,"even") else (t,"odd"))
res199: Array[(Int, String)] = Array((1,odd), (2,even), (3,odd), (4,even), (5,odd), (6,even), (7,odd), (8,even), (9,odd))

//多重集合的基础使用
//最高成绩
scala> val arr = Array(("zhangsan",Array(100,120,50,45,34)),("zhaosi",Array(90,98,34,32,13)))
arr: Array[(String, Array[Int])] = Array((zhangsan,Array(100, 120, 50, 45, 34)), (zhaosi,Array(90, 98, 34, 32, 13)))

scala> arr.map(tp=>(tp._1,tp._2.max))
res200: Array[(String, Int)] = Array((zhangsan,120), (zhaosi,98))

//多层嵌套array的处理
//每个人的及格成绩有几个
scala> arr
res201: Array[(String, Array[Int])] = Array((zhangsan,Array(100, 120, 50, 45, 34)), (zhaosi,Array(90, 98, 34, 32, 13)))

scala> arr.map(tp=>(tp._1,tp._2.map(t=>if(t>=60) 1 else 0).sum))
res202: Array[(String, Int)] = Array((zhangsan,2), (zhaosi,2))
//平均成绩
scala> arr
res203: Array[(String, Array[Int])] = Array((zhangsan,Array(100, 120, 50, 45, 34)), (zhaosi,Array(90, 98, 34, 32, 13)))

scala> arr.map(tp=> (tp._1,tp._2.sum * 1.0 / tp._2.size))
res204: Array[(String, Double)] = Array((zhangsan,69.8), (zhaosi,53.4))

3. flatten扁平化

当有一个集合的集合，然后你想对这些集合的所有元素进行操作时，就会用到 flatten；

List(List(1,2), List(3,4)) —–> List(1,2,3,4)

List(Array(1,2),Array(3,4)) —–> List(1,2,3,4)

List(Map(“a”->1,“b”->2), Map(“c”->3,“d”->4)) —–> List((a,1), (b,2), (c,3), (d,4))

scala> val arr = Array(List(1,2,3),List(4,5,6))
arr: Array[List[Int]] = Array(List(1, 2, 3), List(4, 5, 6))

scala> arr.flatten
res4: Array[Int] = Array(1, 2, 3, 4, 5, 6)

scala> val arr1 = Array(1,2,3,4,5,6,Array(7,8,9))
arr1: Array[Any] = Array(1, 2, 3, 4, 5, 6, Array(7, 8, 9))

scala> arr1.flatten
<console>:13: error: No implicit view available from Any => Traversable[U].
       arr1.flatten
            ^

scala> val set = Set(1,2,3)
set: scala.collection.immutable.Set[Int] = Set(1, 2, 3)

scala> val set1 = Set(set,set,set)
set1: scala.collection.immutable.Set[scala.collection.immutable.Set[Int]] = Set(Set(1, 2, 3))

scala> val set1 = Set(Set(1,2,3),Set(4,5,6))
set1: scala.collection.immutable.Set[scala.collection.immutable.Set[Int]] = Set(Set(1, 2, 3), Set(4, 5, 6))

scala> set1.flatten
res6: scala.collection.immutable.Set[Int] = Set(5, 1, 6, 2, 3, 4)

scala> ((1,2),(3,4))
res7: ((Int, Int), (Int, Int)) = ((1,2),(3,4))

scala> res7.flatten
<console>:13: error: value flatten is not a member of ((Int, Int), (Int, Int))
       res7.flatten
            ^

scala> val arr1 =Array(Array(arr,arr))
arr1: Array[Array[Array[List[Int]]]] = Array(Array(Array(List(1, 2, 3), List(4, 5, 6)), Array(List(1, 2, 3), List(4, 5, 6))))

scala> arr1.flatten
res9: Array[Array[List[Int]]] = Array(Array(List(1, 2, 3), List(4, 5, 6)), Array(List(1, 2, 3), List(4, 5, 6)))

scala> arr1.flatten.flatten
res10: Array[List[Int]] = Array(List(1, 2, 3), List(4, 5, 6), List(1, 2, 3), List(4, 5, 6))

scala> arr1.flatten.flatten.flatten
res11: Array[Int] = Array(1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6)

注意：flatten 不支持元组，因为元组不是集合。

// 下面的方法报错
val list = List((1,2), (3,4))
list.flatten

4. faltmap（map + faltten）

flatMap的执行过程： map –> flatten

括号内完成map的逻辑，自动帮你完成faltten扁平化操作。

scala> val arr = Array(1,2,3,4)
arr: Array[Int] = Array(1, 2, 3, 4)

scala> val arr1 = Array(arr,arr)
arr1: Array[Array[Int]] = Array(Array(1, 2, 3, 4), Array(1, 2, 3, 4))

scala> arr1.flatten
res14: Array[Int] = Array(1, 2, 3, 4, 1, 2, 3, 4)

scala> arr1.flatten.map(t=>t*2)
res15: Array[Int] = Array(2, 4, 6, 8, 2, 4, 6, 8)

scala> val arr = Array("a b c d e","a d e f g")
arr: Array[String] = Array(a b c d e, a d e f g)

scala> arr.map(t=> t.split(" "))
res16: Array[Array[String]] = Array(Array(a, b, c, d, e), Array(a, d, e, f, g))

scala> res16.flatten
res17: Array[String] = Array(a, b, c, d, e, a, d, e, f, g)

scala> val arr = Array(("a","b"),("c","d"))
arr: Array[(String, String)] = Array((a,b), (c,d))

scala> arr.map(t=>Array(t._1,t._2))
res18: Array[Array[String]] = Array(Array(a, b), Array(c, d))

scala> res18.flatten
res20: Array[String] = Array(a, b, c, d)

scala> arr
res21: Array[(String, String)] = Array((a,b), (c,d))
//加入flatMap
scala> arr.map(t=> Array(t._1,t._2)).flatten
res22: Array[String] = Array(a, b, c, d)

scala> arr.flatMap(t=>Array(t._1,t._2))
res23: Array[String] = Array(a, b, c, d)

scala> val arr = Array("a b c d e","a d e f g")
arr: Array[String] = Array(a b c d e, a d e f g)

scala> arr.flatMap(t=> t.split(" "))
res24: Array[String] = Array(a, b, c, d, e, a, d, e, f, g)

注意：同flatten一样，不支持元组

5. filter过滤

遍历一个集合并从中获取满足指定条件的元素组成一个新的集合；

元素不会发生变化，集合也不会发生变化

但是元素的个数可能会发生变化。

scala> val arr1 = Array(1,2,3,4,5,6,7,8,9)
arr1: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr1.filter
filter   filterNot

scala> arr1.filter
   def filter(p: Int => Boolean): Array[Int]

scala> arr1.filter(t=> if(t>5) true else false)
res205: Array[Int] = Array(6, 7, 8, 9)

scala> arr1.filter(t=> t>5)
res206: Array[Int] = Array(6, 7, 8, 9)

高级使用

scala> arr
res207: Array[(String, Array[Int])] = Array((zhangsan,Array(100, 120, 50, 45, 34)), (zhaosi,Array(90, 98, 34, 32, 13)))

scala> arr.map(t=> (t._1,t._2.filter(t=> t>=60).size))
res208: Array[(String, Int)] = Array((zhangsan,2), (zhaosi,2))

filterNot

scala> arr
res207: Array[(String, Array[Int])] = Array((zhangsan,Array(100, 120, 50, 45, 34)), (zhaosi,Array(90, 98, 34, 32, 13)))

scala> arr1
res209: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr1.filter(t=> t%2==0)
res210: Array[Int] = Array(2, 4, 6, 8)

scala> arr1.filterNot(t=> t%2==0)
res211: Array[Int] = Array(1, 3, 5, 7, 9)

scala> arr.map(t=> (t._1,t._2.filterNot(t=> t<60).size))
res212: Array[(String, Int)] = Array((zhangsan,2), (zhaosi,2))

6. sorted、sortBy、sortWith排序

sorted：按照元素自身进行排序；

sortBy: 按照应用函数 f 之后产生的元素进行排序；

sortWith:传入ab两个值，然后返回比较规则比如a>b

val arr = Array(1,9,2,8,3,7,4,6,5)
arr: Array[Int] = Array(1, 9, 2, 8, 3, 7, 4, 6, 5)

scala> arr.sort
sortBy   sortWith   sorted

scala> arr.sorted
res30: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.sorted.reverse
res31: Array[Int] = Array(9, 8, 7, 6, 5, 4, 3, 2, 1)

scala> val arr = Array(Array(1,2,3),Array(3,4,5,6,7,8))
arr: Array[Array[Int]] = Array(Array(1, 2, 3), Array(3, 4, 5, 6, 7, 8))

scala> arr.sorted
<console>:13: error: No implicit Ordering defined for Array[Int].
       arr.sorted
           ^

scala> 

scala> val arr1 = Array(1,9,2,8,3,7,4,6,5)
arr1: Array[Int] = Array(1, 9, 2, 8, 3, 7, 4, 6, 5)

scala> arr1.sortBy(t=> t)
res33: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr1.sortBy(t=> -t)
res34: Array[Int] = Array(9, 8, 7, 6, 5, 4, 3, 2, 1)

scala> 

scala> val arr2 = Array(("zhangsan",28000),("lisi",25000),("wangwu",26000))
arr2: Array[(String, Int)] = Array((zhangsan,28000), (lisi,25000), (wangwu,26000))

scala> arr2.sorted
res35: Array[(String, Int)] = Array((lisi,25000), (wangwu,26000), (zhangsan,28000))

scala> val arr2 = Array(("zhangsan",28000),("lisi",25000),("wangwu",26000),("azhen",30000))
arr2: Array[(String, Int)] = Array((zhangsan,28000), (lisi,25000), (wangwu,26000), (azhen,30000))

scala> arr2.sorted
res36: Array[(String, Int)] = Array((azhen,30000), (lisi,25000), (wangwu,26000), (zhangsan,28000))

scala> arr2.sortBy(t=> t._2)
res37: Array[(String, Int)] = Array((lisi,25000), (wangwu,26000), (zhangsan,28000), (azhen,30000))

scala> arr2.sortBy(t=> -t._2)
res38: Array[(String, Int)] = Array((azhen,30000), (zhangsan,28000), (wangwu,26000), (lisi,25000))

scala> 

scala> arr1
res39: Array[Int] = Array(1, 9, 2, 8, 3, 7, 4, 6, 5)

scala> arr
res40: Array[Array[Int]] = Array(Array(1, 2, 3), Array(3, 4, 5, 6, 7, 8))

scala> arr.sortBy(t=> t.sum/t.size)
res41: Array[Array[Int]] = Array(Array(1, 2, 3), Array(3, 4, 5, 6, 7, 8))

scala> arr.sortBy(t=> -t.sum/t.size)
res42: Array[Array[Int]] = Array(Array(3, 4, 5, 6, 7, 8), Array(1, 2, 3))

scala> 

scala> arr.map(t=> t.sortBy(e=> -e))
res43: Array[Array[Int]] = Array(Array(3, 2, 1), Array(8, 7, 6, 5, 4, 3))

scala> 

scala> arr1
res44: Array[Int] = Array(1, 9, 2, 8, 3, 7, 4, 6, 5)

scala> arr1.sortWith((a,b)=> a<b)
res45: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr1.sortWith((a,b)=> a>b)
res46: Array[Int] = Array(9, 8, 7, 6, 5, 4, 3, 2, 1)

scala> arr
res47: Array[Array[Int]] = Array(Array(1, 2, 3), Array(3, 4, 5, 6, 7, 8))

scala> arr.sortWith((a,b)=> a.sum/a.length > b.sum/b.length)
res48: Array[Array[Int]] = Array(Array(3, 4, 5, 6, 7, 8), Array(1, 2, 3))

scala> arr2
res49: Array[(String, Int)] = Array((zhangsan,28000), (lisi,25000), (wangwu,26000), (azhen,30000))

scala> arr2.sortWith((a,b)=> a._2 > b._2)
res50: Array[(String, Int)] = Array((azhen,30000), (zhangsan,28000), (wangwu,26000), (lisi,25000))

7. 并行集合

通过list.par 会将集合变成并行集合，可以利用多线程来进行运算。

def main(args: Array[String]): Unit = {
    val arr = Array(1,2,3,4,5,6,7,8,9)
    arr.par.map(t=>{
      val name = Thread.currentThread().getName
      (name,t)
    }).foreach(println)
  }

8. reduce、reduceLeft、reduceRight 

reduce：reduce(op: (A1, A1) => A1): A1 。reduce操作是按照从左到右的顺序进行规约。(((1+2)+3)+4)+5

reduceLeft：reduceLeft[B >: A](f: (B, A) => B): B。是按照从左到右的顺序进行规约。 (((1+2)+3)+4)+5

reduceRight：reduceRight[B >: A](op: (A, B) => B): B。是按照从右到左的顺序进行规约。1+(2+(3+(4+5)))

单线程下： reduce 和 reduceLeft一样

并行集合运行下： reduce利用CPU数运行， reduceLeft 有方向，只能单线程运行

reduce

scala> arr.reduce((a,b)=> a+b)
res4: Int = 45

scala> arr.reduce((a,b)=> a*b)
res5: Int = 362880

scala> arr.reduce((a,b)=> a-b)
res6: Int = -43

scala> arr.reduce((a,b)=> a*(a-b))
res7: Int = 0

scala> arr.reduce((a,b)=> if(a>b) a else b)
res8: Int = 9

scala> arr.reduce((a,b)=> if(a<b) a else b)
res9: Int = 1

scala> val arr1 = Array(("a",10),("b",20),("c",15))
arr1: Array[(String, Int)] = Array((a,10), (b,20), (c,15))

scala> arr1.reduce((a,b)=> a._2 + b._2)
<console>:13: error: type mismatch;
 found   : Int
 required: (String, Int)
       arr1.reduce((a,b)=> a._2 + b._2)
                                ^

scala> arr1.reduce((a,b)=> ("",a._2+b._2))
res11: (String, Int) = ("",45)
def main(args: Array[String]): Unit = {
    val arr = Array(1,2,3,4,5,6,7,8,9)
    val num = arr.par.reduce((a,b) => {
        val name = Thread.currentThread().getName
        println(name,a,b)
        a+b
    })
    println(num)
}

reduceLeft和reduceRight

val arr = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)
//    val num1 = arr.reduceLeft((a, b) => a + b)
//    val num2 = arr.reduceRight((a, b) => a + b)
//    println(num1)
//    println(num2)
val num = arr.par.reduce((a, b) => {
    val name = Thread.currentThread().getName
    println(name,a, b)
    a - b
})
val num1 = arr.par.reduceLeft((a, b) => {
    val name = Thread.currentThread().getName
    println(name, a, b)
    a - b
})
val num2 = arr.par.reduceRight((a, b) => {
    val name = Thread.currentThread().getName
    println(name, a, b)
    a - b
})
println(num1)
println(num2)
}

类型展示

scala> arr.reduce((a,b)=> a+b)
res3: Int = 15

scala> arr.reduceLeft((a,b)=> a+b)
res4: Int = 15

scala> arr.reduceLeft((a:AnyVal,b:Int) => (a.asInstanceOf[Int]+b).asInstanceOf[AnyVal])
res5: AnyVal = 15

scala> arr.reduce((a:AnyVal,b:Int) => (a.asInstanceOf[Int]+b).asInstanceOf[AnyVal])
<console>:13: error: type mismatch;
 found   : (AnyVal, Int) => AnyVal
 required: (AnyVal, AnyVal) => AnyVal
       arr.reduce((a:AnyVal,b:Int) => (a.asInstanceOf[Int]+b).asInstanceOf[AnyVal])
                                   ^

scala> arr.reduceRight
reduceRight   reduceRightOption

scala> arr.reduceRight
   override def reduceRight[B >: Int](op: (Int, B) => B): B

scala> arr.reduceRight((a:Int,b:AnyVal) => (a+b.asInstanceOf[Int]).asInstanceOf[AnyVal])
res7: AnyVal = 15

如何简写？

scala> val list = List(1,2,3,4,5)
list: List[Int] = List(1, 2, 3, 4, 5)
scala> list.reduce((a:Int,b:Int) => {a + b})
res18: Int = 15
scala> list.reduce((a:Int,b:Int) => {println(s"${a} + ${b} = ${a + b}");a + b})
1 + 2 = 3
3 + 3 = 6
6 + 4 = 10
10 + 5 = 15
res19: Int = 15
scala> list.sum
res20: Int = 15
// 求最大值
scala> list.max
res21: Int = 5
// 用reduce实现求最大值
scala> list.reduce((a:Int,b:Int) => {println(s"${a} vs ${b}");if(a > b) a else b})
1 vs 2
2 vs 3
3 vs 4
4 vs 5
res22: Int = 5
scala> list.reduce((a:Int,b:Int) => {a + b})
res23: Int = 15
scala> list.reduce(_ + _)
res24: Int = 15
scala> list.par.reduce(_ + _)
res25: Int = 15

9. fold, foldLeft, foldRight （带有初始值的reduce）

fold：fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1 。带有初始值的reduce，从一个初始值开始，从左向右将两个元素合并成一个，最终把列表合并成单一元素。((((10+1)+2)+3)+4)+5

foldLeft：foldLeft[B](z: B)(f: (B, A) => B): B 。带有初始值的reduceLeft。((((10+1)+2)+3)+4)+5

foldRight：foldRight[B](z: B)(op: (A, B) => B): B。带有初始值的reduceRight。1+(2+(3+(4+(5+10))))

fold代码

scala> arr
res8: Array[Int] = Array(1, 2, 3, 4, 5)

scala> arr.fold
fold   foldLeft   foldRight

scala> arr.fold
   def fold[A1 >: Int](z: A1)(op: (A1, A1) => A1): A1

scala> arr.fold(0)((a,b)=> a+b)
res9: Int = 15

scala> val arr = Array(("a",10),("b",20),("c",15))
arr: Array[(String, Int)] = Array((a,10), (b,20), (c,15))

scala> arr.fold(0)((a,b)=>a+b._2)
<console>:13: error: value _2 is not a member of Any
       arr.fold(0)((a,b)=>a+b._2)
                              ^

scala> arr.fold(("",0))((a,b)=>a._2+b._2)
<console>:13: error: type mismatch;
 found   : Int
 required: (String, Int)
       arr.fold(("",0))((a,b)=>a._2+b._2)
                                   ^

scala> arr.fold(("",0))((a,b)=>("",a._2+b._2))
res12: (String, Int) = ("",45)
//fold多线程
  def main(args: Array[String]): Unit = {
    val arr = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)
    val num = arr.par.fold(0)((a,b) => {
      val name = Thread.currentThread().getName
      println(name,a,b)
      a+b
    })
    println(num)
  }

foldLeft和foldRight

scala> val arr = Array(1,2,3,4,5,6,7,8,9)
arr: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.foldLeft(0)((a,b)=>a-b)
res14: Int = -45

scala> arr.foldRight(0)((a,b)=> a-b)
res15: Int = 5

scala> arr.foldRight(0)((a,b)=> {
     | println(a,b)
     | a-b
     | })
(9,0)
(8,9)
(7,-1)
(6,8)
(5,-2)
(4,7)
(3,-3)
(2,6)
(1,-4)
res16: Int = 5

scala> arr.fold
   def fold[A1 >: Int](z: A1)(op: (A1, A1) => A1): A1

scala> arr.foldLeft
   override def foldLeft[B](z: B)(op: (B, Int) => B): B

scala> val arr = Array(("a",10),("b",20),("c",15))
arr: Array[(String, Int)] = Array((a,10), (b,20), (c,15))

scala> arr.foldLeft(0)((a,b)=>a+b._2)
res17: Int = 45

scala> arr.foldRight
   override def foldRight[B](z: B)(op: ((String, Int), B) => B): B

scala> arr.foldRight(0)((a,b)=> b+a._2)
res18: Int = 45
def main(args: Array[String]): Unit = {
    val arr = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)
    val num = arr.par.foldRight(0)((a, b) => {
      val name = Thread.currentThread().getName
      println(name, a, b)
      a + b
    })
    println(num)
  }

10. aggregate （分区内元素聚合）

将每个分区里面的元素进行聚合，然后用combine函数将每个分区的结果和初始值进行combine操作；

val list = List(1,2,3,4,5)
    // 当集合不是并行集合时，combop函数不执行
    val sum: Int = list.aggregate(0)((a: Int, b: Int) => {
      println(s"step1:a:${a}, b:${b}")
      a + b
    },
      (a: Int, b: Int) => {
        println(s"step2:a:${a}, b:${b}")
        a + b
      }
    )
    println(sum)
//-------运行结果-----------------------------
step1:a:0, b:1
step1:a:1, b:2
step1:a:3, b:3
step1:a:6, b:4
step1:a:10, b:5
15

//--------------------------------------------------------------
    val list = List(1,2,3,4,5)
    // 当集合是并行集合时，combop函数执行
    // step1:做基础聚合， step2:在step1 基础上做聚合，相当于combiner
    val sum2: Int = list.par.aggregate(0)((a: Int, b: Int) => {
      println(s"step1:a:${a}, b:${b}")
      a + b
    },
      (a: Int, b: Int) => {
        println(s"step2:a:${a}, b:${b}")
        a + b
      }
    )
    println(sum2)
//-------运行结果-----------------------------
15
step1:a:0, b:1
step1:a:0, b:4
step1:a:0, b:5
step1:a:0, b:2
step1:a:0, b:3
step2:a:1, b:2
step2:a:4, b:5
step2:a:3, b:9
step2:a:3, b:12
15

总结：

reduce/reduceLeft/reduceRight: 认为每个元素类型一样

fold: 带有初始值的reduce，初始值类型和元素类型一样；并行集合下注意初始值的设定；

foldLeft/foldRight： 初始值类型和元素类型可以不一样，规约结果和初始值类型一致；并行集合下是单线程运算

aggregate：初始值类型和元素类型可以不一样，规约结果和初始值类型一致；并行集合下是利用CPU核数运算

11. groupBy、grouped （分组）

groupBy：将list 按照某个元素内的字段分组，返回map。 List((k,v),(k,v)) –> Map(k, List(k,v))

grouped：按列表按照固定的大小进行分组，返回迭代器。List(1,2,3,4,5) –> Iterator[List[A]]

scala> val arr = Array(1,2,3,4,5,6,7,8,9,10)
arr: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

scala> arr.groupBy
   def groupBy[K](f: Int => K): scala.collection.immutable.Map[K,Array[Int]]

scala> arr.groupBy(t=> if(t%2 == 0) "even" else "odd")
res21: scala.collection.immutable.Map[String,Array[Int]] = Map(odd -> Array(1, 3, 5, 7, 9), even -> Array(2, 4, 6, 8, 10))

scala> arr.groupBy(t=> t%2==0)
res22: scala.collection.immutable.Map[Boolean,Array[Int]] = Map(false -> Array(1, 3, 5, 7, 9), true -> Array(2, 4, 6, 8, 10))

scala> val arr = Array("hello","hehe","world","hahah","what","kitty","kill","zoo","zookeeper")
arr: Array[String] = Array(hello, hehe, world, hahah, what, kitty, kill, zoo, zookeeper)

scala> arr.groupBy(t=> if(t.startswith("w")) "w" else ....)
<console>:1: error: illegal start of simple expression
       arr.groupBy(t=> if(t.startswith("w")) "w" else ....)
                                                      ^

scala> arr.groupBy(t=> t.substring(0,1))
res23: scala.collection.immutable.Map[String,Array[String]] = Map(h -> Array(hello, hehe, hahah), w -> Array(world, what), k -> Array(kitty, kill), z -> Array(zoo, zookeeper))

scala> val arr2 = Array(("zhangsan",100),("lisi",98),("wangwu",45),("zhaosi",36))
arr2: Array[(String, Int)] = Array((zhangsan,100), (lisi,98), (wangwu,45), (zhaosi,36))

scala> arr2.groupBy(t=> t._2 >= 60)
res24: scala.collection.immutable.Map[Boolean,Array[(String, Int)]] = Map(true -> Array((zhangsan,100), (lisi,98)), false -> Array((wangwu,45), (zhaosi,36)))

grouped分组

scala> val arr = Array(1,2,3,4,5,6,7,8,9)
arr: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.grouped(3)
res25: Iterator[Array[Int]] = <iterator>

scala> res25.foreach(println)
[I@161c8305
[I@696f286d
[I@1aebf638

scala> res25.foreach(t=>println(t.toList))

scala> arr.grouped(3).foreach(t=> println(t.toList))
List(1, 2, 3)
List(4, 5, 6)
List(7, 8, 9)

scala> arr
res29: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> arr.grouped(2)
res30: Iterator[Array[Int]] = <iterator>

scala> res30.map(t=> t.reverse)
res31: Iterator[Array[Int]] = <iterator>

scala> res31.flatten
res32: Iterator[Int] = <iterator>

scala> res32.mkString(" ")
res33: String = 2 1 4 3 6 5 8 7 9

12. mapValues

对map映射里每个key的value 进行操作。

val map = Map("b" -> List(1,2,3), "a" -> List(4,5,6))
// 对每个key的value 求和
val n1 = map.mapValues(_.sum)
println(n1)

group by 和 mapValues 组合

scala> arr
res34: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> //奇数和偶数的平均值

scala> arr.groupBy(t=> t%2==0)
res35: scala.collection.immutable.Map[Boolean,Array[Int]] = Map(false -> Array(1, 3, 5, 7, 9), true -> Array(2, 4, 6, 8))

scala> res35.map(t=>(t._1,t._2.sum * 1.0/t._2.size))
res36: scala.collection.immutable.Map[Boolean,Double] = Map(false -> 5.0, true -> 5.0)

scala> val arr1 = Array("hello","hello","tom","world","tom","world")
arr1: Array[String] = Array(hello, hello, tom, world, tom, world)

scala> arr1.groupBy(t=> t)
res37: scala.collection.immutable.Map[String,Array[String]] = Map(hello -> Array(hello, hello), tom -> Array(tom, tom), world -> Array(world, world))

scala> res37.map(t=> (t._1,t._2.size))
res38: scala.collection.immutable.Map[String,Int] = Map(hello -> 2, tom -> 2, world -> 2)

scala> val map = Map(("zhangsan",20000),("lisi",30000),("wangwu",18000))
map: scala.collection.immutable.Map[String,Int] = Map(zhangsan -> 20000, lisi -> 30000, wangwu -> 18000)

scala> map.map(t=>(t._1,t._2+1000))
res39: scala.collection.immutable.Map[String,Int] = Map(zhangsan -> 21000, lisi -> 31000, wangwu -> 19000)

scala> map.mapValues(t=> t+1000)
res40: scala.collection.immutable.Map[String,Int] = Map(zhangsan -> 21000, lisi -> 31000, wangwu -> 19000)

scala> arr1
res41: Array[String] = Array(hello, hello, tom, world, tom, world)

scala> arr1.groupBy(t=> t)
res42: scala.collection.immutable.Map[String,Array[String]] = Map(hello -> Array(hello, hello), tom -> Array(tom, tom), world -> Array(world, world))

scala> res42.mapValues(t=> t.size)
res43: scala.collection.immutable.Map[String,Int] = Map(hello -> 2, tom -> 2, world -> 2)

scala> //操作的数据必须是map集合

scala> //不能是List((k,v),(k,v))

scala> val list = List(("a",10),("b",20))
list: List[(String, Int)] = List((a,10), (b,20))

scala> list.mapValues(t=> t+1)
<console>:13: error: value mapValues is not a member of List[(String, Int)]
       list.mapValues(t=> t+1)
            ^

13. diff, union, intersect

diff : 两个集合的差集；

union : 两个集合的并集；

intersect: 两个集合的交集；

scala> val arr = Array(1,2,3,4,5)
arr: Array[Int] = Array(1, 2, 3, 4, 5)

scala> val arr2 = Array(3,4,5,6,7)
arr2: Array[Int] = Array(3, 4, 5, 6, 7)

scala> arr intersect arr2
res45: Array[Int] = Array(3, 4, 5)

scala> arr diff arr2
res46: Array[Int] = Array(1, 2)

scala> arr2 diff arr
res47: Array[Int] = Array(6, 7)

scala> arr union arr2
res48: Array[Int] = Array(1, 2, 3, 4, 5, 3, 4, 5, 6, 7)

scala> val arr = Array(("zhangsan","math"),("lisi","math"),("zhangsan","chinese"),("zhaosi","english"))
arr: Array[(String, String)] = Array((zhangsan,math), (lisi,math), (zhangsan,chinese), (zhaosi,english))

scala> val arr1 = Array("zhangsan","lisi","wangwu","zhaosi","guangkun")
arr1: Array[String] = Array(zhangsan, lisi, wangwu, zhaosi, guangkun)

scala> arr.map(t=>t._1)
res49: Array[String] = Array(zhangsan, lisi, zhangsan, zhaosi)

scala> res49 intersect arr1
res50: Array[String] = Array(zhangsan, lisi, zhaosi)

scala> arr1 diff res49
res51: Array[String] = Array(wangwu, guangkun)

14.实现 wordcount

实现统计单词的个数

scala> val list = List("hello tom hello jack","hello tom hello world world world")
list: List[String] = List(hello tom hello jack, hello tom hello world world world)

scala> list.flatMap(t=> t.split(" "))
res52: List[String] = List(hello, tom, hello, jack, hello, tom, hello, world, world, world)

scala> res52.groupBy(t=> t)
res53: scala.collection.immutable.Map[String,List[String]] = Map(hello -> List(hello, hello, hello, hello), tom -> List(tom, tom), jack -> List(jack), world -> List(world, world, world))

scala> res53.mapValues(t=> t.size)
res54: scala.collection.immutable.Map[String,Int] = Map(hello -> 4, tom -> 2, jack -> 1, world -> 3)

scala> res54.foreach(println)
(hello,4)
(tom,2)
(jack,1)
(world,3)

idea版本统计单词次数

def main(args: Array[String]): Unit = {
    val it: Iterator[String] = Source.fromFile("data/word.txt").getLines() //hello tom hello world
    val data1:Iterator[String] = it.flatMap(t=> t.split(" "))
    val data2:Iterator[(String,Int)] = data1.map(t=>(t,1))
    val list:List[(String,Int)] = data2.toList
    val map:Map[String,List[(String,Int)]] = list.groupBy(t=> t._1)
    val res:Map[String,Int] = map.mapValues(t=> t.size)
    res.foreach(println)
}

wordcount的改版

def main(args: Array[String]): Unit = {
    val it: Iterator[String] = Source.fromFile("data/word.txt").getLines() //hello tom hello world
    val data1: Iterator[String] = it.flatMap(t => t.split(" "))
    val data2: Iterator[(String, Int)] = data1.map(t => (t, 1))
    val list: List[(String, Int)] = data2.toList
    val map: Map[String, List[(String, Int)]] = list.groupBy(t => t._1)
    //    val res:Map[String,Int] = map.mapValues(t=> t.size)
    //    val res = map.mapValues(t => t.map(e => e._2).sum)
//    val res = map.mapValues(t => t.reduce((a, b) => ("", a._2 + b._2))._2)
//    val res = map.mapValues(t => t.foldLeft(0)((a, b) => a + b._2))
    val res = map.mapValues(t=> t.par.aggregate(0)((a,b)=> a+b._2,(a,b)=> a+b))
    res.foreach(println)
  }

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/2209258.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

达梦数据库（DM）单机典型安装

达梦数据库（DM）单机典型安装

达梦数据库&#xff08;DM&#xff09;单机典型安装 环境&#xff1a;centos7.6 1、创建用户 #增加用户和组&#xff0c;用于安装管理达梦数据库。 新建用户组&#xff1a;groupadd dinstall 新建用户&#xff1a;指定用户组&#xff0c;家目录&#xff0c;shell。useradd -g…
阅读更多...
反转链表解题思路

反转链表解题思路

题目描述 给定一个单链表的头结点pHead&#xff0c;长度为n&#xff0c;反转该链表后&#xff0c;返回新链表的表头。 示例&#xff1a;当输入链表{1,2,3}时&#xff0c;经反转后&#xff0c;原链表变为{3,2,1}&#xff0c;所以对应的输出为{3,2,1}。 解题思路&#xff1a;迭…
阅读更多...
股市大涨下的会展业创新者

股市大涨下的会展业创新者

近期&#xff0c;股市涨势强劲有力&#xff0c;各大指数普遍上扬&#xff0c;市场活力空前。与此同时&#xff0c;伴随全球经济逐步复苏及会展行业不断发展&#xff0c;上市展览公司机遇与挑战并存。国内外市场需求持续增长拓展了广阔发展空间&#xff0c;但同时行业竞争愈发激…
阅读更多...
中国宏观经济与产业发展：挑战与机遇并存

中国宏观经济与产业发展：挑战与机遇并存

#长沙屿# 在复杂多变的国内外经济形势之下&#xff0c;中国经济已然步入一个至关重要的发展阶段。今日&#xff0c;让我们深入剖析当前经济形势&#xff0c;对中国宏观经济的运行现状及产业发展的趋势展开深度探讨。 2024年&#xff0c;中国经济运行总体平稳、稳中有进&#x…
阅读更多...
职场启悟：没有靠山的你，45岁前必知的5大潜规则

职场启悟：没有靠山的你，45岁前必知的5大潜规则

我是农村孩子&#xff0c;父母都是农民&#xff0c;毕业一切都是靠着自己找工作&#xff0c;在陌生的大城市除了认识老师就是同学。记得那是我初入职场的第三个月&#xff0c;每天我都沉浸在无尽的工作中&#xff0c;加班到深夜&#xff0c;周末也时常无休。我觉的农村孩子只能…
阅读更多...
Chainbase ：链原生的 Web3 AI 基建设施

Chainbase ：链原生的 Web3 AI 基建设施

“随着 Chainbase 在生态系统和市场方面的进一步拓展&#xff0c;其作为链原生 Web3 AI 基建设施的价值将愈发显著。” 算法、算力和数据是 AI 技术的三大核心要素。实际上&#xff0c;几乎所有的 AI 大模型都在不断革新算法&#xff0c;以确保模型能够跟上行业的发展趋势&…
阅读更多...
机器学习中的模型设计与训练流程详解

机器学习中的模型设计与训练流程详解

目录 前言1. 模型设计1.1 数据特性分析1.2 计算资源限制1.3 应用场景需求 2. 模型训练2.1 训练集与验证集的划分2.2 损失函数的选择2.3 模型参数更新 3. 优化方法3.1 梯度下降法3.2 正则化方法 4. 模型测试4.1 性能评估指标4.2 模型的泛化能力 5. 模型选择5.1 数据规模与模型复…
阅读更多...
怎么提取人声去掉背景音乐？人声提取秘籍：去掉背景音乐的技巧

怎么提取人声去掉背景音乐？人声提取秘籍：去掉背景音乐的技巧

在数字化时代&#xff0c;音频处理变得越来越普遍&#xff0c;我们经常需要从一段音频或视频中提取出纯净的人声&#xff0c;而去除掉背景音乐或其他杂音。这种需求在视频编辑、音乐制作、甚至在学习和娱乐中都十分常见。本文将介绍几种简单易行的方法&#xff0c;帮助你轻松提…
阅读更多...
【Spring】获取 Cookie和Session

【Spring】获取 Cookie和Session

回顾 Cookie HTTP 协议自身是属于“无状态”协议 无状态&#xff1a;默认情况下&#xff0c;HTTP 协议的客户端和服务器之间的这次通信和下次通信之间没有直接的联系 但是在实际开发中&#xff0c;我们很多时候是需要知道请求之间的关联关系的 例如登录网站成功后&#xff…
阅读更多...
抖音小游戏画图位置移动

抖音小游戏画图位置移动

文章目录 画图移动图形位置 画图 const canvas tt.createCanvas(); const context canvas.getContext(2d);context.width 500; context.height 500;let isPressing false; // 是否按下 let startX 0; let startY 0;context.fillStyle "#f00"; context.fillR…
阅读更多...
@zabbix监控网站黑链接监控及数据推送

@zabbix监控网站黑链接监控及数据推送

zabbix监控网站黑链接及数据推送 文章目录 zabbix监控网站黑链接及数据推送1.检测脚本1》编写python脚本2》脚本执行 2.数据推送1》方案一2》方案二 3.zabbix web 1.检测脚本 1》编写python脚本 创建脚本check_black_links.py&#xff0c;使用python脚本实现网站黑链接检测&a…
阅读更多...
93. 复原 IP 地址【回溯】

93. 复原 IP 地址【回溯】

文章目录 93. 复原 IP 地址解题思路Go代码 93. 复原 IP 地址 93. 复原 IP 地址 有效 IP 地址 正好由四个整数&#xff08;每个整数位于 0 到 255 之间组成&#xff0c;且不能含有前导 0&#xff09;&#xff0c;整数之间用 . 分隔。 例如&#xff1a;"0.1.2.201" …
阅读更多...
Datawhale组队学习|全球AI攻防挑战赛——赛道二：AI核身之金融场景凭证篡改检测

Datawhale组队学习|全球AI攻防挑战赛——赛道二：AI核身之金融场景凭证篡改检测

目录 前言Baseline代码解读 前言 Datawhale 2024.10 组队学习来了&#xff01;这次选择的是动手实践专区——CV方向——“全球AI攻防挑战赛—赛道二&#xff1a;AI核身之金融场景凭证篡改检测”。 Baseline代码解读 1、读取数据集 !apt update > /dev/null; apt install…
阅读更多...
美团测试面试真题学习

美团测试面试真题学习

美团真题1–测试基础-业务场景说下你的测试用例设计 功能角度 方法论 边界值、等价类划分、错误推测法示例 输入已注册的用户名和正确的密码&#xff0c;验证是否登录成功;输入已注册的用户名和不正确的密码&#xff0c;验证是否登录失败输入未注册的用户名和任意密码&#xff…
阅读更多...
Win10自带录屏神器？这4款工具让你秒变剪辑达人！

Win10自带录屏神器？这4款工具让你秒变剪辑达人！

小伙伴们&#xff0c;随着电子设备使用率越来越高&#xff0c;日常工作中我们需要进行一些操作的演示&#xff0c;或者是游戏中精彩的瞬间都希望录下来&#xff0c;那就少不了好用的录屏工具了。这次我来跟大家聊聊Windows 10自带的那些让人惊艳的录屏工具。这不仅仅是我个人推…
阅读更多...
Halcon 3D应用 - 胶路提取

Halcon 3D应用 - 胶路提取

1. 需求 本文基于某手环&#xff08;拆机打磨处理&#xff09;做的验证性工作&#xff0c;为了项目保密性&#xff0c;只截取部分数据进行测试。 这里使用的是海康3D线激光轮廓相机直线电机的方式进行的高度数据采集&#xff0c;我们拿到的是高度图亮度图数据。 提取手环上的胶…
阅读更多...
IBM Flex System服务器硬件监控指标解读

IBM Flex System服务器硬件监控指标解读

随着企业IT架构的日益复杂&#xff0c;服务器的稳定运行对于保障业务连续性至关重要。IBM Flex System作为一款模块化、可扩展的服务器解决方案&#xff0c;广泛应用于各种企业级环境中。为了确保IBM Flex System服务器的稳定运行&#xff0c;监控易作为一款专业的IT基础设施监…
阅读更多...
[Linux#65][TCP] 详解 延迟应答 | 捎带应答 | 流量控制 | 拥塞控制

[Linux#65][TCP] 详解 延迟应答 | 捎带应答 | 流量控制 | 拥塞控制

目录 一、延迟应答 二、捎带应答 三. 流量控制 总结 四. 拥塞控制 1. 拥塞控制 2. 慢启动机制&#xff1a; 3.思考 4.拥塞避免算法 5. 快速恢复算法 一、延迟应答 1. 立即应答问题 接收数据的主机若立刻返回ACK应答&#xff0c;可能返回的窗口较小。例如&#xff1…
阅读更多...
数字化转型新引擎：中小企业信息化建设的破局与未来-亿发

数字化转型新引擎：中小企业信息化建设的破局与未来-亿发

随着全球数字经济的蓬勃发展&#xff0c;信息化建设已成为企业提升竞争力、提高运营效率的重要途径。中小企业作为经济的重要组成部分&#xff0c;也逐步认识到信息化的重要性。然而&#xff0c;尽管中小企业在信息化方面有强烈的需求&#xff0c;但在实际推进过程中却面临诸多…
阅读更多...
简单粗暴理解GNN、GCN、GAT

简单粗暴理解GNN、GCN、GAT

GNN 思想&#xff1a;近朱者赤近墨者黑 GNN的流程&#xff1a; 聚合&#xff08;把邻居的信息贴到自己身上来&#xff0c;作为它自己特征的补足&#xff09;更新循环&#xff08;为什么要多次&#xff1f;看以下例子&#xff09; GNN能干嘛&#xff1f; 1.结点分类&#xf…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023