在导入Excel的时候,如果文件比较大,行数很多,一行行读往往速度比较慢,为了加快导入速度,我们可以采用多线程的方式
话不多说直接上代码
首先是Controller
import com.sakura.base.service.ExcelService;
import com.sakura.common.api.ApiResult;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
@RestController
@RequestMapping("/excel")
public class ExcelController {
@Autowired
private ExcelService excelService;
@PostMapping("/import")
public ApiResult<Boolean> importExcel(@ModelAttribute MultipartFile file) throws Exception {
boolean flag = excelService.importExcel(file);
return ApiResult.result(flag);
}
}
然后Service
import org.springframework.web.multipart.MultipartFile;
public interface ExcelService {
boolean importExcel(MultipartFile file) throws Exception;
}
ServiceImpl
import com.sakura.base.entity.User;
import com.sakura.base.mapper.UserMapper;
import com.sakura.base.service.ExcelService;
import lombok.extern.java.Log;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@Service
@Log
public class ExcelServiceImpl implements ExcelService {
@Autowired
private UserMapper userMapper;
@Override
public boolean importExcel(MultipartFile file) throws Exception {
// 文件为空这些校验大家自己加,这里只是做一个示例
// 每次处理的数据量,大家可以自己调整,比如每次处理1000条
int batchSize = 5;
// 最大线程数,大家可以自己调整,根据自己服务器性能来调整
int maxThreads = 3;
// 创建一个线程池并设置最大线程数
ExecutorService executorService = Executors.newFixedThreadPool(maxThreads);
Workbook workbook = null;
try {
workbook = new XSSFWorkbook(file.getInputStream());
// 获取第一页
Sheet sheet = workbook.getSheetAt(0);
// 获取总行数
int rowCount = sheet.getLastRowNum();
// 第0行一般为表头,从第一行开始
int startRow = 1;
// 结束行,Math.min用来比较两个数的大小,取最小值
int endRow = Math.min(batchSize, rowCount);
while (startRow <= rowCount) {
// 提交任务到线程池
executorService.execute(new ExcelRowProcessorTask(sheet, startRow, endRow));
// 下一批数据的起始行
startRow = endRow + 1;
// 下一批数据的结束行
endRow = Math.min(startRow + batchSize - 1, rowCount);
}
// 关闭线程池
executorService.shutdown();
} catch (Exception e) {
e.printStackTrace();
} finally {
// 关闭流
if (workbook != null) {
try {
workbook.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return true;
}
private class ExcelRowProcessorTask implements Runnable {
private final Sheet sheet;
private final int startRow;
private final int endRow;
public ExcelRowProcessorTask(Sheet sheet, int startRow, int endRow) {
this.sheet = sheet;
this.startRow = startRow;
this.endRow = endRow;
}
@Override
public void run() {
// _________________________________________
// 测试用,模拟处理数据的耗时,实际应用删除
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
// _________________________________________
for (int i = startRow; i <= endRow; i++) {
Row row = sheet.getRow(i);
if (row != null) {
Cell nameCell = row.getCell(0); // 第一列
Cell phoneCell = row.getCell(1); // 第二列
nameCell.setCellType(CellType.STRING);
String name = nameCell.getStringCellValue();
phoneCell.setCellType(CellType.STRING);
String phoneNumber = phoneCell.getStringCellValue();
User user = new User();
user.setName(name);
user.setPhoneNumber(phoneNumber);
userMapper.insert(user);
}
}
}
}
}
实体类User就不贴了没啥好说的
还有就是poi的jar包
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>3.15</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>3.15</version>
</dependency>
用postman验证上面的代码
可以看下数据库的数据,因为我限制了每次处理的数据为5条,同时最多有3个线程,所以可以看到同一时间段导进去的数据为15条
上面这个还有一个问题就是主线程不会等数据导入完就会返回,如果你需要主线程等待数据导入完可以加上下面这行代码
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS); // 等待所有任务执行完毕 Long.MAX_VALUE为超时时间,可以自由设置
就放在关闭线程池后面就可以了
有想看下怎么用多线程导出Excel的移步 Java多线程导出Excel示例