公众号：赵侠客
原文：https://mp.weixin.qq.com/s/i49mqqZ9km22gk2A6umaQQ
一、前言
下载文件在我们项目很常见，有下载视频、文件、图片、附件、导出Excel、导出Zip压缩文件等等，这里我对常见的下载做个简单的总结，主要有文件下载、限速下载、多文件打包下载、URL文件打包下载、Excel导出下载、Excel批量导出Zip包下载、多线程加速下载。
二、搭建Spring Boot项目
搭建个SpringBoot Web项目，引用常用依赖，commons-io作常用IO操作，hutool-all、poi-ooxml做导出Excel操作，commons-compress做多线程压缩。
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.21</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>4.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.20</version>
</dependency>三、文件下载
3.1 单文件下载
最简单的下载就是提供一个文件下载接口，浏览器请求接口后预览或者下载文件，这里以下载一个1.2G的视频为例，直接看 /download接口
java">@GetMapping("/download")
public void download(HttpServletResponse response) throws IOException {
File file = new File("/Users/zxk/Movies/1.2G.mp4");
response.setContentType("video/mp4;charset=utf8");
//设置下载文件名
response.setHeader("Content-Disposition", "attachment;filename=" + file.getName());
//中文乱码处理
//response.setHeader("Content-Disposition", "attachment;filename=" + URLEncoder.encode(file.getName(), "UTF-8") );
//网页直接播放
//response.setHeader("Content-Disposition", "inline");
//下载进度
response.setContentLengthLong(file.length());
try (InputStream inputStream = new FileInputStream(file);
OutputStream outputStream = response.getOutputStream()
) {
IOUtils.copy(inputStream, outputStream);
}
}
这里有以下几点需要注意：
-
1. response.setContentType设置文件的类型
-
2. Content-Disposition设置文件下载时显示的文件名，如果有中文乱码，需要URLEncode，如果希望浏览器直接打开可以设置"inline"
-
3. response.setContentLengthLong(file.length())，设置Http body长度可以在下载时显示进度
-
4. 下载完成需要关闭流，这里使用try-with-resource自动关闭流
3.2限速下载
使用第一种下载速度会非常快，可能瞬间就将你的服务器带宽占满了，所以就需要限制下载速度。某盘开会员和不开会员下载速度相差非常大，就是针对不用同步给限制了不同的下载速度
java">@GetMapping("/limitSpeed")
public void limitSpeed(@RequestParam(value = "speed", defaultValue = "1024") int speed, HttpServletResponse response) throws IOException {
File path = new File("/Users/zxk/Movies/1.2G.mp4");
response.setContentType("video/mp4;charset=utf8");
response.setHeader("Content-Disposition", "attachment;filename=" + path.getName());
response.setContentLengthLong(path.length());
try (
InputStream inputStream = new FileInputStream(path);
OutputStream outputStream = response.getOutputStream()
) {
byte[] buffer = new byte[1024];
int length;
SpeedLimiter speedLimiter = new SpeedLimiter(speed);
while ((length = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, length);
speedLimiter.delayNextBytes(length);
}
}
}
public class SpeedLimiter {
/** 速度上限(KB/s), 0=不限速 */
private int maxRate = 1024;
private long getMaxRateBytes(){
return this.maxRate * 1024L;
}
private long getLessCountBytes() {
long lcb = getMaxRateBytes() / 10;
if (lcb < 10240) lcb = 10240;
return lcb;
}
public SpeedLimiter(int maxRate) {
this.setMaxRate(maxRate);
}
public synchronized void setMaxRate(int maxRate){
this.maxRate = Math.max(maxRate, 0);
}
private long totalBytes = 0;
private long tmpCountBytes = 0;
private final long lastTime = System.currentTimeMillis();
public synchronized void delayNextBytes(int len) {
if (maxRate <= 0) return;
totalBytes += len;
tmpCountBytes += len;
//未达到指定字节数跳过...
if (tmpCountBytes < getLessCountBytes()) {
return;
}
long nowTime = System.currentTimeMillis();
long sendTime = nowTime - lastTime;
long workTime = (totalBytes * 1000) / getMaxRateBytes();
long delayTime = workTime - sendTime;
if (delayTime > 0) {
try {
Thread.sleep(delayTime);
} catch (InterruptedException e) {
e.printStackTrace();
}
tmpCountBytes = 0;
}
}
}3.3多文件打成ZIP包下载
有了单文件下载，肯定就用多文件下载，一般浏览器下载多个文件是将多个文件打包成一个Zip文件下载。
java">@GetMapping("/zip")
public void zip(HttpServletResponse response) throws IOException {
File file1 = new File("/Users/zxk/Movies/2.mp4");
File file2 = new File("/Users/zxk/Movies/2.mp4");
List<File> files = Arrays.asList(file2, file1);
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
try (ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream())) {
zipOutputStream.setLevel(0);
files.forEach(f -> {
try (FileInputStream inputStream = new FileInputStream(f)) {
zipOutputStream.putNextEntry(new ZipEntry(f.getName()));
IOUtils.copy(inputStream, zipOutputStream);
zipOutputStream.closeEntry();
} catch (Exception e) {
e.printStackTrace();
}
});
zipOutputStream.flush();
zipOutputStream.finish();
}
}多文件打成Zip包注意事项：
-
1. zipOutputStream.setLevel(0)设置压缩等级，0为不压缩，这样可以提升下载速度
-
2. 多个文件打包下载时并不是所有文件压缩完成后才开始下载，而是边压缩边下载，这样用户点了下载后，下载任务会直接进入浏览器下载列表
3.4整个文件夹下载
有时需要递归将整个文件夹打包下载下来
java"> @GetMapping("/dir")
public void dir(HttpServletResponse response) throws IOException {
File dir = new File("/Users/zxk/Movies/download");
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
try (ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream())) {
zipOutputStream.setLevel(0);
zip(zipOutputStream, "", dir);
zipOutputStream.flush();
zipOutputStream.finish();
}
}
public void zip(ZipOutputStream zipOutputStream, String parentPath, File file) throws IOException {
if (file.isDirectory()) {
File[] subFiles = file.listFiles();
if (subFiles != null) {
for (File f : subFiles) {
zip(zipOutputStream, parentPath + file.getName() + "/", f);
}
}
} else {
try (FileInputStream fileInputStream = new FileInputStream(file)) {
zipOutputStream.putNextEntry(new ZipEntry(parentPath + file.getName()));
IOUtils.copy(fileInputStream, zipOutputStream);
}
}
}注意事项：
-
1. 会递归整个文件夹，文件夹文件不能太多
3.5通过URL打包下载
有时我们的文件可能是存在云存储上，数据库里存的是文件的ULR，下载时我们就需要通过URL将多个文件打包下载
java"> @GetMapping("/urlZip")
public void urlZip(HttpServletResponse response) throws IOException {
List<String> urls = Arrays.asList("https://demo.com/11666832527556.jpeg",
"https://demo.com/11666831385156.jpeg",
"https://demo.com/11666829917700.jpeg",
"https://demo.com/11666762702021.png",
"https://demo.com/11666762702020.webp",
"https://demo.com/11666549651972.jpg",
"https://demo.com/11666524497476.jpeg",
"https://demo.com/11666507113092.jpg");
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
try (ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream())) {
zipOutputStream.setLevel(0);
urls.forEach(f -> {
try {
zipOutputStream.putNextEntry(new ZipEntry(StringUtils.getFilename(f)));
HttpUtil.download(f, zipOutputStream, false);
zipOutputStream.closeEntry();
} catch (Exception e) {
e.printStackTrace();
}
});
zipOutputStream.flush();
zipOutputStream.finish();
}
}3.6导出Excel下载
有些下载的文件并不存在，而是先从数据库中查出数据，再动态生成文件，再提供给用户下载，这里我们以导出单个Excel文件为例：
java">@GetMapping("/excel")
public void excel(HttpServletResponse response) throws IOException {
List<List<Integer>> rows = new ArrayList<>();
for (int i = 0; i < 10000; i++) {
rows.add(IntStream.range(i, i + 100).boxed().collect(Collectors.toList()));
}
response.setContentType("application/vnd.ms-excel;charset=utf-8");
response.setHeader("Content-Disposition", "attachment;filename=test.xls");
try (OutputStream out = response.getOutputStream();
ExcelWriter writer = ExcelUtil.getWriter()) {
writer.write(rows);
writer.flush(out, true);
}
}3.7批量导出Excel打包下载
很多业务需要一次性导出多个Excel，这里我们可以将多个Excel压缩成一个Zip文件下载下来，这里以动态生成10个Excel主例：
java"> @GetMapping("/excelZip")
public void excelZip(HttpServletResponse response) throws IOException {
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
try (ZipOutputStream zipOutputStream = new ZipOutputStream(response.getOutputStream())) {
zipOutputStream.setLevel(0);
for (int i = 0; i < 10; i++) {
zipOutputStream.putNextEntry(new ZipEntry(String.format("%s.xls", i)));
try (ExcelWriter writer = ExcelUtil.getWriter()) {
writer.write(generateData());
writer.flush(zipOutputStream);
}
}
zipOutputStream.flush();
zipOutputStream.finish();
}
}
private List<List<Integer>> generateData() {
List<List<Integer>> rows = new ArrayList<>();
for (int i = 0; i < 10000; i++) {
rows.add(IntStream.range(i, i + 100).boxed().collect(Collectors.toList()));
}
return rows;
}3.8多线程加速下载
有时下载数据量比较多，单线程打包会比较慢，这里我们就需要使用多线程并发打包提高打包速度，这里我以多线程下载多个URL文件为例使用commons-compress的ParallelScatterZipCreator多线程并发打包
java">public static final ThreadFactory factory = new ThreadFactoryBuilder().setNamePrefix("compressFileList-pool-").build();
public static final ExecutorService executor = new ThreadPoolExecutor(10, 10, 60, TimeUnit.SECONDS, new LinkedBlockingQueue<>(20), factory);
@GetMapping("/parallelZip")
public void excelZipThread(HttpServletResponse response) throws IOException {
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
List<String> urls = Arrays.asList("https://demo.com/11671291835144.png",
"https://demo.com/11671291834824.png",
"https://demo.com/11671291833928.png",
"https://demo.com/11671291833800.png",
"https://demo.com/11671291833480.png",
"https://demo.com/11671291828232.png",
"https://demo.com/11671291827528.png",
"https://demo.com/11671291825737.png",
"https://demo.com/11671291825736.png");
ParallelScatterZipCreator parallelScatterZipCreator = new ParallelScatterZipCreator(executor);
try (ZipArchiveOutputStream zipArchiveOutputStream = new ZipArchiveOutputStream(response.getOutputStream())) {
zipArchiveOutputStream.setLevel(0);
urls.forEach(x -> {
ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry(StringUtils.getFilename(x));
zipArchiveEntry.setMethod(ZipArchiveEntry.STORED);
InputStreamSupplier inputStreamSupplier = () -> URLUtil.getStream(URLUtil.url(x));
parallelScatterZipCreator.addArchiveEntry(zipArchiveEntry, inputStreamSupplier);
});
parallelScatterZipCreator.writeTo(zipArchiveOutputStream);
} catch (Exception e) {
e.printStackTrace();
}
}3.9多线程批量导Excel打包下载
这种是比较复杂的，动态生成多个Excel文件后，使用多线程打成ZIP包下载
java"> @GetMapping("/parallelexcelZip")
public void parallelexcelZip(HttpServletResponse response) throws IOException {
response.setContentType("application/zip");
response.setHeader("Content-Disposition", "attachment;filename=demo.zip");
ParallelScatterZipCreator parallelScatterZipCreator = new ParallelScatterZipCreator(executor);
try (ZipArchiveOutputStream zipArchiveOutputStream = new ZipArchiveOutputStream(response.getOutputStream())) {
zipArchiveOutputStream.setLevel(0);
IntStream.range(1,10).forEach(x -> {
InputStreamSupplier inputStreamSupplier = () ->{
ByteArrayOutputStream outputStream=new ByteArrayOutputStream();
try(ExcelWriter writer=ExcelUtil.getWriter()) {
writer.write(generateData());
writer.flush(outputStream);
}
return new ByteArrayInputStream(outputStream.toByteArray());
};
ZipArchiveEntry zipArchiveEntry = new ZipArchiveEntry(String.format("%s.xls",x));
zipArchiveEntry.setMethod(ZipArchiveEntry.STORED);
parallelScatterZipCreator.addArchiveEntry(zipArchiveEntry, inputStreamSupplier);
});
parallelScatterZipCreator.writeTo(zipArchiveOutputStream);
} catch (Exception e) {
e.printStackTrace();
}
}
四、总结
本文主要总结了常用9种常见的文件下载操作，并提供对应的演示代码，当然还有一些没有总结到的，如分片下载、断点结续下、分布式下载限速等，这些高级下载在普通项目中用到的不太多所以没有总结。
