有一个在线文件(如http://www.example.com/information.asp),我需要抓取并保存到一个目录。我知道有几种逐行抓取和读取在线文件(url)的方法,但是否有一种方法可以使用Java下载并保存文件?


当前回答

下面是用Java代码从网上下载电影的示例代码:

URL url = new
URL("http://103.66.178.220/ftp/HDD2/Hindi%20Movies/2018/Hichki%202018.mkv");
    BufferedInputStream bufferedInputStream = new  BufferedInputStream(url.openStream());
    FileOutputStream stream = new FileOutputStream("/home/sachin/Desktop/test.mkv");

    int count = 0;
    byte[] b1 = new byte[100];

    while((count = bufferedInputStream.read(b1)) != -1) {
        System.out.println("b1:" + b1 + ">>" + count + ">> KB downloaded:" + new File("/home/sachin/Desktop/test.mkv").length()/1024);
        stream.write(b1, 0, count);
    }

其他回答

这个答案几乎和选中的答案完全一样,但是有两个增强:它是一个方法,它关闭了FileOutputStream对象:

    public static void downloadFileFromURL(String urlString, File destination) {
        try {
            URL website = new URL(urlString);
            ReadableByteChannel rbc;
            rbc = Channels.newChannel(website.openStream());
            FileOutputStream fos = new FileOutputStream(destination);
            fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
            fos.close();
            rbc.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
public void saveUrl(final String filename, final String urlString)
        throws MalformedURLException, IOException {
    BufferedInputStream in = null;
    FileOutputStream fout = null;
    try {
        in = new BufferedInputStream(new URL(urlString).openStream());
        fout = new FileOutputStream(filename);

        final byte data[] = new byte[1024];
        int count;
        while ((count = in.read(data, 0, 1024)) != -1) {
            fout.write(data, 0, count);
        }
    } finally {
        if (in != null) {
            in.close();
        }
        if (fout != null) {
            fout.close();
        }
    }
}

您将需要处理异常,可能是该方法的外部异常。

当使用Java 7+时,使用以下方法从Internet下载文件并将其保存到某个目录:

private static Path download(String sourceURL, String targetDirectory) throws IOException
{
    URL url = new URL(sourceURL);
    String fileName = sourceURL.substring(sourceURL.lastIndexOf('/') + 1, sourceURL.length());
    Path targetPath = new File(targetDirectory + File.separator + fileName).toPath();
    Files.copy(url.openStream(), targetPath, StandardCopyOption.REPLACE_EXISTING);

    return targetPath;
}

文档在这里。

在java.net.http.HttpClient上使用授权的解决方案:

HttpClient client = HttpClient.newHttpClient();

HttpRequest request = HttpRequest.newBuilder()
        .GET()
        .header("Accept", "application/json")
        // .header("Authorization", "Basic ci5raG9kemhhZXY6NDdiYdfjlmNUM=") if you need
        .uri(URI.create("https://jira.google.ru/secure/attachment/234096/screenshot-1.png"))
        .build();

HttpResponse<InputStream> response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());

try (InputStream in = response.body()) {
    Files.copy(in, Paths.get(target + "screenshot-1.png"), StandardCopyOption.REPLACE_EXISTING);
}

下面是一个简洁的、可读的、仅使用jdk的解决方案,其中包含适当的封闭资源:

static long download(String url, String fileName) throws IOException {
    try (InputStream in = URI.create(url).toURL().openStream()) {
        return Files.copy(in, Paths.get(fileName));
    }
}

两行代码,没有依赖关系。

下面是一个完整的文件下载示例程序,包含输出、错误检查和命令行参数检查:

package so.downloader;

import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;

public class Application {
    public static void main(String[] args) throws IOException {
        if (2 != args.length) {
            System.out.println("USAGE: java -jar so-downloader.jar <source-URL> <target-filename>");
            System.exit(1);
        }

        String sourceUrl = args[0];
        String targetFilename = args[1];

        long bytesDownloaded = download(sourceUrl, targetFilename);

        System.out.println(String.format("Downloaded %d bytes from %s to %s.", bytesDownloaded, sourceUrl, targetFilename));
    }

    static long download(String url, String fileName) throws IOException {
        try (InputStream in = URI.create(url).toURL().openStream()) {
            return Files.copy(in, Paths.get(fileName));
        }
    }    
}

正如so-downloader存储库README中所指出的:

运行文件下载程序:

java -jar so-downloader.jar <source-URL> <target-filename>

例如:

java -jar so-downloader.jar https://github.com/JanStureNielsen/so-downloader/archive/main.zip so-downloader-source.zip