如何将整个输入流读到字节数组?


当前回答

你可以完全避免缓冲。

从http://www.exampledepot.com/egs/java.io/File2ByteArray.html复制的代码(是的,它非常冗长,但需要的内存大小是其他解决方案的一半)。

// Returns the contents of the file in a byte array.
public static byte[] getBytesFromFile(File file) throws IOException {
    InputStream is = new FileInputStream(file);

    // Get the size of the file
    long length = file.length();

    // You cannot create an array using a long type.
    // It needs to be an int type.
    // Before converting to an int type, check
    // to ensure that file is not larger than Integer.MAX_VALUE.
    if (length > Integer.MAX_VALUE) {
        // File is too large
    }

    // Create the byte array to hold the data
    byte[] bytes = new byte[(int)length];

    // Read in the bytes
    int offset = 0;
    int numRead = 0;
    while (offset < bytes.length
           && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
        offset += numRead;
    }

    // Ensure all the bytes have been read in
    if (offset < bytes.length) {
        throw new IOException("Could not completely read file "+file.getName());
    }

    // Close the input stream and return bytes
    is.close();
    return bytes;
}

其他回答

如果您不想使用Apache common -io库,则此代码片段取自sun.misc.IOUtils类。它的速度几乎是使用ByteBuffers的普通实现的两倍:

public static byte[] readFully(InputStream is, int length, boolean readAll)
        throws IOException {
    byte[] output = {};
    if (length == -1) length = Integer.MAX_VALUE;
    int pos = 0;
    while (pos < length) {
        int bytesToRead;
        if (pos >= output.length) { // Only expand when there's no room
            bytesToRead = Math.min(length - pos, output.length + 1024);
            if (output.length < pos + bytesToRead) {
                output = Arrays.copyOf(output, pos + bytesToRead);
            }
        } else {
            bytesToRead = output.length - pos;
        }
        int cc = is.read(output, pos, bytesToRead);
        if (cc < 0) {
            if (readAll && length != Integer.MAX_VALUE) {
                throw new EOFException("Detect premature EOF");
            } else {
                if (output.length != pos) {
                    output = Arrays.copyOf(output, pos);
                }
                break;
            }
        }
        pos += cc;
    }
    return output;
}

我知道已经太迟了,但我认为这里有更清晰的解决方案,更易于阅读……

/**
 * method converts {@link InputStream} Object into byte[] array.
 * 
 * @param stream the {@link InputStream} Object.
 * @return the byte[] array representation of received {@link InputStream} Object.
 * @throws IOException if an error occurs.
 */
public static byte[] streamToByteArray(InputStream stream) throws IOException {

    byte[] buffer = new byte[1024];
    ByteArrayOutputStream os = new ByteArrayOutputStream();

    int line = 0;
    // read bytes from stream, and store them in buffer
    while ((line = stream.read(buffer)) != -1) {
        // Writes bytes from byte array (buffer) into output stream.
        os.write(buffer, 0, line);
    }
    stream.close();
    os.flush();
    os.close();
    return os.toByteArray();
}

在将S3对象转换为ByteArray时,我们看到一些AWS事务的延迟。

注意:S3对象为PDF文档(最大大小为3mb)。

我们使用选项#1将S3对象转换为ByteArray。我们注意到S3提供了内置IOUtils方法来将S3对象转换为ByteArray,我们请求您确认将S3对象转换为ByteArray的最佳方法以避免延迟。

选项1:

import org.apache.commons.io.IOUtils;
is = s3object.getObjectContent();
content =IOUtils.toByteArray(is);

选项2:

import com.amazonaws.util.IOUtils;
is = s3object.getObjectContent();
content =IOUtils.toByteArray(is);

也让我知道,如果我们有任何其他更好的方法来转换s3对象到bytearray

Kotlin中的解决方案(当然也可以在Java中工作),其中包括当你知道大小时的两种情况:

    fun InputStream.readBytesWithSize(size: Long): ByteArray? {
        return when {
            size < 0L -> this.readBytes()
            size == 0L -> ByteArray(0)
            size > Int.MAX_VALUE -> null
            else -> {
                val sizeInt = size.toInt()
                val result = ByteArray(sizeInt)
                readBytesIntoByteArray(result, sizeInt)
                result
            }
        }
    }

    fun InputStream.readBytesIntoByteArray(byteArray: ByteArray,bytesToRead:Int=byteArray.size) {
        var offset = 0
        while (true) {
            val read = this.read(byteArray, offset, bytesToRead - offset)
            if (read == -1)
                break
            offset += read
            if (offset >= bytesToRead)
                break
        }
    }

如果您知道大小,那么与其他解决方案相比,它可以节省两倍的内存(在很短的时间内,但仍然可能有用)。这是因为您必须将整个流读到末尾,然后将其转换为字节数组(类似于将数组转换为数组的ArrayList)。

例如,如果你在Android上,你有一些Uri要处理,你可以尝试用这个来获取大小:

    fun getStreamLengthFromUri(context: Context, uri: Uri): Long {
        context.contentResolver.query(uri, arrayOf(MediaStore.MediaColumns.SIZE), null, null, null)?.use {
            if (!it.moveToNext())
                return@use
            val fileSize = it.getLong(it.getColumnIndex(MediaStore.MediaColumns.SIZE))
            if (fileSize > 0)
                return fileSize
        }
        //if you wish, you can also get the file-path from the uri here, and then try to get its size, using this: https://stackoverflow.com/a/61835665/878126
        FileUtilEx.getFilePathFromUri(context, uri, false)?.use {
            val file = it.file
            val fileSize = file.length()
            if (fileSize > 0)
                return fileSize
        }
        context.contentResolver.openInputStream(uri)?.use { inputStream ->
            if (inputStream is FileInputStream)
                return inputStream.channel.size()
            else {
                var bytesCount = 0L
                while (true) {
                    val available = inputStream.available()
                    if (available == 0)
                        break
                    val skip = inputStream.skip(available.toLong())
                    if (skip < 0)
                        break
                    bytesCount += skip
                }
                if (bytesCount > 0L)
                    return bytesCount
            }
        }
        return -1L
    }

我用这个。

public static byte[] toByteArray(InputStream is) throws IOException {
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        try {
            byte[] b = new byte[4096];
            int n = 0;
            while ((n = is.read(b)) != -1) {
                output.write(b, 0, n);
            }
            return output.toByteArray();
        } finally {
            output.close();
        }
    }