我有一个应用程序,读取一个CSV文件与成堆的数据行。我根据数据类型向用户提供了行数的摘要,但我想确保不会读入太多行数据而导致OutOfMemoryErrors。每一行转换成一个对象。有没有一种简单的方法以编程方式找出该对象的大小?是否有一个引用定义了一个VM的基本类型和对象引用有多大?

现在,我有代码说读取多达32,000行,但我还希望有代码说读取尽可能多的行,直到我使用了32MB的内存。也许这是另一个问题,但我还是想知道。


当前回答

我偶然发现了一个java类 "jdk.nashorn.internal.ir.debug. objectsizecalculator ",已经在jdk中, 这很容易使用,似乎对确定物体的大小非常有用。

System.out.println(ObjectSizeCalculator.getObjectSize(new gnu.trove.map.hash.TObjectIntHashMap<String>(12000, 0.6f, -1)));
System.out.println(ObjectSizeCalculator.getObjectSize(new HashMap<String, Integer>(100000)));
System.out.println(ObjectSizeCalculator.getObjectSize(3));
System.out.println(ObjectSizeCalculator.getObjectSize(new int[]{1, 2, 3, 4, 5, 6, 7 }));
System.out.println(ObjectSizeCalculator.getObjectSize(new int[100]));

结果:

164192
48
16
48
416

其他回答

可以使用java.lang.instrument包。

编译并将这个类放入JAR:

import java.lang.instrument.Instrumentation;

public class ObjectSizeFetcher {
    private static Instrumentation instrumentation;

    public static void premain(String args, Instrumentation inst) {
        instrumentation = inst;
    }

    public static long getObjectSize(Object o) {
        return instrumentation.getObjectSize(o);
    }
}

将以下内容添加到您的清单中。MF:

Premain-Class: ObjectSizeFetcher

使用getObjectSize()方法:

public class C {
    private int x;
    private int y;

    public static void main(String [] args) {
        System.out.println(ObjectSizeFetcher.getObjectSize(new C()));
    }
}

调用:

java -javaagent:ObjectSizeFetcherAgent.jar C

我怀疑您是否希望以编程方式完成它,除非您只是想执行一次并将其存储起来以供将来使用。这是一件代价高昂的事情。在Java中没有sizeof()操作符,即使有,它也只会计算引用其他对象的代价和原语的大小。

你可以这样做的一种方法是将它序列化到File中,然后查看文件的大小,就像这样:

Serializable myObject;
ObjectOutputStream oos = new ObjectOutputStream (new FileOutputStream ("obj.ser"));
oos.write (myObject);
oos.close ();

当然,这假设每个对象都是不同的,并且不包含对其他任何对象的非瞬时引用。

另一种策略是获取每个对象并通过反射检查其成员,并将大小相加(boolean & byte = 1字节,short & char = 2字节,等等),沿着成员层次结构向下工作。但这既乏味又昂贵,而且最终与序列化策略所做的事情相同。

您可以生成一个堆转储(例如,使用jmap),然后分析输出以查找对象大小。这是一种离线解决方案,但是您可以检查浅尺寸和深尺寸等。

许多其他答案提供了浅的大小——例如,没有任何键或值的HashMap的大小,这可能不是您想要的。

jamm项目使用上面的java.lang.instrumentation包,但是遍历树,因此可以为您提供深层内存使用。

new MemoryMeter().measureDeep(myHashMap);

https://github.com/jbellis/jamm

要使用MemoryMeter,请使用“-javaagent:/jam .jar”启动JVM

下面是我使用一些链接示例制作的实用程序,用于处理32位、64位和64位压缩OOP。它使用sun.misc.不安全。

它使用Unsafe. addresssize()来获取本机指针的大小和Unsafe。arrayIndexScale(Object[].class)表示Java引用的大小。

它使用已知类的字段偏移量来计算对象的基大小。

import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.IdentityHashMap;
import java.util.Stack;
import sun.misc.Unsafe;

/** Usage: 
 * MemoryUtil.sizeOf( object )
 * MemoryUtil.deepSizeOf( object )
 * MemoryUtil.ADDRESS_MODE
 */
public class MemoryUtil
{
    private MemoryUtil()
    {
    }

    public static enum AddressMode
    {
        /** Unknown address mode. Size calculations may be unreliable. */
        UNKNOWN,
        /** 32-bit address mode using 32-bit references. */
        MEM_32BIT,
        /** 64-bit address mode using 64-bit references. */
        MEM_64BIT,
        /** 64-bit address mode using 32-bit compressed references. */
        MEM_64BIT_COMPRESSED_OOPS
    }

    /** The detected runtime address mode. */
    public static final AddressMode ADDRESS_MODE;

    private static final Unsafe UNSAFE;

    private static final long ADDRESS_SIZE; // The size in bytes of a native pointer: 4 for 32 bit, 8 for 64 bit
    private static final long REFERENCE_SIZE; // The size of a Java reference: 4 for 32 bit, 4 for 64 bit compressed oops, 8 for 64 bit
    private static final long OBJECT_BASE_SIZE; // The minimum size of an Object: 8 for 32 bit, 12 for 64 bit compressed oops, 16 for 64 bit
    private static final long OBJECT_ALIGNMENT = 8;

    /** Use the offset of a known field to determine the minimum size of an object. */
    private static final Object HELPER_OBJECT = new Object() { byte b; };


    static
    {
        try
        {
            // Use reflection to get a reference to the 'Unsafe' object.
            Field f = Unsafe.class.getDeclaredField( "theUnsafe" );
            f.setAccessible( true );
            UNSAFE = (Unsafe) f.get( null );

            OBJECT_BASE_SIZE = UNSAFE.objectFieldOffset( HELPER_OBJECT.getClass().getDeclaredField( "b" ) );

            ADDRESS_SIZE = UNSAFE.addressSize();
            REFERENCE_SIZE = UNSAFE.arrayIndexScale( Object[].class );

            if( ADDRESS_SIZE == 4 )
            {
                ADDRESS_MODE = AddressMode.MEM_32BIT;
            }
            else if( ADDRESS_SIZE == 8 && REFERENCE_SIZE == 8 )
            {
                ADDRESS_MODE = AddressMode.MEM_64BIT;
            }
            else if( ADDRESS_SIZE == 8 && REFERENCE_SIZE == 4 )
            {
                ADDRESS_MODE = AddressMode.MEM_64BIT_COMPRESSED_OOPS;
            }
            else
            {
                ADDRESS_MODE = AddressMode.UNKNOWN;
            }
        }
        catch( Exception e )
        {
            throw new Error( e );
        }
    }


    /** Return the size of the object excluding any referenced objects. */
    public static long shallowSizeOf( final Object object )
    {
        Class<?> objectClass = object.getClass();
        if( objectClass.isArray() )
        {
            // Array size is base offset + length * element size
            long size = UNSAFE.arrayBaseOffset( objectClass )
                    + UNSAFE.arrayIndexScale( objectClass ) * Array.getLength( object );
            return padSize( size );
        }
        else
        {
            // Object size is the largest field offset padded out to 8 bytes
            long size = OBJECT_BASE_SIZE;
            do
            {
                for( Field field : objectClass.getDeclaredFields() )
                {
                    if( (field.getModifiers() & Modifier.STATIC) == 0 )
                    {
                        long offset = UNSAFE.objectFieldOffset( field );
                        if( offset >= size )
                        {
                            size = offset + 1; // Field size is between 1 and PAD_SIZE bytes. Padding will round up to padding size.
                        }
                    }
                }
                objectClass = objectClass.getSuperclass();
            }
            while( objectClass != null );

            return padSize( size );
        }
    }


    private static final long padSize( final long size )
    {
        return (size + (OBJECT_ALIGNMENT - 1)) & ~(OBJECT_ALIGNMENT - 1);
    }


    /** Return the size of the object including any referenced objects. */
    public static long deepSizeOf( final Object object )
    {
        IdentityHashMap<Object,Object> visited = new IdentityHashMap<Object,Object>();
        Stack<Object> stack = new Stack<Object>();
        if( object != null ) stack.push( object );

        long size = 0;
        while( !stack.isEmpty() )
        {
            size += internalSizeOf( stack.pop(), stack, visited );
        }
        return size;
    }


    private static long internalSizeOf( final Object object, final Stack<Object> stack, final IdentityHashMap<Object,Object> visited )
    {
        // Scan for object references and add to stack
        Class<?> c = object.getClass();
        if( c.isArray() && !c.getComponentType().isPrimitive() )
        {
            // Add unseen array elements to stack
            for( int i = Array.getLength( object ) - 1; i >= 0; i-- )
            {
                Object val = Array.get( object, i );
                if( val != null && visited.put( val, val ) == null )
                {
                    stack.add( val );
                }
            }
        }
        else
        {
            // Add unseen object references to the stack
            for( ; c != null; c = c.getSuperclass() )
            {
                for( Field field : c.getDeclaredFields() )
                {
                    if( (field.getModifiers() & Modifier.STATIC) == 0 
                            && !field.getType().isPrimitive() )
                    {
                        field.setAccessible( true );
                        try
                        {
                            Object val = field.get( object );
                            if( val != null && visited.put( val, val ) == null )
                            {
                                stack.add( val );
                            }
                        }
                        catch( IllegalArgumentException e )
                        {
                            throw new RuntimeException( e );
                        }
                        catch( IllegalAccessException e )
                        {
                            throw new RuntimeException( e );
                        }
                    }
                }
            }
        }

        return shallowSizeOf( object );
    }
}