我正在从事一个涉及大量数据库写入的项目(70%的插入和30%的读取)。这个比率还包括我认为是一个读一个写的更新。读取可能是脏的(例如,在读取时我不需要100%准确的信息)。 该任务每小时将处理超过100万个数据库事务。

我在网上读了一堆关于MyISAM和InnoDB之间区别的东西,对于我将用于这个任务的特定数据库/表来说,MyISAM似乎是显而易见的选择。从我看来,InnoDB在需要事务时是很好的,因为它支持行级锁。

有人有这种负载(或更高)的经验吗?MyISAM是正确的选择吗?


当前回答

如果它是70%的插入和30%的读取,那么它更像是在InnoDB端。

其他回答

简而言之,如果你的工作需要一个可靠的数据库来处理大量的INSERT和UPDATE指令,InnoDB是很好的选择。

而且,如果你需要的数据库需要大量的读(SELECT)指令而不是写(INSERT和UPDATES), MyISAM是很好的,考虑到它在表锁方面的缺点。

你可能想去看看; InnoDB的优缺点 MyISAM的利弊

对于这样的读写比率,我猜InnoDB会表现得更好。 既然您可以接受脏读,那么您可以(如果您负担得起)复制到一个从服务器,并让您的所有读都到从服务器。另外,考虑批量插入,而不是一次插入一条记录。

为了增加广泛的选择,这里涵盖了两个发动机之间的机械差异,我提出了一个经验速度比较研究。

就纯粹的速度而言,MyISAM并不总是比InnoDB快,但根据我的经验,在pure READ工作环境中,MyISAM往往快2.0-2.5倍。显然,这并不适用于所有环境——正如其他人所写的那样,MyISAM缺少事务和外键之类的东西。

我在下面做了一些基准测试——我使用python进行循环,使用timeit库进行时间比较。出于兴趣,我还包括了内存引擎,这提供了最好的性能,尽管它只适用于较小的表(当您超过MySQL内存限制时,您会不断遇到表'tbl'已满)。我研究的四种选择类型是:

香草选择 计数 有条件的选择 索引和非索引子选择

首先,我使用以下SQL创建了三个表

CREATE TABLE
    data_interrogation.test_table_myisam
    (
        index_col BIGINT NOT NULL AUTO_INCREMENT,
        value1 DOUBLE,
        value2 DOUBLE,
        value3 DOUBLE,
        value4 DOUBLE,
        PRIMARY KEY (index_col)
    )
    ENGINE=MyISAM DEFAULT CHARSET=utf8

在第二和第三个表中用“MyISAM”替换“InnoDB”和“memory”。

 

1)香草选择

查询:SELECT * FROM tbl WHERE index_col = xx

结果:画

它们的速度基本上是相同的,并且正如预期的那样,与要选择的列数成线性关系。InnoDB似乎比MyISAM快一点,但这真的是微不足道的。

代码:

import timeit
import MySQLdb
import MySQLdb.cursors
import random
from random import randint

db = MySQLdb.connect(host="...", user="...", passwd="...", db="...", cursorclass=MySQLdb.cursors.DictCursor)
cur = db.cursor()

lengthOfTable = 100000

# Fill up the tables with random data
for x in xrange(lengthOfTable):
    rand1 = random.random()
    rand2 = random.random()
    rand3 = random.random()
    rand4 = random.random()

    insertString = "INSERT INTO test_table_innodb (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
    insertString2 = "INSERT INTO test_table_myisam (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
    insertString3 = "INSERT INTO test_table_memory (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"

    cur.execute(insertString)
    cur.execute(insertString2)
    cur.execute(insertString3)

db.commit()

# Define a function to pull a certain number of records from these tables
def selectRandomRecords(testTable,numberOfRecords):

    for x in xrange(numberOfRecords):
        rand1 = randint(0,lengthOfTable)

        selectString = "SELECT * FROM " + testTable + " WHERE index_col = " + str(rand1)
        cur.execute(selectString)

setupString = "from __main__ import selectRandomRecords"

# Test time taken using timeit
myisam_times = []
innodb_times = []
memory_times = []

for theLength in [3,10,30,100,300,1000,3000,10000]:

    innodb_times.append( timeit.timeit('selectRandomRecords("test_table_innodb",' + str(theLength) + ')', number=100, setup=setupString) )
    myisam_times.append( timeit.timeit('selectRandomRecords("test_table_myisam",' + str(theLength) + ')', number=100, setup=setupString) )
    memory_times.append( timeit.timeit('selectRandomRecords("test_table_memory",' + str(theLength) + ')', number=100, setup=setupString) )

 

2)计算

查询:SELECT count(*) FROM tbl

结果:MyISAM获胜

这个说明了MyISAM和InnoDB之间的一个很大的不同——MyISAM(和内存)跟踪表中的记录数量,所以这个事务是快速的,O(1)。在我调查的范围内,InnoDB计数所需的时间随着表的大小超线性增加。我怀疑在实践中观察到的许多MyISAM查询的加速都是由于类似的效果。

代码:

myisam_times = []
innodb_times = []
memory_times = []

# Define a function to count the records
def countRecords(testTable):

    selectString = "SELECT count(*) FROM " + testTable
    cur.execute(selectString)

setupString = "from __main__ import countRecords"

# Truncate the tables and re-fill with a set amount of data
for theLength in [3,10,30,100,300,1000,3000,10000,30000,100000]:

    truncateString = "TRUNCATE test_table_innodb"
    truncateString2 = "TRUNCATE test_table_myisam"
    truncateString3 = "TRUNCATE test_table_memory"

    cur.execute(truncateString)
    cur.execute(truncateString2)
    cur.execute(truncateString3)

    for x in xrange(theLength):
        rand1 = random.random()
        rand2 = random.random()
        rand3 = random.random()
        rand4 = random.random()

        insertString = "INSERT INTO test_table_innodb (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
        insertString2 = "INSERT INTO test_table_myisam (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
        insertString3 = "INSERT INTO test_table_memory (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"

        cur.execute(insertString)
        cur.execute(insertString2)
        cur.execute(insertString3)

    db.commit()

    # Count and time the query
    innodb_times.append( timeit.timeit('countRecords("test_table_innodb")', number=100, setup=setupString) )
    myisam_times.append( timeit.timeit('countRecords("test_table_myisam")', number=100, setup=setupString) )
    memory_times.append( timeit.timeit('countRecords("test_table_memory")', number=100, setup=setupString) )

 

3)有条件选择

查询:SELECT * FROM tbl WHERE value1<0.5 AND value2<0.5 AND value3<0.5 AND value4<0.5

结果:MyISAM获胜

在这里,MyISAM和内存的性能大致相同,对于更大的表,它比InnoDB高出50%左右。在这类查询中,MyISAM的好处似乎得到了最大化。

代码:

myisam_times = []
innodb_times = []
memory_times = []

# Define a function to perform conditional selects
def conditionalSelect(testTable):
    selectString = "SELECT * FROM " + testTable + " WHERE value1 < 0.5 AND value2 < 0.5 AND value3 < 0.5 AND value4 < 0.5"
    cur.execute(selectString)

setupString = "from __main__ import conditionalSelect"

# Truncate the tables and re-fill with a set amount of data
for theLength in [3,10,30,100,300,1000,3000,10000,30000,100000]:

    truncateString = "TRUNCATE test_table_innodb"
    truncateString2 = "TRUNCATE test_table_myisam"
    truncateString3 = "TRUNCATE test_table_memory"

    cur.execute(truncateString)
    cur.execute(truncateString2)
    cur.execute(truncateString3)

    for x in xrange(theLength):
        rand1 = random.random()
        rand2 = random.random()
        rand3 = random.random()
        rand4 = random.random()

        insertString = "INSERT INTO test_table_innodb (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
        insertString2 = "INSERT INTO test_table_myisam (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
        insertString3 = "INSERT INTO test_table_memory (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"

        cur.execute(insertString)
        cur.execute(insertString2)
        cur.execute(insertString3)

    db.commit()

    # Count and time the query
    innodb_times.append( timeit.timeit('conditionalSelect("test_table_innodb")', number=100, setup=setupString) )
    myisam_times.append( timeit.timeit('conditionalSelect("test_table_myisam")', number=100, setup=setupString) )
    memory_times.append( timeit.timeit('conditionalSelect("test_table_memory")', number=100, setup=setupString) )

 

4)子

结果:InnoDB胜出

对于这个查询,我为子选择创建了一组额外的表。每个都是简单的两列bigint,一列有主键索引,另一列没有任何索引。由于表的大小很大,我没有测试内存引擎。SQL表创建命令为

CREATE TABLE
    subselect_myisam
    (
        index_col bigint NOT NULL,
        non_index_col bigint,
        PRIMARY KEY (index_col)
    )
    ENGINE=MyISAM DEFAULT CHARSET=utf8;

在第二个表中,'MyISAM'再次替换'InnoDB'。

在这个查询中,我将选择表的大小保留为1000000,而是改变子选择列的大小。

在这一点上,InnoDB很容易获胜。在我们得到一个合理的大小表后,两个引擎都线性缩放子选择的大小。索引加快了MyISAM命令的速度,但有趣的是,它对InnoDB的速度几乎没有影响。 subSelect.png

代码:

myisam_times = []
innodb_times = []
myisam_times_2 = []
innodb_times_2 = []

def subSelectRecordsIndexed(testTable,testSubSelect):
    selectString = "SELECT * FROM " + testTable + " WHERE index_col in ( SELECT index_col FROM " + testSubSelect + " )"
    cur.execute(selectString)

setupString = "from __main__ import subSelectRecordsIndexed"

def subSelectRecordsNotIndexed(testTable,testSubSelect):
    selectString = "SELECT * FROM " + testTable + " WHERE index_col in ( SELECT non_index_col FROM " + testSubSelect + " )"
    cur.execute(selectString)

setupString2 = "from __main__ import subSelectRecordsNotIndexed"

# Truncate the old tables, and re-fill with 1000000 records
truncateString = "TRUNCATE test_table_innodb"
truncateString2 = "TRUNCATE test_table_myisam"

cur.execute(truncateString)
cur.execute(truncateString2)

lengthOfTable = 1000000

# Fill up the tables with random data
for x in xrange(lengthOfTable):
    rand1 = random.random()
    rand2 = random.random()
    rand3 = random.random()
    rand4 = random.random()

    insertString = "INSERT INTO test_table_innodb (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"
    insertString2 = "INSERT INTO test_table_myisam (value1,value2,value3,value4) VALUES (" + str(rand1) + "," + str(rand2) + "," + str(rand3) + "," + str(rand4) + ")"

    cur.execute(insertString)
    cur.execute(insertString2)

for theLength in [3,10,30,100,300,1000,3000,10000,30000,100000]:

    truncateString = "TRUNCATE subselect_innodb"
    truncateString2 = "TRUNCATE subselect_myisam"

    cur.execute(truncateString)
    cur.execute(truncateString2)

    # For each length, empty the table and re-fill it with random data
    rand_sample = sorted(random.sample(xrange(lengthOfTable), theLength))
    rand_sample_2 = random.sample(xrange(lengthOfTable), theLength)

    for (the_value_1,the_value_2) in zip(rand_sample,rand_sample_2):
        insertString = "INSERT INTO subselect_innodb (index_col,non_index_col) VALUES (" + str(the_value_1) + "," + str(the_value_2) + ")"
        insertString2 = "INSERT INTO subselect_myisam (index_col,non_index_col) VALUES (" + str(the_value_1) + "," + str(the_value_2) + ")"

        cur.execute(insertString)
        cur.execute(insertString2)

    db.commit()

    # Finally, time the queries
    innodb_times.append( timeit.timeit('subSelectRecordsIndexed("test_table_innodb","subselect_innodb")', number=100, setup=setupString) )
    myisam_times.append( timeit.timeit('subSelectRecordsIndexed("test_table_myisam","subselect_myisam")', number=100, setup=setupString) )
        
    innodb_times_2.append( timeit.timeit('subSelectRecordsNotIndexed("test_table_innodb","subselect_innodb")', number=100, setup=setupString2) )
    myisam_times_2.append( timeit.timeit('subSelectRecordsNotIndexed("test_table_myisam","subselect_myisam")', number=100, setup=setupString2) )

我认为所有这些的关键信息是,如果你真的关心速度,你需要对你正在执行的查询进行基准测试,而不是假设哪个引擎更适合。

我知道这不会受欢迎,但我还是这么说:

myISAM缺乏对数据库基本要素的支持,如事务和引用完整性,这通常会导致应用程序出现故障/ bug。如果您的数据库引擎甚至不支持这些基础知识,那么您不可能不学习正确的数据库设计基础知识。

在数据库世界中不使用引用完整性或事务就像在软件世界中不使用面向对象编程一样。

InnoDB已经存在了,使用它吧!尽管myISAM是所有遗留系统的默认引擎,但MySQL开发人员最终还是同意在新版本中将其更改为默认引擎。

不,无论您是在读写还是有什么性能考虑,使用myISAM都会导致各种各样的问题,比如我刚刚遇到的这个问题:我正在执行数据库同步,同时其他人访问了一个访问myISAM表的应用程序。由于缺乏事务支持,这个引擎的可靠性也很差,导致整个数据库崩溃,我不得不手动重启mysql!

在过去15年的开发中,我使用了许多数据库和引擎。在此期间,myISAM在我身上崩溃了十几次,其他数据库只有一次!那是一个microsoft SQL数据库,其中一些开发人员编写了错误的CLR代码(公共语言运行时-基本上是在数据库内部执行的c#代码),顺便说一下,这并不是数据库引擎的错误。

我同意这里的其他答案,即高质量的高可用性、高性能应用程序不应该使用myISAM,因为它不起作用,它不够健壮或不够稳定,无法带来无挫折的体验。详见Bill Karwin的回答。

附注:我喜欢myISAM的粉丝们投反对票,但不能告诉你这个答案的哪一部分是错误的。

我认为这是一篇很好的文章,解释了两者之间的区别,以及什么时候应该使用其中一种: http://tag1consulting.com/MySQL_Engines_MyISAM_vs_InnoDB