为什么SELECT *是不好的做法?如果您添加了想要的新列,这难道不意味着需要更改的代码更少吗?

我知道SELECT COUNT(*)在某些db上是一个性能问题,但是如果您真的想要每个列呢?


当前回答

这里有一个重要的区别,我认为大多数答案都忽略了。

SELECT *不是问题。返回SELECT *的结果是问题所在。

举个例子,在我看来:

WITH data_from_several_tables AS (
    SELECT * FROM table1_2020
        UNION ALL
    SELECT * FROM table1_2021
    ...
)
SELECT id, name, ...
FROM data_from_several_tables
WHERE ...
GROUP BY ...
...

这避免了大多数答案中提到的使用SELECT *的所有“问题”:

读取的数据比预期的多?现代数据库中的优化器会意识到实际上并不需要所有列 源表的列顺序会影响输出吗?我们仍然选择和 显式返回数据。 消费者不能看到他们从SQL?您所操作的列在代码中是显式的。 索引可能不被使用?同样,现代优化器应该处理这个问题,就像我们没有选择*一样

这里有一个可读性/可重构性的优势——不需要重复很长的列列表或其他常见的查询子句(如过滤器)。如果在使用SELECT *和SELECT <columns>(在绝大多数情况下-显然总是在关键情况下配置运行代码)时,查询计划有任何不同,我会感到惊讶。

其他回答

还有一个更实际的原因:钱。当你使用云数据库时,你必须为数据处理付费,没有任何解释来读取你将立即丢弃的数据。

例如:BigQuery:

查询价格 查询定价是指运行SQL命令和用户定义函数的成本。BigQuery根据一个指标对查询收费:处理的字节数。

和控制投影-避免选择*:

最佳实践:控制投影—只查询所需的列。 投影指的是查询读取的列数。投影多余的列会导致额外的(浪费的)I/O和物化(写入结果)。 使用SELECT *是最昂贵的查询数据的方法。当您使用SELECT *时,BigQuery会对表中的每一列进行全面扫描。

当您只需要几列时使用SELECT *意味着传输的数据比您需要的多得多。这增加了数据库上的处理,并增加了将数据获取到客户端的延迟。此外,它在加载时将使用更多内存,在某些情况下会使用更多内存,例如大型BLOB文件,这主要是关于效率。

然而,除此之外,在查看查询时更容易看到正在加载哪些列,而不必查找表中的内容。

是的,如果您确实添加了一个额外的列,它会更快,但在大多数情况下,您希望/需要使用查询更改代码以接受新列,并且有可能获得您不想要/期望的列会导致问题。例如,如果获取所有列,然后依赖循环中的顺序来分配变量,然后再添加一个,或者如果列的顺序发生了变化(从备份恢复时就发生过这种情况),它可能会丢弃所有内容。

这也是为什么在执行INSERT操作时总是要指定列的原因。

在设计方案之前理解您的需求(如果可能的话)。

了解数据, 1)索引 2)所使用的存储类型; 3)供应商引擎或功能;即……缓存,内存功能 4)数据类型 5)桌子大小 6)查询频率 7)如果资源是共享的,相关的工作量 8)测试

A)要求会有所不同。如果硬件不能支持预期的工作负载,则应该重新评估如何在工作负载中提供需求。关于向表中添加的列。如果数据库支持视图,您可以使用特定的命名列创建特定数据的索引(?)视图(vs.选择'*')。定期检查您的数据和模式,以确保您永远不会遇到“输入垃圾”->“输出垃圾”综合征。

假设没有其他解;你可以考虑以下几点。一个问题总是有多种解决方案。

1)索引:select *将执行一个表罐。根据各种因素,这可能涉及到磁盘寻道和/或与其他查询的争用。如果表是多用途的,请确保所有查询都是高性能的,并在您的目标时间以下执行。如果有大量数据,而您的网络或其他资源没有调优;你需要考虑到这一点。数据库是一个共享环境。

2)存储类型。Ie:如果你使用SSD,磁盘或内存。I/O时间和系统/cpu上的负载会有所不同。

3) DBA是否可以调优数据库/表以获得更高的性能?假设出于某种原因,团队已经决定选择“*”是问题的最佳解决方案;可以将DB或表加载到内存中。(或者其他方法…也许反应被设计成有2-3秒的延迟?——而广告的作用是为公司赚取收入……)

4)从基线开始。了解您的数据类型,以及如何显示结果。更小的数据类型、字段数量会减少结果集中返回的数据量。这将为其他系统需求留下可用资源。系统资源通常是有限制的;“总是”工作低于这些限制,以确保稳定性和可预测的行为。

5)表/数据的大小。选择“*”在小表中很常见。它们通常适合内存,并且响应时间很快。再次……回顾您的需求。特征蠕变计划;总是为当前和未来可能的需求做计划。

6)查询/查询频率。了解系统上的其他工作负载。如果这个查询每秒发出一次,并且表很小。结果集可以设计为保留在缓存/内存中。然而,如果查询是一个频繁的批处理过程,有千兆字节/兆兆字节的数据……最好分配额外的资源以确保其他工作负载不受影响。

7) Related workloads. Understand how the resources are used. Is the network/system/database/table/application dedicated, or shared? Who are the stakeholders? Is this for production, development, or QA? Is this a temporary "quick fix". Have you tested the scenario? You'll be surprised how many problems can exist on current hardware today. (Yes, performance is fast...but the design/performance is still degraded.) Does the system need to performance 10K queries per second vs. 5-10 queries per second. Is the database server dedicated, or do other applications, monitoring execute on the shared resource. Some applications/languages; O/S's will consume 100% of the memory causing various symptoms/problems.

8)测试:测试你的理论,尽可能多地理解。你选择的“*”问题可能是一个大问题,或者它可能是你甚至不需要担心的事情。

如果您将字段添加到表中,它们将自动包含在使用select *的所有查询中。这看起来很方便,但它会使您的应用程序变慢,因为您获取的数据比您需要的要多,并且它实际上会在某些时候使您的应用程序崩溃。

在结果的每一行中可以获取多少数据是有限制的。如果向表中添加字段导致结果超过了限制,则在尝试运行查询时将收到错误消息。

这是一种很难发现的错误。您在一个地方做了更改,而在另一个实际上根本不使用新数据的地方它就会崩溃。它甚至可能是一个不太常用的查询,以至于在有人使用它之前需要一段时间,这使得将错误与更改联系起来更加困难。

如果指定希望在结果中显示哪些字段,就不会出现这种开销溢出。

有三个主要原因:

Inefficiency in moving data to the consumer. When you SELECT *, you're often retrieving more columns from the database than your application really needs to function. This causes more data to move from the database server to the client, slowing access and increasing load on your machines, as well as taking more time to travel across the network. This is especially true when someone adds new columns to underlying tables that didn't exist and weren't needed when the original consumers coded their data access. Indexing issues. Consider a scenario where you want to tune a query to a high level of performance. If you were to use *, and it returned more columns than you actually needed, the server would often have to perform more expensive methods to retrieve your data than it otherwise might. For example, you wouldn't be able to create an index which simply covered the columns in your SELECT list, and even if you did (including all columns [shudder]), the next guy who came around and added a column to the underlying table would cause the optimizer to ignore your optimized covering index, and you'd likely find that the performance of your query would drop substantially for no readily apparent reason. Binding Problems. When you SELECT *, it's possible to retrieve two columns of the same name from two different tables. This can often crash your data consumer. Imagine a query that joins two tables, both of which contain a column called "ID". How would a consumer know which was which? SELECT * can also confuse views (at least in some versions SQL Server) when underlying table structures change -- the view is not rebuilt, and the data which comes back can be nonsense. And the worst part of it is that you can take care to name your columns whatever you want, but the next guy who comes along might have no way of knowing that he has to worry about adding a column which will collide with your already-developed names.

但这对SELECT *来说也不全是坏事。我在以下用例中大量使用它:

Ad-hoc queries. When trying to debug something, especially off a narrow table I might not be familiar with, SELECT * is often my best friend. It helps me just see what's going on without having to do a boatload of research as to what the underlying column names are. This gets to be a bigger "plus" the longer the column names get. When * means "a row". In the following use cases, SELECT * is just fine, and rumors that it's a performance killer are just urban legends which may have had some validity many years ago, but don't now: SELECT COUNT(*) FROM table; in this case, * means "count the rows". If you were to use a column name instead of * , it would count the rows where that column's value was not null. COUNT(*), to me, really drives home the concept that you're counting rows, and you avoid strange edge-cases caused by NULLs being eliminated from your aggregates. Same goes with this type of query: SELECT a.ID FROM TableA a WHERE EXISTS ( SELECT * FROM TableB b WHERE b.ID = a.B_ID); in any database worth its salt, * just means "a row". It doesn't matter what you put in the subquery. Some people use b's ID in the SELECT list, or they'll use the number 1, but IMO those conventions are pretty much nonsensical. What you mean is "count the row", and that's what * signifies. Most query optimizers out there are smart enough to know this. (Though to be honest, I only know this to be true with SQL Server and Oracle.)