“N+1选择问题”在对象关系映射(ORM)讨论中通常被称为一个问题,我理解这与必须为对象世界中看似简单的东西进行大量数据库查询有关。
有人对这个问题有更详细的解释吗?
“N+1选择问题”在对象关系映射(ORM)讨论中通常被称为一个问题,我理解这与必须为对象世界中看似简单的东西进行大量数据库查询有关。
有人对这个问题有更详细的解释吗?
当前回答
N+1查询问题是什么
当数据访问框架执行N个额外的SQL语句以获取执行主SQL查询时可能检索到的相同数据时,就会出现N+1查询问题。
N值越大,执行的查询越多,性能影响越大。而且,与可以帮助您查找运行缓慢的查询的慢速查询日志不同,N+1问题不会出现,因为每个单独的附加查询运行速度都足够快,不会触发慢速查询日志。
问题是执行大量额外的查询,总的来说,这些查询需要足够的时间来降低响应时间。
让我们考虑以下post和post_comments数据库表,它们形成了一对多的表关系:
我们将创建以下4个柱行:
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 1', 1)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 2', 2)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 3', 3)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 4', 4)
此外,我们还将创建4个post_comment子记录:
INSERT INTO post_comment (post_id, review, id)
VALUES (1, 'Excellent book to understand Java Persistence', 1)
INSERT INTO post_comment (post_id, review, id)
VALUES (2, 'Must-read for Java developers', 2)
INSERT INTO post_comment (post_id, review, id)
VALUES (3, 'Five Stars', 3)
INSERT INTO post_comment (post_id, review, id)
VALUES (4, 'A great reference book', 4)
普通SQL的N+1查询问题
如果使用此SQL查询选择post_comments:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
""", Tuple.class)
.getResultList();
稍后,您决定获取每个post_comment的相关文章标题:
for (Tuple comment : comments) {
String review = (String) comment.get("review");
Long postId = ((Number) comment.get("postId")).longValue();
String postTitle = (String) entityManager.createNativeQuery("""
SELECT
p.title
FROM post p
WHERE p.id = :postId
""")
.setParameter("postId", postId)
.getSingleResult();
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
您将触发N+1查询问题,因为您执行了5(1+4)而不是一个SQL查询:
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
SELECT p.title FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.title FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.title FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.title FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
修复N+1查询问题非常简单。您只需提取原始SQL查询中所需的所有数据,如下所示:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
p.title AS postTitle
FROM post_comment pc
JOIN post p ON pc.post_id = p.id
""", Tuple.class)
.getResultList();
for (Tuple comment : comments) {
String review = (String) comment.get("review");
String postTitle = (String) comment.get("postTitle");
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
这次,只执行一个SQL查询来获取我们进一步感兴趣的所有数据。
JPA和Hibernate的N+1查询问题
在使用JPA和Hibernate时,有几种方法可以触发N+1查询问题,因此了解如何避免这些情况非常重要。
对于下一个示例,考虑我们将post和post_comments表映射到以下实体:
JPA映射如下所示:
@Entity(name = "Post")
@Table(name = "post")
public class Post {
@Id
private Long id;
private String title;
//Getters and setters omitted for brevity
}
@Entity(name = "PostComment")
@Table(name = "post_comment")
public class PostComment {
@Id
private Long id;
@ManyToOne
private Post post;
private String review;
//Getters and setters omitted for brevity
}
获取类型.EAGER
隐式或显式地为JPA关联使用FetchType.EAGER是一个坏主意,因为您将获取更多所需的数据。此外,FetchType.EAGER策略还容易出现N+1个查询问题。
不幸的是,@ManyToOne和@OneToOne关联默认使用FetchType.EAGER,因此如果映射如下所示:
@ManyToOne
private Post post;
您使用的是FetchType.EAGER策略,每当您在使用JPQL或Criteria API查询加载某些PostComment实体时忘记使用JOIN FETCH时:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
您将触发N+1查询问题:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
请注意执行的其他SELECT语句,因为在返回PostComment实体列表之前必须获取post关联。
与调用EntityManager的find方法时使用的默认获取计划不同,JPQL或Criteria API查询定义了Hibernate无法通过自动注入JOIN fetch来更改的显式计划。因此,您需要手动执行。
如果你根本不需要post关联,那么你在使用FetchType.EAGER时就不走运了,因为无法避免获取它。这就是为什么默认情况下最好使用FetchType.LAZY。
但是,如果您想使用后关联,那么可以使用JOIN FETCH来避免N+1查询问题:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
这次,Hibernate将执行一条SQL语句:
SELECT
pc.id as id1_1_0_,
pc.post_id as post_id3_1_0_,
pc.review as review2_1_0_,
p.id as id1_0_1_,
p.title as title2_0_1_
FROM
post_comment pc
INNER JOIN
post p ON pc.post_id = p.id
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
获取类型.LAZY
即使您切换到对所有关联显式使用FetchType.LAZY,您仍然会遇到N+1问题。
这一次,后关联映射如下:
@ManyToOne(fetch = FetchType.LAZY)
private Post post;
现在,当您获取PostComment实体时:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
Hibernate将执行一条SQL语句:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
但是,如果之后,您将引用延迟加载的post关联:
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
您将获得N+1查询问题:
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
由于后期关联是延迟获取的,因此在访问延迟关联时将执行一个辅助SQL语句,以便生成日志消息。
同样,修复方法包括在JPQL查询中添加JOIN FETCH子句:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
而且,就像FetchType.EAGER示例中一样,这个JPQL查询将生成一个SQL语句。
即使您正在使用FetchType.LAZY,并且没有引用双向@OneToOne JPA关系的子关联,您仍然可以触发N+1查询问题。
如何自动检测N+1查询问题
如果您想在数据访问层中自动检测N+1查询问题,可以使用db-util开源项目。
首先,您需要添加以下Maven依赖项:
<dependency>
<groupId>com.vladmihalcea</groupId>
<artifactId>db-util</artifactId>
<version>${db-util.version}</version>
</dependency>
之后,您只需使用SQLStatementCountValidator实用程序来断言生成的底层SQL语句:
SQLStatementCountValidator.reset();
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
SQLStatementCountValidator.assertSelectCount(1);
如果您正在使用FetchType.EAGER并运行上述测试用例,则会出现以下测试用例失败:
SELECT
pc.id as id1_1_,
pc.post_id as post_id3_1_,
pc.review as review2_1_
FROM
post_comment pc
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 2
-- SQLStatementCountMismatchException: Expected 1 statement(s) but recorded 3 instead!
其他回答
正如其他人更优雅地指出的那样,问题是您要么拥有OneToMany列的笛卡尔积,要么正在进行N+1选择。无论是可能的巨大结果集,还是与数据库的聊天。
我很惊讶没有提到这一点,但我是如何解决这个问题的。。。我制作了一个半临时ID表。当您有IN()条款限制时,我也会这样做。
这并不适用于所有情况(可能甚至不适用于大多数情况),但如果您有很多子对象,使得笛卡儿乘积无法控制(即大量的OneToMany列,结果的数量将是列的乘积),并且它更像是一个批处理作业,那么它就特别适用。
首先,将父对象ID作为批处理插入到ID表中。batch_id是我们在应用程序中生成并保存的东西。
INSERT INTO temp_ids
(product_id, batch_id)
(SELECT p.product_id, ?
FROM product p ORDER BY p.product_id
LIMIT ? OFFSET ?);
现在,对于每个OneToMany列,您只需在id表INNER上执行SELECT,然后使用WHERE batch_id=(反之亦然)将子表JOIN。您只需要确保按id列排序,因为这将使合并结果列更容易(否则,您将需要一个HashMap/Table用于整个结果集,这可能不会那么糟糕)。
然后,只需定期清理ids表。
如果用户选择例如100个左右不同的项目进行某种批量处理,这也特别有效。将100个不同的ID放入临时表中。
现在,您正在执行的查询数量是OneToMany列的数量。
N+1查询问题是什么
当数据访问框架执行N个额外的SQL语句以获取执行主SQL查询时可能检索到的相同数据时,就会出现N+1查询问题。
N值越大,执行的查询越多,性能影响越大。而且,与可以帮助您查找运行缓慢的查询的慢速查询日志不同,N+1问题不会出现,因为每个单独的附加查询运行速度都足够快,不会触发慢速查询日志。
问题是执行大量额外的查询,总的来说,这些查询需要足够的时间来降低响应时间。
让我们考虑以下post和post_comments数据库表,它们形成了一对多的表关系:
我们将创建以下4个柱行:
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 1', 1)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 2', 2)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 3', 3)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 4', 4)
此外,我们还将创建4个post_comment子记录:
INSERT INTO post_comment (post_id, review, id)
VALUES (1, 'Excellent book to understand Java Persistence', 1)
INSERT INTO post_comment (post_id, review, id)
VALUES (2, 'Must-read for Java developers', 2)
INSERT INTO post_comment (post_id, review, id)
VALUES (3, 'Five Stars', 3)
INSERT INTO post_comment (post_id, review, id)
VALUES (4, 'A great reference book', 4)
普通SQL的N+1查询问题
如果使用此SQL查询选择post_comments:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
""", Tuple.class)
.getResultList();
稍后,您决定获取每个post_comment的相关文章标题:
for (Tuple comment : comments) {
String review = (String) comment.get("review");
Long postId = ((Number) comment.get("postId")).longValue();
String postTitle = (String) entityManager.createNativeQuery("""
SELECT
p.title
FROM post p
WHERE p.id = :postId
""")
.setParameter("postId", postId)
.getSingleResult();
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
您将触发N+1查询问题,因为您执行了5(1+4)而不是一个SQL查询:
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
SELECT p.title FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.title FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.title FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.title FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
修复N+1查询问题非常简单。您只需提取原始SQL查询中所需的所有数据,如下所示:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
p.title AS postTitle
FROM post_comment pc
JOIN post p ON pc.post_id = p.id
""", Tuple.class)
.getResultList();
for (Tuple comment : comments) {
String review = (String) comment.get("review");
String postTitle = (String) comment.get("postTitle");
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
这次,只执行一个SQL查询来获取我们进一步感兴趣的所有数据。
JPA和Hibernate的N+1查询问题
在使用JPA和Hibernate时,有几种方法可以触发N+1查询问题,因此了解如何避免这些情况非常重要。
对于下一个示例,考虑我们将post和post_comments表映射到以下实体:
JPA映射如下所示:
@Entity(name = "Post")
@Table(name = "post")
public class Post {
@Id
private Long id;
private String title;
//Getters and setters omitted for brevity
}
@Entity(name = "PostComment")
@Table(name = "post_comment")
public class PostComment {
@Id
private Long id;
@ManyToOne
private Post post;
private String review;
//Getters and setters omitted for brevity
}
获取类型.EAGER
隐式或显式地为JPA关联使用FetchType.EAGER是一个坏主意,因为您将获取更多所需的数据。此外,FetchType.EAGER策略还容易出现N+1个查询问题。
不幸的是,@ManyToOne和@OneToOne关联默认使用FetchType.EAGER,因此如果映射如下所示:
@ManyToOne
private Post post;
您使用的是FetchType.EAGER策略,每当您在使用JPQL或Criteria API查询加载某些PostComment实体时忘记使用JOIN FETCH时:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
您将触发N+1查询问题:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
请注意执行的其他SELECT语句,因为在返回PostComment实体列表之前必须获取post关联。
与调用EntityManager的find方法时使用的默认获取计划不同,JPQL或Criteria API查询定义了Hibernate无法通过自动注入JOIN fetch来更改的显式计划。因此,您需要手动执行。
如果你根本不需要post关联,那么你在使用FetchType.EAGER时就不走运了,因为无法避免获取它。这就是为什么默认情况下最好使用FetchType.LAZY。
但是,如果您想使用后关联,那么可以使用JOIN FETCH来避免N+1查询问题:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
这次,Hibernate将执行一条SQL语句:
SELECT
pc.id as id1_1_0_,
pc.post_id as post_id3_1_0_,
pc.review as review2_1_0_,
p.id as id1_0_1_,
p.title as title2_0_1_
FROM
post_comment pc
INNER JOIN
post p ON pc.post_id = p.id
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
获取类型.LAZY
即使您切换到对所有关联显式使用FetchType.LAZY,您仍然会遇到N+1问题。
这一次,后关联映射如下:
@ManyToOne(fetch = FetchType.LAZY)
private Post post;
现在,当您获取PostComment实体时:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
Hibernate将执行一条SQL语句:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
但是,如果之后,您将引用延迟加载的post关联:
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
您将获得N+1查询问题:
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
由于后期关联是延迟获取的,因此在访问延迟关联时将执行一个辅助SQL语句,以便生成日志消息。
同样,修复方法包括在JPQL查询中添加JOIN FETCH子句:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
而且,就像FetchType.EAGER示例中一样,这个JPQL查询将生成一个SQL语句。
即使您正在使用FetchType.LAZY,并且没有引用双向@OneToOne JPA关系的子关联,您仍然可以触发N+1查询问题。
如何自动检测N+1查询问题
如果您想在数据访问层中自动检测N+1查询问题,可以使用db-util开源项目。
首先,您需要添加以下Maven依赖项:
<dependency>
<groupId>com.vladmihalcea</groupId>
<artifactId>db-util</artifactId>
<version>${db-util.version}</version>
</dependency>
之后,您只需使用SQLStatementCountValidator实用程序来断言生成的底层SQL语句:
SQLStatementCountValidator.reset();
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
SQLStatementCountValidator.assertSelectCount(1);
如果您正在使用FetchType.EAGER并运行上述测试用例,则会出现以下测试用例失败:
SELECT
pc.id as id1_1_,
pc.post_id as post_id3_1_,
pc.review as review2_1_
FROM
post_comment pc
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 2
-- SQLStatementCountMismatchException: Expected 1 statement(s) but recorded 3 instead!
N+1选择问题是一个难题,在单元测试中检测这种情况是有意义的。我已经开发了一个小型库,用于验证给定测试方法或任意代码块执行的查询数量-JDBC Sniffer
只需在测试类中添加一个特殊的JUnit规则,并在测试方法上放置带有预期查询数的注释:
@Rule
public final QueryCounter queryCounter = new QueryCounter();
@Expectation(atMost = 3)
@Test
public void testInvokingDatabase() {
// your JDBC or JPA code
}
N+1的推广
N+1问题是一个ORM特有的问题名称,它将可以在服务器上合理执行的循环移动到客户端。通用问题不是ORM特有的,您可以通过任何远程API解决。在本文中,我展示了如果您调用一个API N次而不是仅调用1次,JDBC往返是如何代价高昂的。示例中的区别在于您是否调用Oracle PL/SQL过程:
dbms_output.get_lines(调用一次,接收N个项目)dbms_output.get_line(调用N次,每次接收1项)
它们在逻辑上是等价的,但由于服务器和客户端之间的延迟,您需要在循环中添加N个延迟等待,而不是只等待一次。
ORM案例
事实上,ORM-y N+1问题甚至不是ORM特有的,您也可以通过手动运行自己的查询来实现,例如,当您在PL/SQL中执行以下操作时:
-- This loop is executed once
for parent in (select * from parent) loop
-- This loop is executed N times
for child in (select * from child where parent_id = parent.id) loop
...
end loop;
end loop;
使用联接(在本例中)实现这一点会更好:
for rec in (
select *
from parent p
join child c on c.parent_id = p.id
)
loop
...
end loop;
现在,循环只执行一次,并且循环的逻辑已经从客户端(PL/SQL)移动到服务器(SQL),这甚至可以以不同的方式对其进行优化,例如,通过运行哈希连接(O(N))而不是嵌套循环连接(带索引的O(N log N))
自动检测N+1个问题
如果您使用的是JDBC,可以在后台使用jOOQ作为JDBC代理来自动检测N+1问题。jOOQ的解析器规范化您的SQL查询,并缓存有关连续执行父查询和子查询的数据。如果您的查询不完全相同,但在语义上是等价的,这甚至可以起作用。
在Phabricator文档中可以找到问题的简短解释:
N+1查询问题是一个常见的性能反模式。它看起来像这样:$cats=load_cats();foreach($cat作为$cat){$cats_hats=>load_hats_for_cat($cat);// ...}假设load_cats()的实现归结为:从猫的位置选择*。。。..和load_hats_for_cat($cat)的实现如下:SELECT*FROM hat WHERE catID=。。。..当代码执行时,您将发出“N+1”个查询,其中N是猫的数量:从猫的位置选择*。。。SELECT*FROM hat WHERE catID=1SELECT*FROM hat WHERE catID=2SELECT*FROM hat WHERE catID=3从帽子中选择*,其中catID=4...
解决方案:
发出一个返回100个结果的查询比发出发出100个查询,每个查询返回1个结果。
在迭代之前加载所有数据。