一位同事曾经告诉我,当Linux上所有调试都失败时,最后的选择是使用strace。

我试图学习这个奇怪工具背后的科学,但我不是系统管理专家,我没有真正得到结果。

So,

它到底是什么,有什么作用? 如何以及在哪些情况下使用它? 应该如何理解和处理输出?

简而言之,简单地说,这东西是怎么工作的?


当前回答

Strace可以用作调试工具,也可以用作原语分析器。

As a debugger, you can see how given system calls were called, executed and what they return. This is very important, as it allows you to see not only that a program failed, but WHY a program failed. Usually it's just a result of lousy coding not catching all the possible outcomes of a program. Other times it's just hardcoded paths to files. Without strace you get to guess what went wrong where and how. With strace you get a breakdown of a syscall, usually just looking at a return value tells you a lot.

剖析是另一个用途。您可以使用它来分别计时每个系统调用的执行,或者作为一个聚合。虽然这可能不足以解决您的问题,但至少可以大大缩小潜在嫌疑人的范围。如果您在单个文件上看到大量的fopen/close对,那么您可能在每次执行循环时都不必要地打开和关闭文件,而不是在循环之外打开和关闭它。

Ltrace是strace的近亲,也非常有用。你必须学会区分你的瓶颈在哪里。如果执行的总时间是8秒,而你在系统调用上只花了0.05秒,那么对程序进行分段不会有什么好处,问题出在你的代码中,这通常是一个逻辑问题,或者程序实际上需要花那么长时间来运行。

The biggest problem with strace/ltrace is reading their output. If you don't know how the calls are made, or at least the names of syscalls/functions, it's going to be difficult to decipher the meaning. Knowing what the functions return can also be very beneficial, especially for different error codes. While it's a pain to decipher, they sometimes really return a pearl of knowledge; once I saw a situation where I ran out of inodes, but not out of free space, thus all the usual utilities didn't give me any warning, I just couldn't make a new file. Reading the error code from strace's output pointed me in the right direction.

其他回答

下面是我如何使用strace挖掘网站的一些例子。希望这对你有帮助。

检查时间到第一个字节,如下所示:

time php index.php > timeTrace.txt

看看有多少百分比的动作在做什么。大量的lstat和fstat可能表明是时候清理缓存了:

strace -s 200 -c php index.php > traceLstat.txt

输出trace.txt,以便您可以准确地看到正在进行的调用。

strace -Tt -o Fulltrace.txt php index.php

使用它来检查是否有东西在0.1到0.9秒之间加载:

cat Fulltrace.txt | grep "[<]0.[1-9]" > traceSlowest.txt

查看strace中捕获了哪些丢失的文件或目录。这将输出很多涉及我们系统的东西-唯一相关的位涉及客户的文件:

strace -vv php index.php 2>&1 | sed -n '/= -1/p' > traceFailures.txt

Strace是一个很好的工具,用于了解程序如何进行各种系统调用(对内核的请求),并报告失败的调用以及与该失败相关的错误值。并不是所有的失败都是bug。例如,试图搜索文件的代码可能会得到ENOENT(没有这样的文件或目录)错误,但这可能是代码逻辑中可以接受的场景。

使用strace的一个很好的用例是在临时文件创建期间调试竞态条件。例如,通过将进程ID (PID)附加到某个预先确定的字符串来创建文件的程序在多线程场景中可能会遇到问题。PID+TID(进程id +线程id)或更好的系统调用(如mkstemp)将修复此问题。

它还适用于调试崩溃。您可能会发现这篇(我的)关于strace和调试崩溃的文章很有用。

strace -tfp PID将监控PID进程的系统调用,因此我们可以调试/监控我们的进程/程序状态。

我喜欢一些答案,它读取strace检查你如何与操作系统交互。

这正是我们所看到的。系统调用。如果比较strace和ltrace,区别就更明显了。

$>strace -c cd
Desktop  Documents  Downloads  examples.desktop  Music  Pictures  Public  Templates  Videos
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0         7           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0        11           close
  0.00    0.000000           0        10           fstat
  0.00    0.000000           0        17           mmap
  0.00    0.000000           0        12           mprotect
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         2           rt_sigaction
  0.00    0.000000           0         1           rt_sigprocmask
  0.00    0.000000           0         2           ioctl
  0.00    0.000000           0         8         8 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         2           getdents
  0.00    0.000000           0         2         2 statfs
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         9           openat
  0.00    0.000000           0         1           set_robust_list
  0.00    0.000000           0         1           prlimit64
------ ----------- ----------- --------- --------- ----------------
100.00    0.000000                    93        10 total

另一方面,ltrace用于跟踪函数。

$>ltrace -c cd
Desktop  Documents  Downloads  examples.desktop  Music  Pictures  Public  Templates  Videos
% time     seconds  usecs/call     calls      function
------ ----------- ----------- --------- --------------------
 15.52    0.004946         329        15 memcpy
 13.34    0.004249          94        45 __ctype_get_mb_cur_max
 12.87    0.004099        2049         2 fclose
 12.12    0.003861          83        46 strlen
 10.96    0.003491         109        32 __errno_location
 10.37    0.003303         117        28 readdir
  8.41    0.002679         133        20 strcoll
  5.62    0.001791         111        16 __overflow
  3.24    0.001032         114         9 fwrite_unlocked
  1.26    0.000400         100         4 __freading
  1.17    0.000372          41         9 getenv
  0.70    0.000222         111         2 fflush
  0.67    0.000214         107         2 __fpending
  0.64    0.000203         101         2 fileno
  0.62    0.000196         196         1 closedir
  0.43    0.000138         138         1 setlocale
  0.36    0.000114         114         1 _setjmp
  0.31    0.000098          98         1 realloc
  0.25    0.000080          80         1 bindtextdomain
  0.21    0.000068          68         1 opendir
  0.19    0.000062          62         1 strrchr
  0.18    0.000056          56         1 isatty
  0.16    0.000051          51         1 ioctl
  0.15    0.000047          47         1 getopt_long
  0.14    0.000045          45         1 textdomain
  0.13    0.000042          42         1 __cxa_atexit
------ ----------- ----------- --------- --------------------
100.00    0.031859                   244 total

虽然我检查了几次手册,我还没有找到名称strace的起源,但它可能是系统调用跟踪,因为这是显而易见的。

关于strace有三个更重要的说明。

注1:strace和ltrace这两个函数都使用了系统调用ptrace。所以ptrace系统调用是strace有效工作的方式。

ptrace()系统调用提供了一种方法,通过这种方法,一个进程 “跟踪程序”)可以观察和控制另一个进程的执行 (“被追踪者”),检查和改变被追踪者的记忆和 寄存器。它主要用于实现断点调试 以及系统调用跟踪。

Note 2: There are different parameters you can use with strace, since strace can be very verbose. I like to experiment with -c which is like a summary of things. Based on -c you can select one system-call like -e trace=open where you will see only that call. This can be interesting if you are examining what files will be opened during the command you are tracing. And of course, you can use the grep for the same purpose but note you need to redirect like this 2>&1 | grep etc to understand that config files are referenced when the command was issued.

注3:我发现这一点非常重要。您不局限于特定的体系结构。Strace会让你大吃一惊,因为它可以跟踪不同体系结构的二进制文件。

Strace是一种突出的工具,用于研究无法在调试器下运行这些程序的生产系统。具体来说,我们在以下两种情况下使用了strace:

Program foo seems to be in deadlock and has become unresponsive. This could be a target for gdb; however, we haven't always had the source code or sometimes were dealing with scripted languages that weren't straight-forward to run under a debugger. In this case, you run strace on an already running program and you will get the list of system calls being made. This is particularly useful if you are investigating a client/server application or an application that interacts with a database Investigating why a program is slow. In particular, we had just moved to a new distributed file system and the new throughput of the system was very slow. You can specify strace with the '-T' option which will tell you how much time was spent in each system call. This helped to determine why the file system was causing things to slow down.

有关使用strace进行分析的示例,请参阅我对这个问题的回答。