通过调试信息,我指的是TensorFlow在我的终端中显示的关于加载的库和找到的设备等的信息,而不是Python错误。
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:900] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: Graphics Device
major: 5 minor: 2 memoryClockRate (GHz) 1.0885
pciBusID 0000:04:00.0
Total memory: 12.00GiB
Free memory: 11.83GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:717] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Graphics Device, pci bus id: 0000:04:00.0)
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 1.0KiB
...
我为此挣扎了一段时间,尝试了这里几乎所有的解决方案,但无法摆脱TF 1.14中的调试信息,我尝试了以下多个解决方案:
import os
import logging
import sys
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # FATAL
stderr = sys.stderr
sys.stderr = open(os.devnull, 'w')
import tensorflow as tf
tf.get_logger().setLevel(tf.compat.v1.logging.FATAL)
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
logging.getLogger('tensorflow').setLevel(tf.compat.v1.logging.FATAL)
sys.stderr = stderr
import absl.logging
logging.root.removeHandler(absl.logging._absl_handler)
absl.logging._warn_preinit_stderr = False
调试信息仍然显示,最终帮助我重新启动我的电脑(实际上重新启动内核应该工作)。所以如果有人有类似的问题,试着在你设置环境变量后重新启动内核,简单但可能不会想到。
是的,我正在使用tf 2.0 beta,并想启用/禁用默认日志记录。tf1中的环境变量和方法。X似乎不再存在了。
我在PDB中发现了这个方法:
# close the TF2 logger
tf2logger = tf.get_logger()
tf2logger.error('Close TF2 logger handlers')
tf2logger.root.removeHandler(tf2logger.root.handlers[0])
然后添加自己的日志记录器API(在本例中是基于文件的)
logtf = logging.getLogger('DST')
logtf.setLevel(logging.DEBUG)
# file handler
logfile='/tmp/tf_s.log'
fh = logging.FileHandler(logfile)
fh.setFormatter( logging.Formatter('fh %(asctime)s %(name)s %(filename)s:%(lineno)d :%(message)s') )
logtf.addHandler(fh)
logtf.info('writing to %s', logfile)
为了增加一些灵活性,你可以通过编写一个过滤消息的函数来实现对日志级别的更细粒度的控制:
logging.getLogger('tensorflow').addFilter(my_filter_func)
where my_filter_func接受LogRecord对象作为输入[LogRecord docs]和
如果您希望抛出消息,则返回0;非零。
下面是一个示例过滤器,它只保留每n条信息消息(Python 3到期)
在这里使用nonlocal):
def keep_every_nth_info(n):
i = -1
def filter_record(record):
nonlocal i
i += 1
return int(record.levelname != 'INFO' or i % n == 0)
return filter_record
# Example usage for TensorFlow:
logging.getLogger('tensorflow').addFilter(keep_every_nth_info(5))
以上所有都假设TensorFlow已经设置了它的日志记录状态。在添加过滤器之前调用tf.logging.get_verbosity()可以确保没有副作用。