我有一个日志文件正在写的另一个进程,我想观察变化。每次发生更改时,我都希望将新数据读入并对其进行一些处理。

最好的方法是什么?我希望在PyWin32库中有某种钩子。我找到了win32文件。函数FindNextChangeNotification,但不知道如何要求它监视特定的文件。

如果有人做过类似的事情,我真的很感激能听到…

[编辑]我应该提到我追求的是一种不需要轮询的解决方案。

[编辑]诅咒!这似乎不能在映射的网络驱动器上工作。我猜windows不会像在本地磁盘上那样“听到”任何对文件的更新。


当前回答

watchfiles (https://github.com/samuelcolvin/watchfiles)是一个Python API和CLI,它使用了用Rust编写的Notify (https://github.com/notify-rs/notify)库。

rust实现目前(2022-10-09)支持:

Linux / Android: inotify macOS: FSEvents或kqueue,参见特性 Windows: ReadDirectoryChangesW FreeBSD / NetBSD / OpenBSD / DragonflyBSD: kqueue 所有平台:投票

可以在PyPI (https://pypi.org/project/watchfiles/)和conda-forge (https://github.com/conda-forge/watchfiles-feedstock)上获得二进制文件。

其他回答

在对蒂姆·戈尔登的剧本进行了一些修改后,我有了以下似乎工作得相当不错的内容:

import os

import win32file
import win32con

path_to_watch = "." # look at the current directory
file_to_watch = "test.txt" # look for changes to a file called test.txt

def ProcessNewData( newData ):
    print "Text added: %s"%newData

# Set up the bits we'll need for output
ACTIONS = {
  1 : "Created",
  2 : "Deleted",
  3 : "Updated",
  4 : "Renamed from something",
  5 : "Renamed to something"
}
FILE_LIST_DIRECTORY = 0x0001
hDir = win32file.CreateFile (
  path_to_watch,
  FILE_LIST_DIRECTORY,
  win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE,
  None,
  win32con.OPEN_EXISTING,
  win32con.FILE_FLAG_BACKUP_SEMANTICS,
  None
)

# Open the file we're interested in
a = open(file_to_watch, "r")

# Throw away any exising log data
a.read()

# Wait for new data and call ProcessNewData for each new chunk that's written
while 1:
  # Wait for a change to occur
  results = win32file.ReadDirectoryChangesW (
    hDir,
    1024,
    False,
    win32con.FILE_NOTIFY_CHANGE_LAST_WRITE,
    None,
    None
  )

  # For each change, check to see if it's updating the file we're interested in
  for action, file in results:
    full_filename = os.path.join (path_to_watch, file)
    #print file, ACTIONS.get (action, "Unknown")
    if file == file_to_watch:
        newText = a.read()
        if newText != "":
            ProcessNewData( newText )

它可能需要进行更多的错误检查,但对于简单地观察日志文件并在将其输出到屏幕之前对其进行一些处理,这样做效果很好。

谢谢大家的投入-很棒的东西!

下面是一个示例,用于观察每秒写入不超过一行但通常要少得多的输入文件。目标是将最后一行(最近的写入)追加到指定的输出文件。我从我的一个项目中复制了这个,只是删除了所有不相关的行。你必须填写或修改缺失的符号。

from PyQt5.QtCore import QFileSystemWatcher, QSettings, QThread
from ui_main_window import Ui_MainWindow   # Qt Creator gen'd 

class MainWindow(QMainWindow, Ui_MainWindow):
    def __init__(self, parent=None):
        QMainWindow.__init__(self, parent)
        Ui_MainWindow.__init__(self)
        self._fileWatcher = QFileSystemWatcher()
        self._fileWatcher.fileChanged.connect(self.fileChanged)

    def fileChanged(self, filepath):
        QThread.msleep(300)    # Reqd on some machines, give chance for write to complete
        # ^^ About to test this, may need more sophisticated solution
        with open(filepath) as file:
            lastLine = list(file)[-1]
        destPath = self._filemap[filepath]['dest file']
        with open(destPath, 'a') as out_file:               # a= append
            out_file.writelines([lastLine])

当然,并不是严格要求包含QMainWindow类。您可以单独使用QFileSystemWatcher。

最简单的解决方案是在间隔之后获取同一个文件的两个实例并进行比较。你可以试试这样的东西

    while True:
        # Capturing the two instances models.py after certain interval of time
        print("Looking for changes in " + app_name.capitalize() + " models.py\nPress 'CTRL + C' to stop the program")
        with open(app_name.capitalize() + '/filename', 'r+') as app_models_file:
            filename_content = app_models_file.read()
        time.sleep(5)
        with open(app_name.capitalize() + '/filename', 'r+') as app_models_file_1:
            filename_content_1 = app_models_file_1.read()
        # Comparing models.py after certain interval of time
        if filename_content == filename_content_1:
            pass
        else:
            print("You made a change in " + app_name.capitalize() + " filename.\n")
            cmd = str(input("Do something with the file?(y/n):"))
            if cmd == 'y':
                # Do Something
            elif cmd == 'n':
                # pass or do something
            else:
                print("Invalid input")

相关@4Oh4解决方案一个流畅的更改文件列表观看;

import os
import sys
import time

class Watcher(object):
    running = True
    refresh_delay_secs = 1

    # Constructor
    def __init__(self, watch_files, call_func_on_change=None, *args, **kwargs):
        self._cached_stamp = 0
        self._cached_stamp_files = {}
        self.filenames = watch_files
        self.call_func_on_change = call_func_on_change
        self.args = args
        self.kwargs = kwargs

    # Look for changes
    def look(self):
        for file in self.filenames:
            stamp = os.stat(file).st_mtime
            if not file in self._cached_stamp_files:
                self._cached_stamp_files[file] = 0
            if stamp != self._cached_stamp_files[file]:
                self._cached_stamp_files[file] = stamp
                # File has changed, so do something...
                file_to_read = open(file, 'r')
                value = file_to_read.read()
                print("value from file", value)
                file_to_read.seek(0)
                if self.call_func_on_change is not None:
                    self.call_func_on_change(*self.args, **self.kwargs)

    # Keep watching in a loop
    def watch(self):
        while self.running:
            try:
                # Look for changes
                time.sleep(self.refresh_delay_secs)
                self.look()
            except KeyboardInterrupt:
                print('\nDone')
                break
            except FileNotFoundError:
                # Action on file not found
                pass
            except Exception as e:
                print(e)
                print('Unhandled error: %s' % sys.exc_info()[0])

# Call this function each time a change happens
def custom_action(text):
    print(text)
    # pass

watch_files = ['/Users/mexekanez/my_file.txt', '/Users/mexekanez/my_file1.txt']

# watcher = Watcher(watch_file)  # simple



if __name__ == "__main__":
    watcher = Watcher(watch_files, custom_action, text='yes, changed')  # also call custom action function
    watcher.watch()  # start the watch going

似乎没有人张贴fswatch。它是一个跨平台的文件系统监视器。只要安装它,运行它,并按照提示。

我在python和golang程序中使用过它,它只是工作。