我需要锁定一个文件写在Python。它将被多个Python进程同时访问。我在网上找到了一些解决方案,但大多数都无法达到我的目的,因为它们通常只是基于Unix或Windows的。
当前回答
我更喜欢lockfile——平台独立的文件锁定
其他回答
锁定是平台和设备特定的,但一般来说,你有几个选择:
Use flock(), or equivalent (if your os supports it). This is advisory locking, unless you check for the lock, it's ignored. Use a lock-copy-move-unlock methodology, where you copy the file, write the new data, then move it (move, not copy - move is an atomic operation in Linux -- check your OS), and you check for the existence of the lock file. Use a directory as a "lock". This is necessary if you're writing to NFS, since NFS doesn't support flock(). There's also the possibility of using shared memory between the processes, but I've never tried that; it's very OS-specific.
对于所有这些方法,您必须使用自旋锁(失败后重试)技术来获取和测试锁。这确实为错误同步留下了一个小窗口,但它通常小到不会成为一个大问题。
如果您正在寻找跨平台的解决方案,那么您最好通过其他机制将日志记录到另一个系统(其次是上面的NFS技术)。
请注意,sqlite在NFS上受到与普通文件相同的约束,因此您不能在网络共享上写入sqlite数据库并免费获得同步。
场景是这样的: 用户请求一个文件执行某些操作。然后,如果用户再次发送相同的请求,它会通知用户在第一个请求完成之前不会完成第二个请求。这就是为什么我使用锁定机制来处理这个问题。
下面是我的工作代码:
from lockfile import LockFile
lock = LockFile(lock_file_path)
status = ""
if not lock.is_locked():
lock.acquire()
status = lock.path + ' is locked.'
print status
else:
status = lock.path + " is already locked."
print status
return status
锁定文件通常是特定于平台的操作,因此您可能需要考虑在不同的操作系统上运行的可能性。例如:
import os
def my_lock(f):
if os.name == "posix":
# Unix or OS X specific locking here
elif os.name == "nt":
# Windows specific locking here
else:
print "Unknown operating system, lock unavailable"
你会发现pylocker很有用。它可以用于锁定文件或一般的锁定机制,并且可以从多个Python进程一次访问。
如果你只是想锁定一个文件,下面是它的工作原理:
import uuid
from pylocker import Locker
# create a unique lock pass. This can be any string.
lpass = str(uuid.uuid1())
# create locker instance.
FL = Locker(filePath='myfile.txt', lockPass=lpass, mode='w')
# aquire the lock
with FL as r:
# get the result
acquired, code, fd = r
# check if aquired.
if fd is not None:
print fd
fd.write("I have succesfuly aquired the lock !")
# no need to release anything or to close the file descriptor,
# with statement takes care of that. let's print fd and verify that.
print fd
其他解决方案引用了大量的外部代码库。如果您更愿意自己动手,这里有一些跨平台解决方案的代码,在Linux / DOS系统上使用各自的文件锁定工具。
try:
# Posix based file locking (Linux, Ubuntu, MacOS, etc.)
# Only allows locking on writable files, might cause
# strange results for reading.
import fcntl, os
def lock_file(f):
if f.writable(): fcntl.lockf(f, fcntl.LOCK_EX)
def unlock_file(f):
if f.writable(): fcntl.lockf(f, fcntl.LOCK_UN)
except ModuleNotFoundError:
# Windows file locking
import msvcrt, os
def file_size(f):
return os.path.getsize( os.path.realpath(f.name) )
def lock_file(f):
msvcrt.locking(f.fileno(), msvcrt.LK_RLCK, file_size(f))
def unlock_file(f):
msvcrt.locking(f.fileno(), msvcrt.LK_UNLCK, file_size(f))
# Class for ensuring that all file operations are atomic, treat
# initialization like a standard call to 'open' that happens to be atomic.
# This file opener *must* be used in a "with" block.
class AtomicOpen:
# Open the file with arguments provided by user. Then acquire
# a lock on that file object (WARNING: Advisory locking).
def __init__(self, path, *args, **kwargs):
# Open the file and acquire a lock on the file before operating
self.file = open(path,*args, **kwargs)
# Lock the opened file
lock_file(self.file)
# Return the opened file object (knowing a lock has been obtained).
def __enter__(self, *args, **kwargs): return self.file
# Unlock the file and close the file object.
def __exit__(self, exc_type=None, exc_value=None, traceback=None):
# Flush to make sure all buffered contents are written to file.
self.file.flush()
os.fsync(self.file.fileno())
# Release the lock on the file.
unlock_file(self.file)
self.file.close()
# Handle exceptions that may have come up during execution, by
# default any exceptions are raised to the user.
if (exc_type != None): return False
else: return True
现在,AtomicOpen可以在with块中使用,而在这里通常使用open语句。
警告:
如果在Windows上运行并且Python在调用exit之前崩溃,我不确定锁的行为是什么。 这里提供的锁定是建议的,而不是绝对的。所有潜在的竞争进程必须使用“AtomicOpen”类。 截至2020年11月9日,此代码仅锁定Posix系统上的可写文件。在发布之后的某一时刻,在此日期之前,使用fcntl是非法的。锁定只读文件。
推荐文章
- 证书验证失败:无法获得本地颁发者证书
- 当使用pip3安装包时,“Python中的ssl模块不可用”
- 无法切换Python与pyenv
- Python if not == vs if !=
- 如何从scikit-learn决策树中提取决策规则?
- 为什么在Mac OS X v10.9 (Mavericks)的终端中apt-get功能不起作用?
- 将旋转的xtick标签与各自的xtick对齐
- 为什么元组可以包含可变项?
- 如何合并字典的字典?
- 如何创建类属性?
- 不区分大小写的“in”
- 在Python中获取迭代器中的元素个数
- 解析日期字符串并更改格式
- 使用try和。Python中的if
- 如何在Python中获得所有直接子目录