Out-File似乎在使用UTF-8时强制BOM:

$MyFile = Get-Content $MyPath
$MyFile | Out-File -Encoding "UTF8" $MyPath

我怎么能写一个文件在UTF-8没有BOM使用PowerShell?

更新2021

自从10年前我写这个问题以来,PowerShell已经发生了一些变化。检查下面的多个答案,它们有很多有用的信息!


当前回答

注意:这个答案适用于Windows PowerShell;相比之下,在跨平台的PowerShell核心版(v6+)中,在所有cmdlet中,不含BOM的UTF-8是默认编码。

换句话说:如果您使用PowerShell [Core]版本6或更高版本,默认情况下您将获得无bom的UTF-8文件(您也可以显式地使用-Encoding utf8 / -Encoding utf8NoBOM请求,而使用-utf8BOM获得- bom编码)。 如果你正在运行Windows 10,并且你愿意在系统范围内切换到无bom的UTF-8编码——这可能会有副作用——甚至Windows PowerShell也可以一直使用无bom的UTF-8编码——请看这个答案。


为了补充M. Dudley自己简单而务实的回答(以及ForNeVeR更简洁的重新表述):

A simple, (non-streaming) PowerShell-native alternative is to use New-Item, which (curiously) creates BOM-less UTF-8 files by default even in Windows PowerShell: # Note the use of -Raw to read the file as a whole. # Unlike with Set-Content / Out-File *no* trailing newline is appended. $null = New-Item -Force $MyPath -Value (Get-Content -Raw $MyPath) Note: To save the output from arbitrary commands in the same format as Out-File would, pipe to Out-String first; e.g.: $null = New-Item -Force Out.txt -Value (Get-ChildItem | Out-String) For convenience, below is advanced function Out-FileUtf8NoBom, a pipeline-based alternative that mimics Out-File, which means: you can use it just like Out-File in a pipeline. input objects that aren't strings are formatted as they would be if you sent them to the console, just like with Out-File. an additional -UseLF switch allows you use Unix-format LF-only newlines ("`n") instead of the Windows-format CRLF newlines ("`r`n") you normally get.

例子:

(Get-Content $MyPath) | Out-FileUtf8NoBom $MyPath # Add -UseLF for Unix newlines

注意(Get-Content $MyPath)是如何包含在(…)中,这确保在通过管道发送结果之前打开、完整读取和关闭整个文件。为了能够写回相同的文件(在适当的位置更新它),这是必要的。 一般来说,这种技术是不可取的,原因有二:(a)整个文件必须适合内存;(b)如果命令被中断,数据将丢失。

内存使用注意事项:

达德利先生自己的回答 和上面的New-Item替代方案要求首先在内存中构建整个文件内容,这对于大的输入集可能是个问题。 下面的函数不需要这样做,因为它是作为一个代理(包装器)函数实现的(关于如何定义这样的函数的简明摘要,请参阅这个答案)。


Out-FileUtf8NoBom函数源代码:

注意:该功能也可以作为麻省理工学院授权的Gist使用,并且今后只会维护它。

你可以直接使用以下命令安装它(虽然我个人可以向你保证这样做是安全的,但在直接执行脚本之前,你应该总是检查脚本的内容):

# Download and define the function.
irm https://gist.github.com/mklement0/8689b9b5123a9ba11df7214f82a673be/raw/Out-FileUtf8NoBom.ps1 | iex
function Out-FileUtf8NoBom {

  <#
  .SYNOPSIS
    Outputs to a UTF-8-encoded file *without a BOM* (byte-order mark).

  .DESCRIPTION

    Mimics the most important aspects of Out-File:
      * Input objects are sent to Out-String first.
      * -Append allows you to append to an existing file, -NoClobber prevents
        overwriting of an existing file.
      * -Width allows you to specify the line width for the text representations
        of input objects that aren't strings.
    However, it is not a complete implementation of all Out-File parameters:
      * Only a literal output path is supported, and only as a parameter.
      * -Force is not supported.
      * Conversely, an extra -UseLF switch is supported for using LF-only newlines.

  .NOTES
    The raison d'être for this advanced function is that Windows PowerShell
    lacks the ability to write UTF-8 files without a BOM: using -Encoding UTF8 
    invariably prepends a BOM.

    Copyright (c) 2017, 2022 Michael Klement <mklement0@gmail.com> (http://same2u.net), 
    released under the [MIT license](https://spdx.org/licenses/MIT#licenseText).

  #>

  [CmdletBinding(PositionalBinding=$false)]
  param(
    [Parameter(Mandatory, Position = 0)] [string] $LiteralPath,
    [switch] $Append,
    [switch] $NoClobber,
    [AllowNull()] [int] $Width,
    [switch] $UseLF,
    [Parameter(ValueFromPipeline)] $InputObject
  )

  begin {

    # Convert the input path to a full one, since .NET's working dir. usually
    # differs from PowerShell's.
    $dir = Split-Path -LiteralPath $LiteralPath
    if ($dir) { $dir = Convert-Path -ErrorAction Stop -LiteralPath $dir } else { $dir = $pwd.ProviderPath }
    $LiteralPath = [IO.Path]::Combine($dir, [IO.Path]::GetFileName($LiteralPath))
    
    # If -NoClobber was specified, throw an exception if the target file already
    # exists.
    if ($NoClobber -and (Test-Path $LiteralPath)) {
      Throw [IO.IOException] "The file '$LiteralPath' already exists."
    }
    
    # Create a StreamWriter object.
    # Note that we take advantage of the fact that the StreamWriter class by default:
    # - uses UTF-8 encoding
    # - without a BOM.
    $sw = New-Object System.IO.StreamWriter $LiteralPath, $Append
    
    $htOutStringArgs = @{}
    if ($Width) { $htOutStringArgs += @{ Width = $Width } }

    try { 
      # Create the script block with the command to use in the steppable pipeline.
      $scriptCmd = { 
        & Microsoft.PowerShell.Utility\Out-String -Stream @htOutStringArgs | 
          . { process { if ($UseLF) { $sw.Write(($_ + "`n")) } else { $sw.WriteLine($_) } } }
      }  
      
      $steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
      $steppablePipeline.Begin($PSCmdlet)
    }
    catch { throw }

  }

  process
  {
    $steppablePipeline.Process($_)
  }

  end {
    $steppablePipeline.End()
    $sw.Dispose()
  }

}

其他回答

从版本6开始,powershell支持UTF8NoBOM编码用于设置内容和输出文件,甚至将其用作默认编码。

所以在上面的例子中,它应该是这样的:

$MyFile | Out-File -Encoding UTF8NoBOM $MyPath

使用.NET的UTF8Encoding类并将$False传递给构造函数似乎是可行的:

$MyRawString = Get-Content -Raw $MyPath
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($MyPath, $MyRawString, $Utf8NoBomEncoding)

老问题,新答案:

虽然“旧的”powershell写一个BOM,但新的平台不可知的变体确实表现不同:默认是“无BOM”,它可以通过switch配置:

-Encoding Specifies the type of encoding for the target file. The default value is utf8NoBOM. The acceptable values for this parameter are as follows: ascii: Uses the encoding for the ASCII (7-bit) character set. bigendianunicode: Encodes in UTF-16 format using the big-endian byte order. oem: Uses the default encoding for MS-DOS and console programs. unicode: Encodes in UTF-16 format using the little-endian byte order. utf7: Encodes in UTF-7 format. utf8: Encodes in UTF-8 format. utf8BOM: Encodes in UTF-8 format with Byte Order Mark (BOM) utf8NoBOM: Encodes in UTF-8 format without Byte Order Mark (BOM) utf32: Encodes in UTF-32 format.

来源:https://learn.microsoft.com/de-de/powershell/module/Microsoft.PowerShell.Utility/Out-File?view=powershell-7 我特别强调

我建议只使用Set-Content命令,不需要其他任何命令。

我系统中的powershell版本是:-

PS C:\Users\XXXXX> $PSVersionTable.PSVersion | fl


Major         : 5
Minor         : 1
Build         : 19041
Revision      : 1682
MajorRevision : 0
MinorRevision : 1682

PS C:\Users\XXXXX>

所以你需要跟随。

PS C:\Users\XXXXX> Get-Content .\Downloads\finddate.txt
Thursday, June 23, 2022 5:57:59 PM
PS C:\Users\XXXXX> Get-Content .\Downloads\finddate.txt | Set-Content .\Downloads\anotherfile.txt
PS C:\Users\XXXXX> Get-Content .\Downloads\anotherfile.txt
Thursday, June 23, 2022 5:57:59 PM
PS C:\Users\XXXXX>

现在,当我们检查文件,根据截图,它是utf8。 anotherfile.txt

使用该方法编辑UTF8-NoBOM文件,生成编码正确的文件-

$fileD = "file.xml"
(Get-Content $fileD) | ForEach-Object { $_ -replace 'replace text',"new text" } | out-file "file.xml" -encoding ASCII

起初我对这种方法持怀疑态度,但它让我感到惊讶,而且很有效!

使用powershell 5.1版进行测试