是否有一个针对Ubuntu和/或CentOS的包,它有一个命令行工具,可以执行像foo //element@attribute filename.xml或foo //element@attribute < filename.xml这样的XPath一行程序,并逐行返回结果?

我正在寻找一些东西,这将允许我只是apt-get安装foo或yum安装foo,然后只是开箱即用,没有包装或其他必要的适应。

以下是一些很接近的例子:

Nokogiri。如果我写这个包装器,我可以用上面描述的方式调用包装器:

#!/usr/bin/ruby

require 'nokogiri'

Nokogiri::XML(STDIN).xpath(ARGV[0]).each do |row|
  puts row
end

XML:: XPath。将与此包装工作:

#!/usr/bin/perl

use strict;
use warnings;
use XML::XPath;

my $root = XML::XPath->new(ioref => 'STDIN');
for my $node ($root->find($ARGV[0])->get_nodelist) {
  print($node->getData, "\n");
}

xpath从XML:: xpath返回太多噪音,——NODE——和attribute = "value"。

来自XML::Twig的xml_grep不能处理不返回元素的表达式,因此不能在没有进一步处理的情况下用于提取属性值。

编辑:

Echo cat //element/@attribute | xmllint——shell filename.xml返回类似xpath的噪声。

Xmllint——xpath //element/@attribute filename.xml返回attribute = "value"。

xmllint——xpath 'string(//element/@attribute)' filename.xml返回我想要的,但只对第一个匹配。

对于另一个几乎可以满足这个问题的解决方案,下面是一个可以用于计算任意XPath表达式的XSLT(需要XSLT处理器中的dyn:evaluate支持):

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
    xmlns:dyn="http://exslt.org/dynamic" extension-element-prefixes="dyn">
  <xsl:output omit-xml-declaration="yes" indent="no" method="text"/>
  <xsl:template match="/">
    <xsl:for-each select="dyn:evaluate($pattern)">
      <xsl:value-of select="dyn:evaluate($value)"/>
      <xsl:value-of select="'&#10;'"/>
    </xsl:for-each> 
  </xsl:template>
</xsl:stylesheet>

使用xsltproc——stringparam pattern //element/@attribute——stringparam value运行。arbitrary-xpath。xslt filename.xml。


当前回答

下面是一个xmlstarlet用例,从这种类型的XML中从嵌套元素elem1、elem2到一行文本中提取数据(还展示了如何处理名称空间):

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<mydoctype xmlns="http://xml-namespace-uri" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xml-namespace-uri http://xsd-uri" format="20171221A" date="2018-05-15">

  <elem1 time="0.586" length="10.586">
      <elem2 value="cue-in" type="outro" />
  </elem1>

</mydoctype>

输出将是

0.586 10.586 cue-in outro

在这个代码片段中,-m匹配嵌套的elem2, -v输出属性值(带有表达式和相对寻址),-o文字文本,-n添加换行符:

xml sel -N ns="http://xml-namespace-uri" -t -m '//ns:elem1/ns:elem2' \
 -v ../@time -o " " -v '../@time + ../@length' -o " " -v @value -o " " -v @type -n file.xml

如果elem1需要更多的属性,可以这样做(也显示concat()函数):

xml sel -N ns="http://xml-namespace-uri" -t -m '//ns:elem1/ns:elem2/..' \
 -v 'concat(@time, " ", @time + @length, " ", ns:elem2/@value, " ", ns:elem2/@type)' -n file.xml

请注意名称空间的复杂性(ns,用-N声明),这让我几乎放弃了xpath和xmlstarlet,而编写了一个快速的临时转换器。

其他回答

你也可以试试我的Xidel。它不在存储库中的包中,但您可以从网页下载它(它没有依赖关系)。

对于这个任务,它有简单的语法:

xidel filename.xml -e '//element/@attribute' 

它是少数支持XPath 2的工具之一。

clacke的回答很好,但我认为只有当你的源代码是格式良好的XML,而不是普通的HTML时才有效。

因此,对于正常的Web内容- html文档(不一定是格式良好的XML),也要做同样的事情:

echo "<p>foo<div>bar</div><p>baz" | python -c "from sys import stdin; \
from lxml import html; \
print '\n'.join(html.tostring(node) for node in html.parse(stdin).xpath('//p'))"

而使用html5lib(以确保您获得与Web浏览器相同的解析行为——因为像浏览器解析器一样,html5lib符合HTML规范中的解析要求)。

echo "<p>foo<div>bar</div><p>baz" | python -c "from sys import stdin; \
import html5lib; from lxml import html; \
doc = html5lib.parse(stdin, treebuilder='lxml', namespaceHTMLElements=False); \
print '\n'.join(html.tostring(node) for node in doc.xpath('//p'))

一个解决方案,即使存在命名空间声明的顶部:

如果xml在顶部声明了名称空间,答案中提出的大多数命令都不能开箱即用。考虑一下:

输入xml:

<elem1 xmlns="urn:x" xmlns:prefix="urn:y">
    <elem2 attr1="false" attr2="value2">
        elem2 value
    </elem2>
    <elem2 attr1="true" attr2="value2.1">
        elem2.1 value
    </elem2>    
    <prefix:elem3>
        elem3 value
    </prefix:elem3>        
</elem1>

不工作:

xmlstarlet sel -t -v "/elem1" input.xml
# nothing printed
xmllint -xpath "/elem1" input.xml
# XPath set is empty

解决方案:

# Requires >=java11 to run like below (but the code requires >=java17 for case syntax to be recognized)

# Prints the whole document
java ExtractXpath.java "/" example-inputs/input.xml

# Prints the contents and self of "elem1"
java ExtractXpath.java "/elem1" input.xml

# Prints the contents and self of "elem2" whose attr2 value is: 'value2'
java ExtractXpath.java "//elem2[@attr2='value2']" input.xml

# Prints the value of the attribute 'attr2': "value2", "value2.1"
java ExtractXpath.java "/elem1/elem2/@attr2" input.xml

# Prints the text inside elem3: "elem3 value"
java ExtractXpath.java "/elem1/elem3/text()" input.xml

# Prints the name of the matched element: "prefix:elem3"
java ExtractXpath.java "name(/elem1/elem3)" input.xml
# Same as above: "prefix:elem3"
java ExtractXpath.java "name(*/elem3)" input.xml

# Prints the count of the matched elements: 2.0
java ExtractXpath.java "count(/elem2)" input.xml


# known issue: while "//elem2" works. "//elem3" does not (it works only with: '*/elem3' )


ExtractXpath.java:


import java.io.File;
import java.io.FileInputStream;
import java.io.StringWriter;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.stream.Collectors;

import javax.xml.XMLConstants;
import javax.xml.namespace.NamespaceContext;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerConfigurationException;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathEvaluationResult;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

public class ExtractXpath {

    public static void main(String[] args) throws Exception {
        assertThat(args.length==2, "Wrong number of args");
        String xpath = args[0];
        File file = new File(args[1]);
             
        assertThat(file.isFile(), file.getAbsolutePath()+" is not a file.");
        FileInputStream fileIS = new FileInputStream(file);
        DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = builderFactory.newDocumentBuilder();
        Document xmlDocument = builder.parse(fileIS);
        XPath xPath = XPathFactory.newInstance().newXPath();
        String expression = xpath;
        XPathExpression xpathExpression =  xPath.compile(expression);
        
        XPathEvaluationResult xpathEvalResult =  xpathExpression.evaluateExpression(xmlDocument);
        System.out.println(applyXpathExpression(xmlDocument, xpathExpression, xpathEvalResult.type().name()));
    }

    private static String applyXpathExpression(Document xmlDocument, XPathExpression expr, String xpathTypeName) throws TransformerConfigurationException, TransformerException, XPathExpressionException {

        // see: https://www.w3.org/TR/1999/REC-xpath-19991116/#corelib
        List<String> retVal = new ArrayList();
        if(xpathTypeName.equals(XPathConstants.NODESET.getLocalPart())){ //e.g. xpath: /elem1/*
            NodeList nodeList = (NodeList)expr.evaluate(xmlDocument, XPathConstants.NODESET);
            for (int i = 0; i < nodeList.getLength(); i++) {
                retVal.add(convertNodeToString(nodeList.item(i)));
            }
        }else if(xpathTypeName.equals(XPathConstants.STRING.getLocalPart())){ //e.g. xpath: name(/elem1/*)
            retVal.add((String)expr.evaluate(xmlDocument, XPathConstants.STRING));
        }else if(xpathTypeName.equals(XPathConstants.NUMBER.getLocalPart())){ //e.g. xpath: count(/elem1/*)
            retVal.add(((Number)expr.evaluate(xmlDocument, XPathConstants.NUMBER)).toString());
        }else if(xpathTypeName.equals(XPathConstants.BOOLEAN.getLocalPart())){ //e.g. xpath: contains(elem1, 'sth')
            retVal.add(((Boolean)expr.evaluate(xmlDocument, XPathConstants.BOOLEAN)).toString());
        }else if(xpathTypeName.equals(XPathConstants.NODE.getLocalPart())){ //e.g. xpath: fixme: find one
            System.err.println("WARNING found xpathTypeName=NODE");
            retVal.add(convertNodeToString((Node)expr.evaluate(xmlDocument, XPathConstants.NODE)));
        }else{
            throw new RuntimeException("Unexpected xpath type name: "+xpathTypeName+". This should normally not happen");
        }
        return retVal.stream().map(str->"==MATCH_START==\n"+str+"\n==MATCH_END==").collect(Collectors.joining ("\n"));
        
    }
    
    private static String convertNodeToString(Node node) throws TransformerConfigurationException, TransformerException {
            short nType = node.getNodeType();
        switch (nType) {
            case Node.ATTRIBUTE_NODE , Node.TEXT_NODE -> {
                return node.getNodeValue();
            }
            case Node.ELEMENT_NODE, Node.DOCUMENT_NODE -> {
                StringWriter writer = new StringWriter();
                Transformer trans = TransformerFactory.newInstance().newTransformer();
                trans.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
                trans.setOutputProperty(OutputKeys.INDENT, "yes");
                trans.transform(new DOMSource(node), new StreamResult(writer));
                return writer.toString();
            }
            default -> {
                System.err.println("WARNING: FIXME: Node type:"+nType+" could possibly be handled in a better way.");
                return node.getNodeValue();
            }
                
        }
    }

    
    private static void assertThat(boolean b, String msg) {
        if(!b){
            System.err.println(msg+"\n\nUSAGE: program xpath xmlFile");
            System.exit(-1);
        }
    }
}

@SuppressWarnings("unchecked")
class NamespaceResolver implements NamespaceContext {
    //Store the source document to search the namespaces
    private final Document sourceDocument;
    public NamespaceResolver(Document document) {
        sourceDocument = document;
    }

    //The lookup for the namespace uris is delegated to the stored document.
    @Override
    public String getNamespaceURI(String prefix) {
        if (prefix.equals(XMLConstants.DEFAULT_NS_PREFIX)) {
            return sourceDocument.lookupNamespaceURI(null);
        } else {
            return sourceDocument.lookupNamespaceURI(prefix);
        }
    }

    @Override
    public String getPrefix(String namespaceURI) {
        return sourceDocument.lookupPrefix(namespaceURI);
    }

    @SuppressWarnings("rawtypes")
    @Override
    public Iterator getPrefixes(String namespaceURI) {
        return null;
    }
}

为了简单起见:

xpath-extract命令:

#!/bin/bash
java ExtractXpath.java "$1" "$2"

系统上很可能已经安装了python-lxml包。如果是这样,这是可能的,无需安装任何额外的包:

python -c "from lxml.etree import parse; from sys import stdin; print('\n'.join(parse(stdin).xpath('//element/@attribute')))"

除了XML::XSH和XML::XSH2之外,还有一些类似grep的实用程序,如App::xml_grep2和XML::Twig(包括xml_grep而不是xml_grep2)。在处理大型或大量XML文件以实现快速联机程序或Makefile目标时,这些功能非常有用。当您想要比$SHELL和xmllint xstlproc提供更多的处理时,XML::Twig特别适合用于perl脚本方法。

应用程序名称中的编号方案表明“2”版本是本质上相同工具的更新/更新版本,可能需要其他模块(或perl本身)的更新版本。