我试图使用sed来清理url行来提取域。
所以从:
http://www.suepearson.co.uk/product/174/71/3816/
我想要:
http://www.suepearson.co.uk/
(不管后面有没有斜杠,都没有关系)
我试过:
sed 's|\(http:\/\/.*?\/\).*|\1|'
And(转义非贪婪量词)
sed 's|\(http:\/\/.*\?\/\).*|\1|'
但我似乎不能让非贪婪量词(?)工作,所以它总是匹配整个字符串。
使用sed,我通常通过搜索除分隔符以外的任何东西来实现非贪婪搜索,直到分隔符:
echo "http://www.suon.co.uk/product/1/7/3/" | sed -n 's;\(http://[^/]*\)/.*;\1;p'
输出:
http://www.suon.co.uk
这是:
don't output -n
search, match pattern, replace and print s/<pattern>/<replace>/p
use ; search command separator instead of / to make it easier to type so s;<pattern>;<replace>;p
remember match between brackets \( ... \), later accessible with \1,\2...
match http://
followed by anything in brackets [], [ab/] would mean either a or b or /
first ^ in [] means not, so followed by anything but the thing in the []
so [^/] means anything except / character
* is to repeat previous group so [^/]* means characters except /.
so far sed -n 's;\(http://[^/]*\) means search and remember http://followed by any characters except / and remember what you've found
we want to search untill the end of domain so stop on the next / so add another / at the end: sed -n 's;\(http://[^/]*\)/' but we want to match the rest of the line after the domain so add .*
now the match remembered in group 1 (\1) is the domain so replace matched line with stuff saved in group \1 and print: sed -n 's;\(http://[^/]*\)/.*;\1;p'
如果你想在域名后面加上反斜杠,那么在组中再加一个反斜杠来记住:
echo "http://www.suon.co.uk/product/1/7/3/" | sed -n 's;\(http://[^/]*/\).*;\1;p'
输出:
http://www.suon.co.uk/
在sed中模拟惰性(非贪婪)量词
以及所有其他正则表达式口味!
Finding first occurrence of an expression:
POSIX ERE (using -r option)
Regex:
(EXPRESSION).*|.
Sed:
sed -r 's/(EXPRESSION).*|./\1/g' # Global `g` modifier should be on
Example (finding first sequence of digits) Live demo:
$ sed -r 's/([0-9]+).*|./\1/g' <<< 'foo 12 bar 34'
12
How does it work?
This regex benefits from an alternation |. At each position engine tries to pick the longest match (this is a POSIX standard which is followed by couple of other engines as well) which means it goes with . until a match is found for ([0-9]+).*. But order is important too.
Since global flag is set, engine tries to continue matching character by character up to the end of input string or our target. As soon as the first and only capturing group of left side of alternation is matched (EXPRESSION) rest of line is consumed immediately as well .*. We now hold our value in the first capturing group.
POSIX BRE
Regex:
\(\(\(EXPRESSION\).*\)*.\)*
Sed:
sed 's/\(\(\(EXPRESSION\).*\)*.\)*/\3/'
Example (finding first sequence of digits):
$ sed 's/\(\(\([0-9]\{1,\}\).*\)*.\)*/\3/' <<< 'foo 12 bar 34'
12
This one is like ERE version but with no alternation involved. That's all. At each single position engine tries to match a digit.
If it is found, other following digits are consumed and captured and the rest of line is matched immediately otherwise since * means
more or zero it skips over second capturing group \(\([0-9]\{1,\}\).*\)* and arrives at a dot . to match a single character and this process continues.
Finding first occurrence of a delimited expression:
This approach will match the very first occurrence of a string that is delimited. We can call it a block of string.
sed 's/\(END-DELIMITER-EXPRESSION\).*/\1/; \
s/\(\(START-DELIMITER-EXPRESSION.*\)*.\)*/\1/g'
Input string:
foobar start block #1 end barfoo start block #2 end
-EDE: end
-SDE: start
$ sed 's/\(end\).*/\1/; s/\(\(start.*\)*.\)*/\1/g'
Output:
start block #1 end
First regex \(end\).* matches and captures first end delimiter end and substitues all match with recent captured characters which
is the end delimiter. At this stage our output is: foobar start block #1 end.
Then the result is passed to second regex \(\(start.*\)*.\)* that is same as POSIX BRE version above. It matches a single character
if start delimiter start is not matched otherwise it matches and captures the start delimiter and matches the rest of characters.
直接回答你的问题
使用方法#2(带分隔符的表达式),你应该选择两个合适的表达式:
艾德:[^]\ /
SDE: http:
用法:
$ sed 's/\([^:/]\/\).*/\1/g; s/\(\(http:.*\)*.\)*/\1/' <<< 'http://www.suepearson.co.uk/product/174/71/3816/'
输出:
http://www.suepearson.co.uk/
注意:对于相同的分隔符,这将不起作用。
这是如何使用sed健壮地进行多字符字符串的非贪婪匹配。假设你想改变每一个foo…Bar to <foo…Bar >,例如这个输入:
$ cat file
ABC foo DEF bar GHI foo KLM bar NOP foo QRS bar TUV
应该变成这样的输出:
ABC <foo DEF bar> GHI <foo KLM bar> NOP <foo QRS bar> TUV
要做到这一点,你将foo和bar转换为单独的字符,然后在它们之间使用这些字符的反字符:
$ sed 's/@/@A/g; s/{/@B/g; s/}/@C/g; s/foo/{/g; s/bar/}/g; s/{[^{}]*}/<&>/g; s/}/bar/g; s/{/foo/g; s/@C/}/g; s/@B/{/g; s/@A/@/g' file
ABC <foo DEF bar> GHI <foo KLM bar> NOP <foo QRS bar> TUV
在上述:
/ / @ @A / g;s / {/ @B / g;s/}/@C/g正在将{和}转换为输入中不存在的占位符字符串,这样这些字符就可以转换为foo和bar。
s / foo / {/ g;S /bar/}/g将foo和bar分别转换为{和}
S /{[^{}]*}/<&>/g正在执行我们想要的操作-将foo…Bar到<foo…Bar >
s /} /酒吧/ g;S /{/foo/g将{和}转换回foo和bar。
s / @C /} / g;s / @B / {/ g;s/@ a /@/g将占位符字符串转换回原始字符。
请注意,上面的方法并不依赖于输入中不存在的任何特定字符串,因为它在第一步中就制造了这样的字符串,它也不关心你想要匹配的任何特定regexp的哪个出现,因为你可以在表达式中使用{[^{}]*}尽可能多的次数来隔离你想要的实际匹配和/或使用seds数值匹配操作符,例如只替换第二个出现:
$ sed 's/@/@A/g; s/{/@B/g; s/}/@C/g; s/foo/{/g; s/bar/}/g; s/{[^{}]*}/<&>/2; s/}/bar/g; s/{/foo/g; s/@C/}/g; s/@B/{/g; s/@A/@/g' file
ABC foo DEF bar GHI <foo KLM bar> NOP foo QRS bar TUV
您还应该考虑没有匹配界限的情况。你是否想输出这一行。如果不匹配,我这里的示例不会输出任何内容。
你需要前缀到第三个/,所以选择两次字符串的任何长度不包含/和后面的/,然后字符串的任何长度不包含/,然后匹配/后面的任何字符串,然后打印选择。这个想法适用于任何单个的char delims。
echo http://www.suepearson.co.uk/product/174/71/3816/ | \
sed -nr 's,(([^/]*/){2}[^/]*)/.*,\1,p'
使用sed命令,您可以快速删除前缀或delim选择,如:
echo 'aaa @cee: { "foo":" @cee: " }' | \
sed -r 't x;s/ @cee: /\n/;D;:x'
这比一次吃焦肉快多了。
如果之前匹配成功,跳转到标签。在第一道线/之前加\n。移除到第一个\n。如果添加了\n,则跳转到结束并打印。
如果有开始和结束delim,很容易删除结束delim,直到你到达你想要的第n -2个元素,然后做D技巧,在结束delim后删除,如果不匹配跳转到删除,在开始delim和打印之前删除。这仅在开始/结束分隔成对出现时有效。
echo 'foobar start block #1 end barfoo start block #2 end bazfoo start block #3 end goo start block #4 end faa' | \
sed -r 't x;s/end//;s/end/\n/;D;:x;s/(end).*/\1/;T y;s/.*(start)/\1/;p;:y;d'
下面的解决方案适用于匹配/使用multiply present(链式;串联;复合)HTML或其他标签。例如,我想编辑HTML代码以删除串联出现的<span>标记。
问题:常规sed正则表达式贪婪地匹配从第一个到最后一个的所有标记。
解决方案:非贪婪模式匹配(每个讨论在这个线程的其他地方;例如,https://stackoverflow.com/a/46719361/1904943)。
例子:
echo '<span>Will</span>This <span>remove</span>will <span>this.</span>remain.' | \
sed 's/<span>[^>]*>//g' ; echo
This will remain.
解释:
S /<span>:查找<span>
[^>]:后面跟着不是>的任何东西
*>:直到你找到>
//g:将任何这样的字符串替换为空。
齿顶高
我试图清理url,但我遇到了困难匹配/排除一个词- href -使用上面的方法。我简要地研究了反向查找(正则表达式来匹配不包含单词的行),但这种方法似乎过于复杂,并没有提供令人满意的解决方案。
我决定用'(反标记)替换href,做正则表达式替换,然后用href替换'。
示例(为便于阅读,此处格式化):
printf '\n
<a aaa h href="apple">apple</a>
<a bbb "c=ccc" href="banana">banana</a>
<a class="gtm-content-click"
data-vars-link-text="nope"
data-vars-click-url="https://blablabla"
data-vars-event-category="story"
data-vars-sub-category="story"
data-vars-item="in_content_link"
data-vars-link-text
href="https:example.com">Example.com</a>\n\n' |
sed 's/href/`/g ;
s/<a[^`]*`/\n<a href/g'
<a href="apple">apple</a>
<a href="banana">banana</a>
<a href="https:example.com">Example.com</a>
解释:基本如上。在这里,
S /href/ ':将href替换为'(反引号)
s/<a:查找URL开头
[^ ']:后面跟着任何不是'(反撇号)
* ':直到你找到一个'
/<a href/g:用<a href替换每一个找到的
不幸的是,如前所述,sed不支持这一点。
为了克服这个问题,我建议使用次优方法(实际上甚至更好),使用类似vim sed的功能。
在.bash-profile中定义
vimdo() { vim $2 --not-a-term -c "$1" -es +"w >> /dev/stdout" -cq! ; }
这将创建无头vim来执行命令。
现在你可以这样做:
回声路径美元| vimdo“% s_ \ c: [a-zA-Z0-9 \ \ /] python (a-zA-Z0-9 \ {-} \\/]\{-}:__ g”,
过滤掉$PATH中的python。
使用-在vimdo中从管道中输入。
而大多数语法是相同的。Vim具有更高级的特性,并且使用\{-}是非贪婪匹配的标准。参见帮助regexp。