我试图写一个bash脚本测试,需要一个参数,并通过curl发送到网站。我需要url编码的值,以确保特殊字符被正确处理。最好的方法是什么?

这是我到目前为止的基本脚本:

#!/bin/bash
host=${1:?'bad host'}
value=$2
shift
shift
curl -v -d "param=${value}" http://${host}/somepath $@

当前回答

安装php后,我使用这种方式:

URL_ENCODED_DATA=`php -r "echo urlencode('$DATA');"`

其他回答

对于那些正在寻找不需要perl的解决方案的人来说,这里有一个只需要hexdump和awk的解决方案:

url_encode() {
 [ $# -lt 1 ] && { return; }

 encodedurl="$1";

 # make sure hexdump exists, if not, just give back the url
 [ ! -x "/usr/bin/hexdump" ] && { return; }

 encodedurl=`
   echo $encodedurl | hexdump -v -e '1/1 "%02x\t"' -e '1/1 "%_c\n"' |
   LANG=C awk '
     $1 == "20"                    { printf("%s",   "+"); next } # space becomes plus
     $1 ~  /0[adAD]/               {                      next } # strip newlines
     $2 ~  /^[a-zA-Z0-9.*()\/-]$/  { printf("%s",   $2);  next } # pass through what we can
                                   { printf("%%%s", $1)        } # take hex value of everything else
   '`
}

通过网络上的几个地方和当地的一些试验和错误拼接在一起。效果很好!

其中一种变体可能很丑,但很简单:

urlencode() {
    local data
    if [[ $# != 1 ]]; then
        echo "Usage: $0 string-to-urlencode"
        return 1
    fi
    data="$(curl -s -o /dev/null -w %{url_effective} --get --data-urlencode "$1" "")"
    if [[ $? != 3 ]]; then
        echo "Unexpected error" 1>&2
        return 2
    fi
    echo "${data##/?}"
    return 0
}

下面是一个单行版本的例子(由Bruno建议):

date | curl -Gso /dev/null -w %{url_effective} --data-urlencode @- "" | cut -c 3-

# If you experience the trailing %0A, use
date | curl -Gso /dev/null -w %{url_effective} --data-urlencode @- "" | sed -E 's/..(.*).../\1/'

This is a simpler pure bash/ksh version without the substring logic. Stated differently the other pure shell solutions reparsed the string to get each character (using parameter substitution ${#str} for the lenght and ${str:$i:1} to discover each character). The below method does just one loop over the string to process each character. It is the difference between O(n^2) and O(n). In this answer: https://stackoverflow.com/a/40833433/1344599 Thunderbeef saw ~150x speed improvement on a large text file. This solution is also a shorter oneliner:

while IFS='' read -n 1 c ; do [[ "$c" =~ [A-Za-z0-9.~_-] ]] && printf "$c" || printf '%%%02X' "'$c" ; done

在函数中,你可以使用stdin或形参:

function urlen_stdin {
  while IFS='' read -n 1 c ; do [[ "$c" =~ [A-Za-z0-9.~_-] ]] && printf "$c" || printf '%%%02X' "'$c" ; done
}
function urlen_param {
  printf '%s' "$1" | while IFS='' read -n 1 c ; do [[ "$c" =~ [A-Za-z0-9.~_-] ]] && printf "$c" || printf '%%%02X' "'$c" ; done
}
function urlen_here {
  while IFS='' read -n 1 c ; do [[ "$c" =~ [A-Za-z0-9.~_-] ]] && printf "$c" || printf '%%%02X' "'$c" ; done <<< "$1"
}

#usage: 
echo -n 'hello !@#$%^&*()[]:;{}\/|-_=+.,? world' | urlen_stdin
urlen_param 'hello !@#$%^&*()[]:;{}\/|-_=+.,? world'
urlen_here 'hello !@#$%^&*()[]:;{}\/|-_=+.,? world'
# all methods render:
hello%20%21%40%23%24%25%5E%26%2A%28%29%5B%5D%3A%3B%7B%7D%2F%7C-_%3D%2B.%2C%3F%20world

解释:

IFS= "使空格像普通字符一样 Read -n 1一次读取1个字符 [[=~]]是一个正则表达式比较。如果字符匹配,则遵循&&路径,否则遵循||路径 printf '%%%02X'打印一个%和字符作为零填充长度为2的十六进制代码

另一个选择是使用jq:

$ printf %s 'input text'|jq -sRr @uri
input%20text
$ jq -rn --arg x 'input text' '$x|@uri'
input%20text

-r(——raw-output)输出字符串的原始内容,而不是JSON字符串字面量。-n(——null-input)不从STDIN读取输入。

-R(——raw-input)将输入行视为字符串,而不是将其解析为JSON,而-sR(——slurp——raw-input)将输入读入单个字符串。如果你的输入只包含一行,或者你不想用%0A替换换行符,你可以用-Rr替换-sRr:

$ printf %s\\n multiple\ lines of\ text|jq -Rr @uri
multiple%20lines
of%20text
$ printf %s\\n multiple\ lines of\ text|jq -sRr @uri
multiple%20lines%0Aof%20text%0A

或者这个百分比编码所有字节:

xxd -p|tr -d \\n|sed 's/../%&/g'

这是节点版本:

uriencode() {
  node -p "encodeURIComponent('${1//\'/\\\'}')"
}