By David


2013-04-06 23:05:18 8 Comments

This is my problem. In bash 3:

$ test='One "This is two" Three'
$ set -- $test
$ echo $2
"This

How to get bash to understand the quotes and return $2 as This is two and not "This? Unfortunately I cannot alter the construction of the variable called test in this example.

5 comments

@Laurent Picquet 2020-04-28 07:18:13

test='One "This is two" Three'
mapfile -t some_args < <(xargs -n1 <<<"$test")
echo "'${some_args[0]}'" "'${some_args[1]}'" "'${some_args[2]}'"

output: 'One' 'This is two' 'Three'

@Grief 2016-03-29 10:25:13

Now we have bash 4 where it's possible to do something like that:

#!/bin/bash

function qs_parse() { 
    readarray -t "$1" < <( printf "%s" "$2"|xargs -n 1 printf "%s\n" ) 
}

tab='   '  # tabulation here
qs_parse test "One 'This is two' Three -n 'foo${tab}bar'"

printf "%s\n" "${test[0]}"
printf "%s\n" "${test[1]}"
printf "%s\n" "${test[2]}"
printf "%s\n" "${test[3]}"
printf "%s\n" "${test[4]}"

Outputs, as expected:

One
This is two
Three
-n
foo     bar  # tabulation saved

Actually, I am not sure but it's probably possible to do that in older bash like that:

function qs_parse() {
    local i=0
    while IFS='' read -r line || [[ -n "$line" ]]; do
        parsed_str[i]="${line}"
        let i++
    done < <( printf "%s\n" "$1"|xargs -n 1 printf "%s\n" )
}

tab='   ' # tabulation here
qs_parse "One 'This is two' Three -n 'foo${tab}bar'"

printf "%s\n" "${parsed_str[0]}"
printf "%s\n" "${parsed_str[1]}"
printf "%s\n" "${parsed_str[2]}"
printf "%s\n" "${parsed_str[3]}"
printf "%s\n" "${parsed_str[4]}"

@Charles Duffy 2016-09-20 15:51:11

The use of echo adds some bugs here, echo with an unquoted argument doubly so. If $t=$'\t'; qs_parse "One \"*\" -n \"tab${t}between${t}each${t}word\"", you don't want those tabs turned to spaces, or the -n treated as an argument to echo. Consider printf '%s\n' "$1" instead.

@Charles Duffy 2017-05-24 16:44:00

@Grief, would you accept an edit making the changes I proposed previously?

@Grief 2017-05-27 16:50:58

@CharlesDuffy sorry for not doing that earlier, but I've updated the answer. It appears that echo must be replaced with printf everywhere to make '-n' work. Hope you like the answer now. Thanks!

@user2350426 2016-05-23 03:23:46

The solution to this problem is to use xargs (eval free).
It retains double quoted strings together:

$ test='One "This is two" Three'
$ IFS=$'\n' arr=( $(xargs -n1 <<<"$test") )
$ printf '<%s>\n' "${arr[@]}"
<One>
<This is two>
<Three>

Of course, you can set the positional arguments with that array:

$ set -- "${arr[@]}"
$ echo "$2"
This is two

@Charles Duffy 2017-05-24 16:44:46

xargs is the right tool for the job, definitely. Unquoted expansion, not so much. If you have test='One "This is two" Three "*"', you don't want the * replaced with a list of filenames.

@RashaMatt 2014-02-26 22:54:48

I wrote a couple native bash functions to do this: https://github.com/mblais/bash_ParseFields

You can use the ParseFields function like this:

$ str='field1 field\ 2 "field 3"'
$ ParseFields -d "$str" a b c d
$ printf "|%s|\n|%s|\n|%s|\n|%s|\n" "$a" "$b" "$c" "$d"
|field1|         
|field 2|
|field 3|
||                

The -d option to ParseFields removes any surrounding quotes and interprets backslashes from the parsed fields.

There is also a simpler ParseField function (used by ParseFields) that parses a single field at a specific offset within a string.

Note that these functions cannot parse a stream, only a string. The IFS variable can also be used to specify field delimiters besides whitespace.

If you require that unescaped apostrophes may appear in unquoted fields, that would require a minor change - let me know.

@joran 2014-02-26 23:08:27

Answers that consist of little more than links are discouraged here. Please consider expanding on your answer to include some concrete examples.

@Gordon Davisson 2013-04-06 23:44:56

The reason this happens is because of the order in which the shell parses the command line: it parses (and removes) quotes and escapes, then replaces variable values. By the time $test gets replaced with One "This is two" Three, it's too late for the quotes to have their intended effect.

The simple (but dangerous) way to do this is by adding another level of parsing with eval:

$ test='One "This is two" Three'
$ eval "set -- $test"
$ echo "$2"
This is two

(Note that the quotes in the echo command are not necessary, but are a good general practice.)

The reason I say this is dangerous is that it doesn't just go back and reparse for quoted strings, it goes back and reparses everything, maybe including things you didn't want interpreted like command substitutions. Suppose you had set

$ test='One `rm /some/important/file` Three'

...eval will actually run the rm command. So if you can't count on the contents of $test to be "safe", do not use this construct.

BTW, the right way to do this sort of thing is with an array:

$ test=(One "This is two" Three)
$ set -- "${test[@]}"
$ echo "$2"
This is two

Unfortunately, this requires control of how the variable is created.

Related Questions

Sponsored Content

20 Answered Questions

[SOLVED] Reference — What does this symbol mean in PHP?

36 Answered Questions

[SOLVED] How do I parse command line arguments in Bash?

44 Answered Questions

[SOLVED] How do I convert a String to an int in Java?

33 Answered Questions

[SOLVED] How to check if a variable is set in Bash?

  • 2010-08-30 14:54:38
  • prosseek
  • 1364200 View
  • 1607 Score
  • 33 Answer
  • Tags:   bash shell variables

19 Answered Questions

[SOLVED] Convert bytes to a string

26 Answered Questions

[SOLVED] How to check if a string contains a substring in Bash

59 Answered Questions

[SOLVED] How do I read / convert an InputStream into a String in Java?

20 Answered Questions

[SOLVED] How to convert a string to lower case in Bash?

32 Answered Questions

[SOLVED] How do I split a string on a delimiter in Bash?

30 Answered Questions

[SOLVED] How to concatenate string variables in Bash

Sponsored Content