By Ramon


2009-11-06 22:07:00 8 Comments

I know **/*.ext expands to all files in all subdirectories matching *.ext, but what is a similar expansion that includes all such files in the current directory as well?

5 comments

@Paused until further notice. 2009-11-06 23:22:49

This will work in Bash 4:

ls -l {,**/}*.ext

In order for the double-asterisk glob to work, the globstar option needs to be set (default: on):

shopt -s globstar

From man bash:

    globstar
                  If set, the pattern ** used in a filename expansion con‐
                  text will match a files and zero or more directories and
                  subdirectories.  If the pattern is followed by a /, only
                  directories and subdirectories match.

Now I'm wondering if there might have once been a bug in globstar processing, because now using simply ls **/*.ext I'm getting correct results.

Regardless, I looked at the analysis kenorb did using the VLC repository and found some problems with that analysis and in my answer immediately above:

The comparisons to the output of the find command are invalid since specifying -type f doesn't include other file types (directories in particular) and the ls commands listed likely do. Also, one of the commands listed, ls -1 {,**/}*.* - which would seem to be based on mine above, only outputs names that include a dot for those files that are in subdirectories. The OP's question and my answer include a dot since what is being sought is files with a specific extension.

Most importantly, however, is that there is a special issue using the ls command with the globstar pattern **. Many duplicates arise since the pattern is expanded by Bash to all file names (and directory names) in the tree being examined. Subsequent to the expansion the ls command lists each of them and their contents if they are directories.

Example:

In our current directory is the subdirectory A and its contents:

A
└── AB
    └── ABC
        ├── ABC1
        ├── ABC2
        └── ABCD
            └── ABCD1

In that tree, ** expands to "A A/AB A/AB/ABC A/AB/ABC/ABC1 A/AB/ABC/ABC2 A/AB/ABC/ABCD A/AB/ABC/ABCD/ABCD1" (7 entries). If you do echo ** that's the exact output you'd get and each entry is represented once. However, if you do ls ** it's going to output a listing of each of those entries. So essentially it does ls A followed by ls A/AB, etc., so A/AB gets shown twice. Also, ls is going to set each subdirectory's output apart:

...
<blank line>
directory name:
content-item
content-item

So using wc -l counts all those blank lines and directory name section headings which throws off the count even farther.

This a yet another reason why you should not parse ls.

As a result of this further analysis, I recommend not using the globstar pattern in any circumstance other than iterating over a tree of files in this manner:

for entry in **
do
    something "$entry"
done

As a final comparison, I used a Bash source repository I had handy and did this:

shopt -s globstar dotglob
diff <(echo ** | tr ' ' '\n') <(find . | sed 's|\./||' | sort)
0a1
> .

I used tr to change spaces to newlines which is only valid here since no names include spaces. I used sed to remove the leading ./ from each line of output from find. I sorted the output of find since it is normally unsorted and Bash's expansion of globs is already sorted. As you can see, the only output from diff was the current directory . output by find. When I did ls ** | wc -l the output had almost twice as many lines.

@Steven Penny 2013-07-01 08:17:12

I tested Ubuntu and Cygwin, and globstar is defaulted off

@gniourf_gniourf 2013-07-01 20:49:04

The best answer! but I think **/*.ext should be enough though. Also, you won't have the hidden files unless you shopt -s dotglob.

@toxalot 2014-03-10 09:29:41

My Centos 6 Bash 4.1.2 also defaults to off for globstar option.

@kenorb 2015-04-18 21:48:15

To disable globstar: shopt -u globstar.

@msciwoj 2015-08-27 14:17:35

@gniourf_gniourf The question actually asks to include the current directory specifically so no, **/*.ext won't be enough

@Benjamin W. 2017-11-04 17:41:00

@msciwoj But **/*.ext does include files in the current directory. The quoted manual snippet even says "zero or more directories and subdirectories".

@dotnetCarpenter 2018-09-04 11:27:26

@BenjaminW. I guess it depends on your config. On my current macOS machine it does not include files in the current directory.

@dotnetCarpenter 2018-09-04 11:29:32

However the following works rename 's/\.js$/\.mjs/' {*,**/*}

@Benjamin W. 2018-09-04 13:42:14

@dotnetCarpenter It looks like ** not matching the current directory was a bug that exists in a few versions of Bash: lists.gnu.org/archive/html/bug-bash/2009-05/msg00062.html

@dotnetCarpenter 2018-09-04 18:27:54

It also seems like the bash version that ships with macOS is handicapped. {*,**/*} only matches current directory plus child directories but not children of these child directories. and the shopt list doesn't have globstar. I've tried the bash shell that comes with homebrew (4.4.23(1)) with globstar on and it does the same with my glob pattern.

@Paused until further notice. 2018-09-04 18:34:33

@dotnetCarpenter: The version of Bash that ships with MacOS is 3.2 which doesn't support globstar, as you found out. A double asterisk is treated the same as a single one. Globstar was introduced in Bash 4.0.

@clone206 2018-05-09 23:30:36

Why not just use brace expansion to include the current directory as well?

./{*,**/*}.ext

Brace expansion happens before glob expansion, so you can effectively do what you want with older versions of bash, and can forego monkeying with globstar in newer versions.

Also, it's considered good practice in bash to include the leading ./ in your glob patterns.

@kenorb 2015-04-19 14:28:00

You can use: **/*.* to include all files recursively (enable by: shopt -s globstar).

Please find below testing of other variations and how they behave.


Testing folder with 3472 files in the sample VLC repository folder:

(Total files of 3472 counted as per: find . -type f | wc -l)

  • ls -1 **/*.* - returns 3338
  • ls -1 {,**/}*.* - returns 3341 (as proposed by Dennis)
  • ls -1 {,**/}* - returns 8265
  • ls -1 **/* - returns 7817, except hidden files (as proposed by Dennis)
  • ls -1 **/{.[^.],}* - returns 7869 (as proposed by Dennis)
  • ls -1 {,**/}.?* - returns 15855
  • ls -1 {,**/}.* - returns 20321

So I think the most closest method to list all files recursively is the first example (**/*.*) as per gniourf-gniourf comment (assuming the files have the proper extensions, or use the specific one), as the second example gives few more duplicates like below:

$ diff -u <(ls -1 {,**/}*.*) <(ls -1 **/*.*)
--- /dev/fd/63  2015-04-19 15:25:07.000000000 +0100
+++ /dev/fd/62  2015-04-19 15:25:07.000000000 +0100
@@ -1,6 +1,4 @@
 COPYING.LIB
-COPYING.LIB
-Makefile.am
 Makefile.am
@@ -45,7 +43,6 @@
 compat/tdestroy.c
 compat/vasprintf.c
 configure.ac
-configure.ac

and the other generate even further duplicates.


To include hidden files, use: shopt -s dotglob (disable by shopt -u dotglob). It's not recommended, because it can affect commands such as mv or rm and you can remove accidentally the wrong files.

@mummybot 2017-01-16 10:16:48

On Mac terminal and bash with globstar enabled, I found the above solution (**/*.*) informative and worked best. The accepted answer caused duplicates of items in the top directory. My working pattern was: "${path}"**/*.*

@Wilf 2018-07-08 22:24:31

It would be interesting to try this with other options like nullglob and dotglob

@Amir Afghani 2009-11-06 22:10:47

$ find . -type f

That will list all of the files in the current directory. You can then do some other command on the output using -exec

$find . -type f -exec grep "foo" {} \;

That will grep each file from the find for the string "foo".

@unutbu 2009-11-06 22:11:25

This wil print all files in the current directory and its subdirectories which end in '.ext'.

find . -name '*.ext' -print

@Paused until further notice. 2019-02-21 02:34:59

While this answer doesn't meet the OP's requested "expansion" in the strictest sense, it is most likely to produce the desired outcome.

Related Questions

Sponsored Content

26 Answered Questions

[SOLVED] Shell command to tar directory excluding certain files/folders

43 Answered Questions

[SOLVED] How to count all the lines of code in a directory recursively?

  • 2009-08-31 17:42:20
  • user77413
  • 764118 View
  • 1586 Score
  • 43 Answer
  • Tags:   bash shell

30 Answered Questions

[SOLVED] Why doesn't "cd" work in a shell script?

  • 2008-11-01 02:07:08
  • ashokgelal
  • 616839 View
  • 756 Score
  • 30 Answer
  • Tags:   linux shell

16 Answered Questions

10 Answered Questions

[SOLVED] Recursively look for files with a specific extension

13 Answered Questions

[SOLVED] Get a filtered list of files in a directory

20 Answered Questions

[SOLVED] Get current directory name (without full path) in a Bash script

  • 2009-09-03 03:11:53
  • Derek Dahmer
  • 658778 View
  • 799 Score
  • 20 Answer
  • Tags:   bash shell

6 Answered Questions

20 Answered Questions

[SOLVED] How to recursively find the latest modified file in a directory?

22 Answered Questions

[SOLVED] Use grep --exclude/--include syntax to not grep through certain files

Sponsored Content