Path: content/Tips/Bash/default.md

Bash

Background tasks

For years I ran 'ps' to see what background tasks there are running (ie, started with & at the end). However, there is a much simpler way:

jobs -p

will list the process IDs of just the backgrounded tasks - ps shows the current shell and the ps as well which we are rarely interested in

Parallelise bash scripts

The obvious way is to use GNU Parallel, but another option is to define a little helper function like this:

waitforjobs() { while test $(jobs -p | wc -w) -ge "$1"; do wait -n; done }

and then call any new tasks with this:

waitforjobs 10 run-another-job &

where 10 is the number of concurrent processes you want to run.

Also if you are spawning jobs with xargs there is a -P option:

   -P max-procs, --max-procs=max-procs
          Run up to max-procs processes at a time; the default is 1.  If max-procs is 0,
          xargs will run as many processes as possible at a time.  Use the -n option or
          the -L option with  -P;  otherwise chances  are  that only one exec will be
          done.  While xargs is running, you can send its process a SIGUSR1 signal to
          increase the number of commands to run simultaneously, or a SIGUSR2 to decrease
          the number.  You cannot increase it above an implementation-defined limit (which
          is shown with --show-limits).

List files with comma separated size

export BLOCK_SIZE="'1"

hostname:~$ ls /var/log -l
total 98,152,448
-rw-r--r--  1 root   root    42,405 Sep 21 15:06 alternatives.log
drwxr-x---  2 root   adm      4,096 Sep 21 09:10 apache2
-rw-r-----  1 root   adm          0 Sep 22 07:58 apport.log
-rw-r-----  1 root   adm      1,538 Sep 21 10:33 apport.log.1

Sum all files older than 'n' days

eg all files > 365 days:

find . -type f -mtime +356 -printf '%s\n' | awk '{a+=$1;} END {printf "%.1f GB\n", a/2**30;}'

Extended Globbing File Matching

Running : shopt -s extglob allows you to match files with a few more methods, eg:

If the extglob shell option is enabled using the shopt builtin, several extended pattern matching operators are recognized. In the following description, a pattern-list is a list of one or more patterns separated by a |. Composite patterns may be formed using one or more of the following sub-patterns:

?(pattern-list)

Matches zero or one occurrence of the given patterns

*(pattern-list)

Matches zero or more occurrences of the given patterns

+(pattern-list)

Matches one or more occurrences of the given patterns

@(pattern-list)

Matches one of the given patterns

!(pattern-list)

Matches anything except one of the given patterns

So you can do ls !(*.tmp) to exclude all tmp files for example

Run shopt -u extglob to turn it off.

Command Recall

To recall previous commands on unix, press control-R and start typing. The most recent matching command will appear. Press control-R again to go back through the list.

The commands are stored into .bash_history when you log out. If you don't want it to store your previous commands you can: unset HIST_FILE

you can also set HISTSIZE and HISTFILESIZE to be more than the default 1000/2000 if needed.

ctrl-E goes to the end of the current line.

ctrl-A goes to the start.

ctrl-U deletes from the current position to the start of the line.