Tried to implement simple barrier in linux shell but fail

[ggodw000]

I think I am doing something wrong when I tried to implement simple barrier in linux shell script but seem to fail spectacularly.

Code: Select all

# some code block before launching scripts in parallel.
<snippet1>
...
for loop
    # should launch instances of each script1 in background since ampersand implied.
    launch another shell script in <script1> in background &
done

#barrier implementation, wait loop until all instances of script1 finished running.
for loop 
   count=`ps -ef | grep <script1>  | wc -l`
   if count -eq 0
      break
      fi
done
...
# some code block after launching scripts in parallel.
<snippet2>

The problem is I can see <snippet1> and all instances of <script1> is launched
But output of code block in <snippet2> appearing before the output of instances of <script1>.
Because there is a dependency where <script1> can not run after <snippet2> is executed it runs into all sort of errors.

[Octocontrabass]

Try replacing your barrier with this:

Code: Select all

wait

[bzt]

Code: Select all

count=`ps -ef | grep <script1>  | wc -l`

This always going to be larger than 0, because grep itself is included in ps output. (btw the flag "f" makes no sense here.)

Code: Select all

$ ps -ef | grep something
bzt       780842  147715  0 13:44 pts/0    00:00:00 grep something

This trick I've learned from an old UNIX guru:

Code: Select all

$ ps -ef | grep [s]omething

This will list all processes containing "something" except the grep The reason is, the command will be "grep omething", however [] is replaced by the shell, and "grep something" will actually be executed, that doesn't match "omething".

Btw, "wait" is a much better solution, I agree.

Cheers,
bzt

[nullplan]

This always going to be larger than 0, because grep itself is included in ps output. (btw the flag "f" makes no sense here.)

Not necessarily (and that can lead to some fun bugs): The shell will spawn all parts of the pipeline in parallel. So whether "grep" has been exec'd by the time ps gets to that process is not defined. There is no sequence that makes that a necessity, and indeed, in some situations ps fails to output the line with "grep" in it (if grep happens to get a very small PID and ps has already iterated over it by the time grep is exec'd). And so you end up with the grep line sometimes being in the output and sometimes not. Happy debugging!

The trick you showed is a good one, however. I already knew it, but it is a good one for newbies (regular expressions don't have to match themselves). Alternatively, you can filter the "grep" line afterwards with "grep -v grep". But for this particular problem, Octocontrabass was entirely correct: There is no need to iterate over all processes when you only need to wait for your child processes.

[ggodw000]

thanks I used wait and apparently working much better.