Posts Tagged hint

sed: Remove lines that match a pattern

sed  '/mypattern/d' file


Deletes all lines that contain the regular expression “mypattern”.

In-place replacement:

sed --in-place '/mypattern/d' file1


sed also supports making backups with e.g. –in-place=.backup .

No Comments

Shutdown when idle

Maybe you want the computer to shutdown after it has done its task. This shuts down your Linux box when the load goes down to zero. Kind of like the “Speed” movies.

while ! cat /proc/loadavg |cut -f2 -d ' '|grep '0\.00'; do sleep 10m; done && poweroff

Alternatively:

while ps 12345; do sleep 10m; done && poweroff

No Comments

Bash: Chain of commands

I wondered more than once how to make a tool that chains together input/output filters using bash.
e.g. you would like to do a iteration of filters:

for i in 1 2 3 4; do grep -v $i; done

So I wrote this chaining tool that does the job in the obvious way:

chain.sh

#!/bin/bash
commands_file=$1
a=$(mktemp)
b=$(mktemp)
cat > $a
while read line; do
$line < $a > $b
mv $b $a
done < $commands_file
cat $a
rm $a

With this, you can do:

for i in 1 2 3 4; do
echo grep -v $i >> mycommands.txt
done
script/chain.sh mycommands.txt

lang=”bash”

No Comments

make: self-documenting Makefiles

With the following grep/sed combinations, you can keep the help for each target inline just above your target (or wherever you want it).

## Makefile for foo

## help: this clutter

help:

        @grep -E '^## ' Makefile|grep -v ':'|sed 's,^## ,,g'

        @echo This Makefile has the targets:

        @grep -E '^## [.a-z]{2,}:' Makefile|sed 's,^## *,\t,g' |sed 's,: ,\t,g'

## tests: run the tests

tests: tests.exe

        ./tests.exe ${TESTNR}

Output of



$ make help

Makefile for foo

This Makefile has the targets:

	help	this clutter

	all	

	tests	run the tests

...

Nice, leet and meta. :-)

Some more words on Makefiles: You can use something like

CFLAGS=-O3 -Wall -Werror -Wextra -g -ansi -pedantic ${CCFLAGS}

and pass CCFLAGS from outside. This avoids having many many versions while trying to test things out. For example:

$ CCFLAGS="-DDEBUG -Os -DUSED_ALGORITHM=binsearch2" make tests

(if you didn’t notice yet, -DFOO lets you jump inside #ifdef FOO).

No Comments

Add collaborator to GitHub repository

Don’t let them clone the public url: It doesn’t go over ssh and I don’t know how to change the origin url in a branch. *

Steps:

  1. create ssh-key on new collaborators computer (ssh-keygen)
  2. create github user
  3. add public-key to github user account
  4. let the project owner add you as project member
  5. git clone git@…

Where 1 < 3, 1 < 2 < 4, 2 < 5 if that means anything to you ;-)

* it works with git remote

1 Comment

Autologin on websites

Some sites provide for their login one form that just contain the login form. Firefox fills it out with username+password, but you still have to aim at the submit button.
This greasemonkey-script provides auto-login for TUWEL, TUWIS and any other site you want that only provides one form. autologin.user (js, 1 KB)

No Comments

Maven: Colorized

Maven was written by people who obviously have no idea on how to create good tui (Text User Interface)-tools. They should sit together with the portage, git or other Linux-devs.
The output is unreadable (not to mention that the startup is slow).

So, what I did was wrap maven in a script that colorizes the output. The result looks like this:
colorizedmaven

This is how to do it:
create the file ~/bin/mvn with the following content (mvn (0 KB))

#!/bin/bash
/usr/bin/mvn $* | sed -e 's/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/Tests run: \1, Failures: \2, Errors: \3, Skipped: \4/g' \
-e 's/\(\[WARN\].*\)/\1/g' \
-e 's/\(\[INFO\].*\)/\1/g' \
-e 's/\(\[ERROR\].*\)/\1/g'

replace /usr/bin/mvn with wherever your mvn is (find out with which mvn).
Double-check that you get the encoding right, the weird sign should be hexadecimal 1b. Use a editor that supports encodings for copy+paste.
Give your ~/bin/mvn execute permissions (chmod +x ~/bin/mvn)

Try it out: call ~/bin/mvn in one of your maven projects

What is left to do is to register your new mvn-wrapper so it is called everytime.
Add the following line to your ~/.bashrc :

alias mvn=~/bin/mvn

Execute the command you just added in your running shells so you can use it there too.

You are done.

Changing the distributed mvn startup script

Even better, but a little more intrusive (you need root rights):
change the distributed mvn file. It is a bash script actually.
In gentoo:

 $ which mvn
/usr/bin/mvn
$ file /usr/bin/mvn
/usr/bin/mvn: symbolic link to `/usr/share/maven-bin-2.0/bin/mvn'
$ file /usr/share/maven-bin-2.0/bin/mvn
/usr/share/maven-bin-2.0/bin/mvn: Bourne shell script text executable
$ 

Change the last call in that file to:

exec "$JAVACMD" \
$MAVEN_OPTS \
-classpath "${M2_HOME}"/boot/classworlds-*.jar \
"-Dclassworlds.conf=${M2_HOME}/bin/m2.conf" \
"-Dmaven.home=${M2_HOME}"  \
${CLASSWORLDS_LAUNCHER} $QUOTED_ARGS |
sed -e 's/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/Tests run: \1, Failures: \2, Errors: \3, Skipped: \4/g' \
-e 's/\(\[WARN\].*\)/\1/g' \
-e 's/\(\[INFO\].*\)/\1/g' \
-e 's/\(\[ERROR\].*\)/\1/g'

That way you have it there directly. Maybe the gentoo devs could add a color use-flag. That’d be fun…

,

No Comments

Bash: Timeouts

You have a command and want to kill it after a certain time if it hasn’t finished.
This script does it for you:

timeoutwrapper.sh

#!/bin/bash
# License: BSD License
# author: Ram
# author: Johannes Buchner
TIMEOUT=$1
shift
COMMAND="$*"
if [[ "$TIMEOUT" == "" ]] || [[ "$COMMAND" == "" ]]; then
echo "USAGE: $0 <seconds to wait> <command with arguments>"
exit
fi
$COMMAND &
PRODPID=$!
export PRODPID
# record own PID
export PID=$$
# define exit function
exit_timeout() {
echo "Timeout. Checking for unfinished."
for i in ${PRODPID} ; do
ps -p $i |grep -v "PID TTY"
if [ $? == 0 ] ; then
# process still alive
echo "Sending SIGTERM to process $i"
kill $i
fi
done
# timeout exit
exit
}
# Handler for signal USR1 for the timer
trap exit_timeout SIGUSR1
# starting timer in subshell. It sends a SIGUSR1 to the father if it timeouts.
export TIMEOUT
(sleep $TIMEOUT ; kill -SIGUSR1 $PID) &
# record PID of timer
TPID=$!
# wait for all production processes to finish
wait ${PRODPID}
# Normal exit
echo "All processes ended normal"
# kill timer
kill $TPID

Usage examples:

$ bash timeoutwrapper.sh
USAGE: timeoutwrapper.sh <seconds to wait> <command with arguments>
$ bash timeoutwrapper.sh 1 sleep 3
Timeout. Checking for unfinished.
26612 pts/3    00:00:00 sleep
Sending SIGTERM to process 26612
$ bash timeoutwrapper.sh 4 sleep 3
All processes ended normal
$ bash timeoutwrapper.sh 2 du -sh .
Timeout. Checking for unfinished.
26622 pts/3    00:00:00 du
Sending SIGTERM to process 26622
$

PS: I found the script here and adopted it
http://redflo.de/tiki-index.php?page=Bash+script+with+timeout+function

1 Comment

SSH -R is even cooler than -L

To be honest with you, I thought for three years now that the -L option of ssh is a cool thing. It connects to the ssh server and offers a remote service (port) locally. Call it port tunnelling. Great against firewalls.

Anyway, consider you have a ssh server in the internet, and two computers behind different firewalls that only let traffic out. But you want to connect from computer A to computer B.
So you could connect to the ssh server from both computers, but you would need some software that reverses your TCP flow. One of the channels has to be the client and one the server. That might even be possible just using pipes and ssh -L.

Ok, so I thought I was clever until I found the -R option.
What it does is to connects to the ssh server and offers a local service (port) remotely. (-L offers a remote service (port) locally.)
Consequent. And works nicely too!

No Comments

Bash: Choose a random value out of a set

The best scripts are the shortest.
This chooses one of the arguments randomly. $# is the number of arguments

#!/bin/bash
echo ${BASH_ARGV[ $(( $RANDOM % $# )) ]}

Example:

$ bash randomchooser.sh 1 2 3 4 5 6 7 8 9 120
3

No Comments