Posts Tagged hint
sed: Remove lines that match a pattern
Posted by JohannesTheDeveloper in fun with Linux on June 27th, 2009
sed '/mypattern/d' file
Deletes all lines that contain the regular expression “mypattern”.
In-place replacement:
sed --in-place '/mypattern/d' file1
sed also supports making backups with e.g. –in-place=.backup .
Shutdown when idle
Posted by JohannesTheDeveloper in fun with Linux on June 20th, 2009
Maybe you want the computer to shutdown after it has done its task. This shuts down your Linux box when the load goes down to zero. Kind of like the “Speed” movies.
while ! cat /proc/loadavg |cut -f2 -d ' '|grep '0\.00'; do sleep 10m; done && poweroff
Alternatively:
while ps 12345; do sleep 10m; done && poweroff
Bash: Chain of commands
Posted by JohannesTheDeveloper in fun with Linux on April 25th, 2009
I wondered more than once how to make a tool that chains together input/output filters using bash.
e.g. you would like to do a iteration of filters:
for i in 1 2 3 4; do grep -v $i; done
So I wrote this chaining tool that does the job in the obvious way:
chain.sh
#!/bin/bash
commands_file=$1
a=$(mktemp)
b=$(mktemp)
cat > $a
while read line; do
$line < $a > $b
mv $b $a
done < $commands_file
cat $a
rm $a
With this, you can do:
for i in 1 2 3 4; do
echo grep -v $i >> mycommands.txt
done
script/chain.sh mycommands.txt
lang=”bash”
make: self-documenting Makefiles
Posted by JohannesTheDeveloper in Happy Hacking on April 19th, 2009
With the following grep/sed combinations, you can keep the help for each target inline just above your target (or wherever you want it).
## Makefile for foo
## help: this clutter
help:
@grep -E '^## ' Makefile|grep -v ':'|sed 's,^## ,,g'
@echo This Makefile has the targets:
@grep -E '^## [.a-z]{2,}:' Makefile|sed 's,^## *,\t,g' |sed 's,: ,\t,g'
## tests: run the tests
tests: tests.exe
./tests.exe ${TESTNR}
Output of
$ make help
Makefile for foo
This Makefile has the targets:
help this clutter
all
tests run the tests
...
Nice, leet and meta. :-)
Some more words on Makefiles: You can use something like
CFLAGS=-O3 -Wall -Werror -Wextra -g -ansi -pedantic ${CCFLAGS}
and pass CCFLAGS from outside. This avoids having many many versions while trying to test things out. For example:
$ CCFLAGS="-DDEBUG -Os -DUSED_ALGORITHM=binsearch2" make tests
(if you didn’t notice yet, -DFOO lets you jump inside #ifdef FOO).
Add collaborator to GitHub repository
Posted by JohannesTheDeveloper in fun with Linux on March 6th, 2009
Don’t let them clone the public url: It doesn’t go over ssh and I don’t know how to change the origin url in a branch. *
Steps:
- create ssh-key on new collaborators computer (ssh-keygen)
- create github user
- add public-key to github user account
- let the project owner add you as project member
- git clone git@…
Where 1 < 3, 1 < 2 < 4, 2 < 5 if that means anything to you ;-)
* it works with git remote
Autologin on websites
Posted by JohannesTheDeveloper in Happy Hacking on January 19th, 2009
Some sites provide for their login one form that just contain the login form. Firefox fills it out with username+password, but you still have to aim at the submit button.
This greasemonkey-script provides auto-login for TUWEL, TUWIS and any other site you want that only provides one form. autologin.user (js, 1 KB)
Maven: Colorized
Posted by JohannesTheDeveloper in fun with Linux on January 7th, 2009
Maven was written by people who obviously have no idea on how to create good tui (Text User Interface)-tools. They should sit together with the portage, git or other Linux-devs.
The output is unreadable (not to mention that the startup is slow).
So, what I did was wrap maven in a script that colorizes the output. The result looks like this:

This is how to do it:
create the file ~/bin/mvn with the following content (mvn (0 KB))
#!/bin/bash
/usr/bin/mvn $* | sed -e 's/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/[1;32mTests run: \1[0m, Failures: [1;31m\2[0m, Errors: [1;33m\3[0m, Skipped: [1;34m\4[0m/g' \
-e 's/\(\[WARN\].*\)/[1;33m\1[0m/g' \
-e 's/\(\[INFO\].*\)/[1;34m\1[0m/g' \
-e 's/\(\[ERROR\].*\)/[1;31m\1[0m/g'
replace /usr/bin/mvn with wherever your mvn is (find out with which mvn).
Double-check that you get the encoding right, the weird sign should be hexadecimal 1b. Use a editor that supports encodings for copy+paste.
Give your ~/bin/mvn execute permissions (chmod +x ~/bin/mvn)
Try it out: call ~/bin/mvn in one of your maven projects
What is left to do is to register your new mvn-wrapper so it is called everytime.
Add the following line to your ~/.bashrc :
alias mvn=~/bin/mvn
Execute the command you just added in your running shells so you can use it there too.
You are done.
Changing the distributed mvn startup script
Even better, but a little more intrusive (you need root rights):
change the distributed mvn file. It is a bash script actually.
In gentoo:
$ which mvn
/usr/bin/mvn
$ file /usr/bin/mvn
/usr/bin/mvn: symbolic link to `/usr/share/maven-bin-2.0/bin/mvn'
$ file /usr/share/maven-bin-2.0/bin/mvn
/usr/share/maven-bin-2.0/bin/mvn: Bourne shell script text executable
$
Change the last call in that file to:
exec "$JAVACMD" \
$MAVEN_OPTS \
-classpath "${M2_HOME}"/boot/classworlds-*.jar \
"-Dclassworlds.conf=${M2_HOME}/bin/m2.conf" \
"-Dmaven.home=${M2_HOME}" \
${CLASSWORLDS_LAUNCHER} $QUOTED_ARGS |
sed -e 's/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/[1;32mTests run: \1[0m, Failures: [1;31m\2[0m, Errors: [1;33m\3[0m, Skipped: [1;34m\4[0m/g' \
-e 's/\(\[WARN\].*\)/[1;33m\1[0m/g' \
-e 's/\(\[INFO\].*\)/[1;34m\1[0m/g' \
-e 's/\(\[ERROR\].*\)/[1;31m\1[0m/g'
That way you have it there directly. Maybe the gentoo devs could add a color use-flag. That’d be fun…
Bash: Timeouts
Posted by JohannesTheDeveloper in fun with Linux on January 7th, 2009
You have a command and want to kill it after a certain time if it hasn’t finished.
This script does it for you:
timeoutwrapper.sh
#!/bin/bash
# License: BSD License
# author: Ram
# author: Johannes Buchner
TIMEOUT=$1
shift
COMMAND="$*"
if [[ "$TIMEOUT" == "" ]] || [[ "$COMMAND" == "" ]]; then
echo "USAGE: $0 <seconds to wait> <command with arguments>"
exit
fi
$COMMAND &
PRODPID=$!
export PRODPID
# record own PID
export PID=$$
# define exit function
exit_timeout() {
echo "Timeout. Checking for unfinished."
for i in ${PRODPID} ; do
ps -p $i |grep -v "PID TTY"
if [ $? == 0 ] ; then
# process still alive
echo "Sending SIGTERM to process $i"
kill $i
fi
done
# timeout exit
exit
}
# Handler for signal USR1 for the timer
trap exit_timeout SIGUSR1
# starting timer in subshell. It sends a SIGUSR1 to the father if it timeouts.
export TIMEOUT
(sleep $TIMEOUT ; kill -SIGUSR1 $PID) &
# record PID of timer
TPID=$!
# wait for all production processes to finish
wait ${PRODPID}
# Normal exit
echo "All processes ended normal"
# kill timer
kill $TPID
Usage examples:
$ bash timeoutwrapper.sh
USAGE: timeoutwrapper.sh <seconds to wait> <command with arguments>
$ bash timeoutwrapper.sh 1 sleep 3
Timeout. Checking for unfinished.
26612 pts/3 00:00:00 sleep
Sending SIGTERM to process 26612
$ bash timeoutwrapper.sh 4 sleep 3
All processes ended normal
$ bash timeoutwrapper.sh 2 du -sh .
Timeout. Checking for unfinished.
26622 pts/3 00:00:00 du
Sending SIGTERM to process 26622
$
PS: I found the script here and adopted it
http://redflo.de/tiki-index.php?page=Bash+script+with+timeout+function
SSH -R is even cooler than -L
Posted by JohannesTheDeveloper in fun with Linux on January 5th, 2009
To be honest with you, I thought for three years now that the -L option of ssh is a cool thing. It connects to the ssh server and offers a remote service (port) locally. Call it port tunnelling. Great against firewalls.
Anyway, consider you have a ssh server in the internet, and two computers behind different firewalls that only let traffic out. But you want to connect from computer A to computer B.
So you could connect to the ssh server from both computers, but you would need some software that reverses your TCP flow. One of the channels has to be the client and one the server. That might even be possible just using pipes and ssh -L.
Ok, so I thought I was clever until I found the -R option.
What it does is to connects to the ssh server and offers a local service (port) remotely. (-L offers a remote service (port) locally.)
Consequent. And works nicely too!
Bash: Choose a random value out of a set
Posted by JohannesTheDeveloper in Happy Hacking on November 28th, 2008
The best scripts are the shortest.
This chooses one of the arguments randomly. $# is the number of arguments
#!/bin/bash
echo ${BASH_ARGV[ $(( $RANDOM % $# )) ]}
Example:
$ bash randomchooser.sh 1 2 3 4 5 6 7 8 9 120
3
Recent Comments