Technomancy

While poking around I discovered that you can use the stty size command to get the dimensions in columns and rows of the current terminal window.

This little bash functions can help. We have to tell stty what terminal (ie tty) to use. By default it’ll use the stdin, which doesn’t work if you’re chaining commands.

TTY=$(tty)
function rows {
    stty --file=${TTY} size | sed 's/^([0-9]*) ([0-9]*)/1/'
}
function cols {
    stty --file=${TTY} size | sed 's/^([0-9]*) ([0-9]*)/2/'
}

Example usage:

dmesg | tail -n $(( $(rows) - 2 ))

I am currently uploading some files to a remote computer, however my internet connection is not great. The connection intermittantly dies, killing the upload. I have to restart the upload as soon as I notice this. Luckily I’m using rsync, so files aren’t uploaded again.

I don’t want to have to restart the command. This little script, ‘do-till-done’ will repeatidly execute a command until it ends successfully (i.e. an exit status of 0). It used the libnotify notification system for GNOME to alert you when it needs to rerun a command. It waits 1 seconds between failures. Pressing Control-C will stop that iteration of the programme. To stop it fully, press Control-C while it’s in the sleep phase. i.e. press Control-C twice quickly.

Script:

#! /bin/bash

PROGNAME=$(basename $0)

COMMAND=$*

$COMMAND

while [ $? -ne 0 ] ;
do
    notify-send --urgency=low "${PROGNAME}" "${COMMAND} failed${WAIT_STR}"

    sleep 1

    $COMMAND
done

notify-send --urgency=normal "${PROGNAME}" "${COMMAND} finished"

This programme is in the public domain.

Sample usage:

$ do-till-done rsync -aPh *.jpg example.org:public_html/photos/

We all know that source code should be under a version control system. However databases are rarely version controlled (so to speak). Many times your application will depend on a certain structure of a database. If someone changes the structure your application can break.

This script will create mysql files for the table structure and tab separated value (TSV) for the values. TSV is used because it is the mysql default and it allows one to easily use many command line unix tools. The primary keys are needed so that the TSV files can be saved in order. Having them ordered means that if you only add one record, the diff if guaranteed to be only that line and not have all the other lines re-organised.

The database user and password is not saved because it’s intented that each developer would call this script after making a database change and would then commit the changes. This also means all your developers will see the change in the commit mails.

Change the DB_HOST, DB_BASE, TABLE_STRUCTURES, TABLE_VALUES and TABLE_PRIMARY_KEYS lines to suit your needs.

Assuming you had 3 tables, table1, table2 and table3 that you wanted to store the structure for. You want the values in table1 and table3 to be stored too. The primary key of table1 is ‘id’, and of table3 is ‘user_id’.

#! /bin/bash

## config

DB_HOST="hostname"
DB_BASE="database"

# we want to back up the structure of these tables
TABLE_STRUCTURES=( "table1" "table2" "table3" )

# we want to back up the values in these tables
TABLE_VALUES=( "table1" "table3" )

# The primary key in each table in TABLE_VALUES. n-th value here is the primary
# key of the n-th table in TABLE_VALUES
TABLE_PRIMARY_KEYS=( "id" "user_id" )

## end of config

# collect our values
echo -n "Username: "
read DB_USER

echo -n "Password: "

# read silently, ie don't echo the password
read -s DB_PASS

# force new line
echo ""

# read in the structure. TODO: spaces in tables might feck this up. TODO check [*] vs [@]
for TABLE_NAME in ${TABLE_STRUCTURES[*]} ; do
    FILE_NAME=${TABLE_NAME}.mysql

    if [ -e ${FILE_NAME} ] ; then
        OLD_MD5SUM=$(md5sum ${FILE_NAME})
    else
        OLD_MD5SUM=""
    fi;

    # TODO having pass on command line is bad...
    mysql -BNe "SHOW CREATE TABLE ${TABLE_NAME};" -u ${DB_USER} -h ${DB_HOST} -p${DB_PASS} ${DB_BASE} | sed -e "s/${TABLE_NAME}t//" -e 's/n/n/g' > ${FILE_NAME}

    # check if it's changed

    NEW_MD5SUM=$(md5sum ${FILE_NAME})

    if [ -z "${OLD_MD5SUM}" ] ; then
        echo "First time saving structure for ${TABLE_NAME}"
    else
        if [ "${OLD_MD5SUM}" != "${NEW_MD5SUM}" ] ; then
            # the structure has changed
            echo "The structure for ${TABLE_NAME} has changed"
        fi
    fi

done

# Now we get the values
for (( INDEX=0; $INDEX < ${#TABLE_VALUES[*]} ; INDEX=$(( $INDEX + 1 ))  )) ; do
    TABLE_NAME=${TABLE_VALUES[$INDEX]}
    TABLE_KEY=${TABLE_PRIMARY_KEYS[$INDEX]}

    # TODO: check TABLE_KEY exists, otherwise the SQL statement would be invalid

    FILE_NAME=${TABLE_NAME}.tsv
    if [ -e ${FILE_NAME} ] ; then
        OLD_MD5SUM=$(md5sum ${FILE_NAME})
    else
        OLD_MD5SUM=""
    fi;

    # TODO having pass on command line is bad...
    mysql -BNe "SELECT * FROM ${TABLE_NAME} ORDER BY ${TABLE_KEY};" -u ${DB_USER} -h ${DB_HOST} -p${DB_PASS} ${DB_BASE} > ${FILE_NAME}

    # check if it’s changed

    NEW_MD5SUM=$(md5sum ${FILE_NAME})

    if [ -z “${OLD_MD5SUM}” ] ; then
        echo “First time saving the values from ${TABLE_NAME}”
    else
        if [ “${OLD_MD5SUM}” != “${NEW_MD5SUM}” ] ; then
            # the structure has changed
            echo “The values in ${TABLE_NAME} have changed”
        fi
    fi

done

This script is copyrighted 2007 Rory McCann and released under the GNU General Public Licence v2 (or at your option a later version)

Shell scripts usually operate on a line by line basis, i.e. each line is a separate field. However MySQL uses “field1″,”field2″,.. type groups (e.g. for “SELECT * FROM table WHERE field IN ( “field1″,”field2″,.. );” ). This little alias will join all those lines together into something copy/pasteable into a MySQL query.

alias 'quotejoin=sed -e "s/^/"/" -e "s/$/",/" | tr -d "n" | sed "s/,$/n/"'

Example use:

$ cat test
line1
line2
line3
$ cat test | quotejoin
"line1","line2","line3"
$

I prefer the mysql command line for some mysql tasks. Other interfaces like PHPMyAdmin are annoying (readline is better than the stupid click to select textarea). To prevent myself making mistakes I have a read only user for my database (has global select and create temporay tables, that’s all). I’d like to know when I’m logged in as the read only user (and hence can do no damage), or something else.

You can use

SELECT CURRENT_USER();

or you can change the mysql prompt. The default is “mysql> “. Mine looks like this: “username@dbhost/database> “. Set your ~/.my.cnf to the following to get this behaviour.

[mysql]
prompt="u@h/d> "

References:

• MySQL documentation on mysql options

Sometimes when testing programmes that call other programmes, it’s important to be able to see what was called and what was on the stdin. This script prints all that info back to you.

#! /bin/bash
echo "script: args: $*"
while read line; do
    echo "script: stdin: $line"
done

exit 0

Example use:

$ cat > foo
Hello
this is line 2
$ cat foo | ./echor arg1 -n arg3
script: args: arg1 -n arg3
script: stdin: Hello
script: stdin: this is line 2

A lot of scripting languages like perl or python have good string and regular expression (regex) support. By using grep, you can have this in bash. Normally grep prints out the lines that match the regex. But with the -q option, it doesn’t print anything and exits as soon as it finds a match (or gets to the end). The return code will be 0 if it wasn’t found, or non-zero if it was found. This is ideal for including in an if clause. Remember to include ‘^’ and ‘$’ to ensure the regex is the only input.

eg:

if echo $VAR | grep -q '^d{3,10}$' ; then
  echo "VAR matches"
fi

Turns out you can do this in bash already but only in bash 3.0 or above.

The problem

You have a script, “script.sh” that needs to be run as root, but you can’t run it as root. It needs to be called by something else, for example a web application. You also need to pass arguments to this script.

setuid

For most programmes you can set the setuid bit on programme. For example if you set the permissions to u=rwxs,g=rx,o= and the file is owned by root and the group is users, then any member of the group users can execute this programme but it will be run as root.

This sounds like what you need, however you can only do this with proper programmes, not interpreted scripts (such as bash, perl, php or python scripts), this is because there is a delay between when the interpreter is started and when the interpreter. In that delay, someone could replace the script with some other file. This is called a race condition. Hence on a lot of unix kernels, you can’t setuid an interpreted script.

The solution

The solution is to create a binary that calls you shell script. This can be done with this simple C programme.

#include <stdlib.h>

int main( int argc, char** argv )
{
    // The uid will be uid of original user and the euid will be root (ie 0). Bash
    // scripts don't necessarly use the euid, so we'll setuid full to root.
    // Since the euid is 0, we can setuid to 0.
    //
    // alternativly we can start the script with "#! /bin/bash -p" to force posix mode
    setuid( 0 );

    // shouldn't have to do this, but for consistancy we might as well. And to
    // be sure to be sure
    seteuid( 0 );

    // replace the current programme with this programme, the return value of
    // this C programme is set to the return value of this script.
    // You should use the full path here, since the current directory can be changed by the caller, opening a security risk.
    execl( "/path/to/script.sh", "script.sh", argv[1], 0 );

    // this is never executed.
    return 1;
}

Compile it like this:

gcc -o wrapper wrapper.c

The programmw will need the following permissions

chmod u=rwxs,g=rx,o= wrapper
chown root:othergroup wrapper

Now any member of the group othergroup can run the script, and it’ll be run as root.

UID, Effictive UID and bash

When you are running a programme that was setuid, the UID will be your original UID and the Effective UID (EUID) will the the UID of the user that was setuided. In this case the UID will be your UID and the EUID would be 0 (for root). The system call setuid() is used to change the UID of the current process. You can’t change to any UID (obviously), though you can change to the EUID, that’s why we were able to change to root’s UID in the C programme.

If we didn’t change it, the bash script would run with the UID and EUID like the C programme. By default bash will only use the UID, which is pointless. If you add “#! /bin/bash -p” to force the script into POSIX mode, it’ll use the EUID. For the sake of ease, I’ve just changed the UID in the C wrapper above.

You can see the UID and EUID in the C wrapper with this line:

printf( "uid: %d euid: %dn", getuid(), geteuid() );

You can see the UID and EUID in the bash script with this line:

echo "uid $UID euid: $EUID"

Sources:

• execl in C
• Introduction to setuid
• Simple C wrapper function

Changelog

1. Monday 21st May 2007 - Fixed typo in C’s #include caused by malformed HTML escaping

gnome-terminal is the default terminal for GNOME. It is a tabbed terminal emulator. You can change the title of the current tab with this command, which will change the title to “TITLE”.

echo -en "33]0;TITLEa"

This function will do that for you, just put it in your ~/.bashrc

function title {
    echo -en "33]0;$*a"
}

Use it like this:

$ title foo

EDIT Sun 10th June 2007: You can now have spaces in your tab title