getting supybot to announce new bugzilla bugs

getting supybot to announce new bugzilla bugs - I've just put here the key non-obvious things that tripped me up when trying to set this up.

All done on Ubuntu 8.04.3 LTS

Install supybot and the supybot bugzilla plugin.

Create a system group (supybot) and user (bugbot) to run supybot as.

Set up your supybot configuration file as desired.

Getting supybot to start at startup:
http://www.schwer.us/journal/2005/04/17/supybot-init-script-for-debian/

Here's my modified init script

$ cat /etc/init.d/bugbot
#! /bin/sh
#
# supybot init script
# http://www.schwer.us/journal/2005/04/17/supybot-init-script-for-debian/
#

PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/supybot
NAME=supybot
DESC=supybot

test -f $DAEMON || exit 0

set -e

case "$1" in
start)
echo -n "Starting $DESC: "
start-stop-daemon --start --quiet \
--chuid bugbot --exec $DAEMON -- --daemon /etc/supybot/bugbot.conf
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
start-stop-daemon --stop --quiet \
--oknodo --exec /usr/bin/python
echo "$NAME."
;;
restart)
$0 stop
$0 start
;;
*)
echo "Usage: $0 {start|stop|restart}" >&2
exit 1
;;
esac

exit 0

Set up bugzilla to send a copy of all bugmail to a local address (eg bugbot@localhost), and configure exim4 to accept local mail (as well as smart host delivery), using the mbox format.

sudo dpkg-reconfigure exim4-config

Start a conversation with bugbot, get it to identify you, then set the required configuration by sending it messages (you can also set these in the supybot .conf file for your bot):

• config plugins.Bugzilla.mbox /var/mail/bugbot


• config plugins.Bugzilla.bugzillas.your-bugzilla-name.watchedItems.all True
  which will turn on the announcements (i had to read the code to find that one!)

Note that supybot doesn't immediately write config changes to disc.

reseting home folder permissions in ubuntu linux

If you are like me and my coworkers, you often end up running stuff as root in your home folder and end up not able to access your own files.

For the record, here's the commands to reset the permissions (leaving all execute flags off, which may not be what you want).

Warning, this could have undesired side effects. If you have executable files in your home folder that you actually want to be executable, you will have to manually mark them as executable again.

# Become root (as you don't currently have permissions to modify your own files)
sudo -i

# Reset ownership & group to yourself
chown -R tim:tim /home/tim

# give yourself the default read-write permissions, set group and other to read only
chmod -R 644 /home/tim

# re-apply excecute permissions on all the directories
find /home/tim -type d -print0 | xargs -0 -II chmod 755 'I'

# Don't leave your private keys showing:
chmod -R 700 /home/tim/.ssh

My latest use for this was after extracting files from my p910i with p3nfs, which as with so many desktop/device tasks needed root to work.

You could do this slightly more neatly if you know what permissions you have ended up with by using chmod u+w etc so you don't have to re-apply directory permissions, but I wanted to easily guarantee permissions are right regardless of what state they have ended up in.

It occurs to me it might be sensible to set the sticky bit on the home folders so that anything added by root stays owned by the user. (chmod u+s g+s /home/tim)

Preparing photos for a digital picture frame

Challenge of the day was to fit as many photos as possible on a single flash card to stick in a digital photo frame. Here's how it's done.

The frame from Philips goes by the memorable name of 9FF2M4 , and by way of a quick review it is very nice. If I were a normal person, I would probably have copied the original 2.5MB / 5 megapixel images to the frame's flash card (1GB Compact Flash in this case, though it can take others), and put up with not being able to fit all the photos on, and having some of them show sideways. But being a perfectionist I instead sacrificed precious sleeping time to figure out what to do. In the end I managed to trim the files down to around 200KB each, and put portrait photos on a black background the right way up in order to save neck ache from squinting at a sideways eiffel tower. This was all done by the power of OSS and bash scripting. Here I present for your convenience the methods I used, and highlight some of the useful things I picked up along the way.

The first thing that taxed me was what size the photos needed to be to display best whilst taking up minimal space. You would think the answer would be emblazened on the product's box, but no! Philips don't seem to be too keen on promoting the resolution of the display, and even the shop keeper struggled to give me a number. The owner's manual states: "Resolution: 800 x 480 pixels (viewing area 680 x 480)" but after some time experimenting with test images created with the gimp I came to the conclusion that it was impossible to get the frame to display an image pixel perfect as it seemed to be re-scaling every picture regardless of original size. There appears to be no guidance from Philips as to what a good resolution for the photos would be, so after some experimentation I settled on 800x600 as this is slightly higher than the frame's native resolution, and fills the screen nicely without loosing too much off the edges when displayed.

The frame does not appear to read orientation from the exif data so I looked into rotating all the portrait images to display correctly. I am using the frame in its landscape orientation as that is the form of most of the photos, even though it can be placed in portrait orientation. When a portrait photos is displayed (eg 480x600), the frame puts a fair amount of the image off the top and bottom of the display, and by default puts it on a full white background which is a little hard on the eyes and detracts from darker photos. I therefore opted to create landscape images of 800x600 with a black background for all the portrait photos. I later discovered that you can on this frame change the background colour as follows: Main menu > Slideshow > Background colour > White / Black Grey.

The process I have used is a little specific to my setup and needs, but hopefully will give you a good starting point. I have created 3 bash scripts that call each other to orchestrate the conversion from my raw photo collection to a new set suitable for the frame, which in turn make use of imageMagick and exiftran to do the work.

I found out about imageMagick through searching, and tutorials such as HowTo - Batch Image Resize on Linux. The version packaged with Ubuntu 7.10 is quite old, so I ended up building and installing the latest version (6.3.7) from source to get all the functionality I needed.

exiftran is a nifty utility that reads the exif orientation information in a photo, losslessly rotates the photo to match and then updates the exif data. It is closely related to jpegtran.

My folder structure in my home folder (so the scripts make sense):

• scripts (for bash scripts)
• photos (originals)
  
• 2005
• 2005-12-31 event name
• etc
  
• 2006
• etc
• photos_frame (for the modified and shrunk photos which will be copied onto the flash card)
  

So without further ado, here's the scripts:

frame.sh - runs the processing scripts on each year folder of interest
#!/bin/bash -v
~/scripts/frame_photo_folder.sh 2005 ~/photos_frame/
~/scripts/frame_photo_folder.sh 2006 ~/photos_frame/
~/scripts/frame_photo_folder.sh 2007 ~/photos_frame/

frame_photo_folder.sh - runs the processing script on subfolder of the year
#!/bin/bash
#arg 1 = input folder
#arg 2 = output folder

INPUTPATH=$1
OUTPATH=$2
cd $INPUTPATH
if [ ! -d "$OUTPATH$INPUTPATH" ]
then
echo creating output folder \"$OUTPATH$INPUTPATH\"
mkdir $OUTPATH$INPUTPATH
fi
for fname in *
do
if [ -d "$fname" ]
then
if [ ! -d "$OUTPATH$INPUTPATH/$fname" ]
then
echo creating output folder \"$OUTPATH$INPUTPATH/$fname\"
mkdir "$OUTPATH$INPUTPATH/$fname"
fi
echo searching for jpg files in \"$fname\"
cd "$fname"
find . -maxdepth 1 -type f -name \*.JPG | xargs -iimgfile ~/scripts/frame_photo.sh "imgfile" "$OUTPATH$INPUTPATH/$fname"
cd ..
fi
done

frame_photo.sh

• creates output folder(s)
• copies original photo into output folder
• uses exiftran to rotate the photo to the correct orientation
• shrinks the photo to a maximum of 800x600, and fills any remaining space with a black background
  

#!/bin/bash

#arg 1 = photo file name
#arg 2 = where to put result
#resizes and pads suitable for a photo frame.

INPUTFILE=$1
OUTPATH=$2
#pwd
echo copying \"$INPUTFILE\" into \"$OUTPATH\"
cp "$INPUTFILE" "$OUTPATH"
cd "$OUTPATH"
#pwd
#echo processing \"$INPUTFILE\"
exiftran -ai "$INPUTFILE"
convert "$INPUTFILE" -resize '800x600>' -background black -gravity center -extent 800x600 "$INPUTFILE"

I timed the whole operation using the time command, and copied all output to a log file as follows.

$ time ./frame.sh 2>&1 | tee frame.log

The conversion of around 6000 photos took around one and a half hours.

The concept of redirection of stdout & stderr was neatly explained by the article CLI magic: need redirection?, so now I know that 2>&1 means redirect ouput number two into output number one, in other words redirect stderr into stdout, which then alows you to pipe the whole lot into something else like "tee" (No, not tea, though it may be interesting redirecting my photos into my tea...)

Add a comment or drop me a line if you find it interesting or useful or if you have any questions or criticisms.

Update:
I've worked this script into a small python gui app, check it out at http://github.com/timabell/photo-frame-prep

backing up your home folder

Here I outline the solution I chose for backing up my life, er... I mean home folder. (I'm sure there's life outside /home/tim somewhere...)

My requirements were:

• backup to dvd+rw


• >20GB of data to back up


• no obscure formats (in case I don't have the backup tool when I need to restore)

I looked at several solutions for backups but ended up writing scripts to meet my needs.

The main script creates a tar file of my home directory, excluding certain items, which is then split into files suitable for writing to dvd+rw discs, with tar based verification, md5sums and file list text files created at the same time.

The reason for splitting to 3 files per disc is that the iso 9660 spec has a 2GB file size limit, and it's important that the discs are as simple as possible (ie no UDF) to aid recovery in awkward situations. This is also why I avoided compression.

backup_home.sh
#!/bin/bash -v
#DVD+R SL capacity 4,700,372,992 bytes DVD, (see wikipedia on DVD)
#ISO max file size 2GB. 4.38GB/3 = 1,566,790,997bytes = 1,494MB
#1,490MB to leave some space for listings and checksums
tar -cvv --directory /home tim --exclude-from backup_home_exclude.txt | split -b 1490m - /var/backups/tim/home/home.tar.split.
cd /var/backups/tim/home
md5sum home.tar.split.* > home.md5
cat home.tar.split.* | tar -t > home_file_list.txt
cat home.tar.split.* | tar -d --directory /home tim > home_diff.txt
ls -l home.* > home_backup_files.txt

backup_home_exclude.txt
tim/work*
tim/.Trash*
tim/.thumbnails*

This leaves me with a big pile of split files (named .aa, .ab etc) and a few text files. I proceeded to write 3 split files per disc, and put the 4 text files on every disc for convenience. I used gnome's built in DVD writing to create the discs.

I also wanted to verify the md5 checksums as the discs were created, so I wrote another little script to make life easier. This ensures the newley written disc has been remounted properly, and runs the md5 check. So long as the 3 relevant checksums came out correctly on each disc I can be reasonably confident of recovering the data should I need it.
"eject -t" closes the cdrom, which is handy.

reload_and_verify.sh
#!/bin/bash -v
cd /media
eject
eject -t
mount /media/cdrom
cd cdrom
md5sum -c home.md5
cd /media
eject

In addition to the above mechanism (which is a pain at best, mostly due to media limitations) I keep my machines in sync with unison which I strongly recommend for both technical and non-technical users. I gather it also runs on microsoft (who?), so you might find it useful if you are mid transition.