Monday, December 21, 2009

Script to record the IP address of your computers

I like to keep a simple script on all of my machines that routinely checks in with a server and records the current IP of the machine. It's great for ssh'ing into a machine that has a constantly changing IP address (like a laptop) and in case of theft, it could help track down the stolen machine.

I like to have the clients check in with a webserver, instead of a machine I own. Basic websites are cheap (like GoDaddy) and have high availability.

There will be three files that this project will use:
-CheckIn.php will be on the server and will record the IP.
-CheckIn.txt will store the IP addresses.
-CheckIn will be on the client. It will use wget to contact the server.

1. Here is CheckIn.php:

<?php
//Begin Setup - Change these variables
$filename = "CheckIn.txt"; //The name of the text file to save the IP addresses to.
$my_pass = "password"; //The password to see the IP addresses.
$max_lines = 100; //The maximum number of IP addresses to store. 0 for no limit.
//End Setup

if (!function_exists('file_put_contents')) {
function file_put_contents($filename, $data) {
$f = @fopen($filename, 'w');
if (!$f) {
return false;
} else {
$bytes = fwrite($f, $data);
fclose($f);
return $bytes;
}
}
}

$computer = htmlentities($_GET['user']);
$password = htmlentities($_GET['password']);
$contents = file_get_contents($filename);
if(!$computer){
if($password==$my_pass){
echo "<table border=1 cellpadding=2 cellspacing=2>";
echo $contents;
echo "</table>";
}
} else {
$output = "<tr>";
$output = $output."<td>".$computer."</td>";
$output = $output."<td>".$_SERVER['REMOTE_ADDR']."</td>";
$output = $output."<td>".date("ymd H:i D")."</td>";
$output = $output."<td>".time()."</td>";
$output = $output."<td>".$_SERVER['HTTP_REFERER']."</td>";
$output = $output."<td>".$_SERVER['HTTP_USER_AGENT']."</td>";
$output = $output."</tr>\n".$contents;
$counter = fopen("Stats", 'w');
fwrite($counter, $output);
fclose($counter);
if($max_lines!=0){
$array = file($filename);
for($i=sizeof($array); $i>=$max_lines;$i--){
unset($array[$i]);
}
$array=array_values($array);
file_put_contents($filename,implode($array));
}
}

?>


This code should work on all machines running PHP v4.3 and later. Change the filename, password, and max_lines at the beginning of the file to the values you want to use.

2. CheckIn.txt is an empty file to start with:

touch CheckIn.txt


3. Upload Checkin.php and Checkin.txt to your webserver. Setup the permissions so that Checkin.php is accessible, but Checkin.txt is not:

chmod 644 CheckIn.php
chmod 600 CheckIn.txt



4a. Add an entry to CheckIn.txt whenever there is a new Internet connection. This only works if you are using NetworkManager. As root, create the file /etc/NetworkManager/dispatcher.d/CheckIn:

#!/bin/sh

if [ "$2" = "up" ]; then
wget http://URL_TO_CHECKIN.PHP?user=COMPUTER -O /dev/null > /dev/null 2> /dev/null
fi


and make it executable:

chmod +x /etc/NetworkManager/dispatcher.d/CheckIn



4b. Add an entry to CheckIn.txt every 10 minutes. Either as a regular user, or as root, create a crontab:

export EDITOR=`which pico`
crontab -e


The choice of editor is completely up to you. Put the following in your crontab:

*/10 * * * * sh /root/script.sh > /dev/null 2> /dev/null


where the "*/10" indicates that the command should be run every 10 minutes. Wikipedia has an great page on the format of crontabs.

Finally, check to make sure that the crontab was installed correctly:

crontab -l

Sunday, December 20, 2009

The case against Swap

Swap, that extra partition you keep around in case you run out of memory, is actually, in most situations, a bad thing to have. Here's why.

Why swap was a good thing.

1. Hard drives and RAM memory used to have similar speeds. RAM was faster, but not by so much that it meant the system was unusable if it was run off of the hard drive instead. When this is the case, it makes sense to use the hard drive as a backup, but slower memory.

2. Applications tended to fill the RAM. Imagine you have a system with 16MB of memory. You have a word processor that uses 12MB and an image editor that uses 12MB. To have both open simultaneously, you need to have an extra 8MB of storage somewhere. It makes sense to use the hard drive. This way, you can keep the entire word processor in memory when you're using it and when you switch to the image editor, the system can swap the word processor to disk and bring the image editor back to RAM. You'll experience a word processor and image editor that both run at full speed and the only slowdown you'll see is when you switch applications.



Why swap is (now) a bad thing.

1. RAM memory is now so much faster than the hard disk that the system is unusable if it runs off of the hard drive. Maybe this will change again someday as SSD's pickup speed, but for now, it's better to have an application crash or fail to open rather than have an almost frozen and completely unusable system.

2. In most cases, no single application consumes a majority of the RAM. Today, a large application might consume 15%-20% of the available RAM on your system. This makes it unlikely that you'll need to have one application's memory on disk while the other one is in RAM. Running out of RAM today is more likely due to having far too many applications open. The solution is not to mask it by swapping to disk, but rather to either change user behavior (keep fewer applications open) or install more RAM.

3. If the machine starts to thrash (i.e., access the swap continuously), it can be rough on a hard drive, especially if you're using a SSD, where the number of writes is limited.



Why not having swap is a good thing.

It can stop bugs. I had a buggy version of Adobe Acrobat that after you "closed" it, it would remain open in the background and slowly consume memory. After a couple of hours, all of the RAM would be gone and it would start in on the swap. After a couple of more hours, the system would be thrashing the disk and completely unusable. Without swap, after Acrobat used up all of the RAM, it would request more, the request would fail, and it would crash (and close). Of course, it would have been better if Adobe had produced a bug-free version of Acrobat, but consider in this case not having swap prevented my system from being brought down due to a buggy application.



Why not having swap could be a bad thing.

If you have a particular use case that requires you to use more memory than you have RAM, then you will need to have swap. But let me suggest that instead of permanently keeping around a swap partition, use a swap file instead and only activate it when you know you're going to be needing it.


My conclusions.

I think for most users, swap is an outdated concept and causes more trouble than its worth. And your applications probably don't require as much memory as you think. A couple of real-world examples:

I have an old P4 system with 384MB of RAM and no swap that's been running fine for word processing and Internet use for years. It has OpenSUSE 11.1 on it with XFCE.

I also have a laptop with 512MB of RAM and no swap, running OpenSUSE 11.1 with KDE 3.5. I use it for word processing, web surfing, watching Hulu and YouTube videos. I even use a TV tuner card to record shows. I used this machine to write a 100 page technical manual filled with graphics (both vector and bitmapped). Again, I've had no problems due to the lack of swap.

Thursday, December 17, 2009

Absolutely silent wget

I had an application where I wanted to use wget to request a file from an http server, but I wanted absolutely no output. No status, no errors, nothing to the console and nothing written to the disk. This is for a laptop "anti-theft" script I'm working on. Everytime an Internet connection is established, the local script calls a PHP script on my webserver, which records the IP address it was called from. As a result, I get a nice list of IP addresses where my laptop has been. However, I don't want the disk filling up with a bunch of HTML files, as is the standard practice for wget. The solution is this:

wget http://URL -O /dev/null > /dev/null 2>&1

The "-O" directs the HTML file to /dev/null, then ">" directs the standard output to /dev/null, and then "2>&1" directs the standard error to the standard output which goes to /dev/null. As a result, this is an absolutely silent (even if its fails) wget.

One quick word of warning, the "2>&1" does not work in the sh shell. So, in your script files, instead of:

#!/bin/sh

you'll need to use another shell (like bash):

#!/bin/bash

or, not use that fancy redirect at all and use "2> /dev/null" instead.

Sunday, November 22, 2009

Use the numpad in Konsole

Konsole, the terminal application in KDE, now by default, disables the use of the numpad in openSuSE 11.2. If you want to scroll back, you have to use the arrow keys and not "8 up" and "2 down". This is easily fixed.

In Konsole, go to "Settings", then "Edit Current Profile". Click on the "Input" tab. Select "Linux console" and press "OK".

Friday, November 20, 2009

Watch TV with the ATI TV Wonder 600 USB stick

The ATI TV Wonder 600 USB stick supports NTSC and ATSC, as well as composite input and S-video. It shows up (if you do a lsusb) with a manufacturer:device combo as 0438:b002 from Advanced Micro Devices (AMD), the company that bought ATI a while back .This guide will walk you through the steps of watching and recording ATSC TV.

1. Add the Packman repositories. You can do this in Yast2, under Software, then Software Repositories.

2. Install the dvb, vlc, and MPlayer packages. As root:

yast -i dvb vlc mplayer


3. Get the firmware file xc3028L-v36.fw from here:

http://steventoth.net/linux/hvr1400/xc3028L-v36.fw

and save it to /lib/firmware (you'll need to be root).

4. Get a channel list. Not as root, perform:

scan /usr/share/dvb/atsc/us-NTSC-center-frequencies-8VSB -o zap -a 0 | tee ~/channels.conf


If you're not in the US, then you'll need a different scanning file other than us-NTSC-center. Look at the list in /usr/share/dvb/atsc and select one that is appropriate for your location.

The "-a 0" switch indicates that it should use the first DVB adapter in your system. If you this is a second adapter, then use "-a 1".

This will take some time and will generate a number of "tuning failed!!!" messages. These are harmless.

When the process is complete, you may want to manually edit the first column of the file, which is the name of the station. Some stations include contact information and that may confuse other programs that try and use the file. For example, if the file contains a line like:

WHSD - 1-800-123-WHSD:562000000:8VSB:81:92:3


Replace it with something more readable, like:

NBC:562000000:8VSB:81:92:3



5. Watch TV

vlc ~/channels.conf


If this is the second adapter, use the switch "--dvb-adapter=0". The navigation controls in vlc change the channels.

If you right-click on the image, there is a de-interlacing option. Enable de-interlacing to improve the video quality on LCD monitors, otherwise motion will appear to "tear".

I am able to watch TV on my 900MHz Celeron netbook. It is important to note that different channels will broadcast at different quality levels. Depending on the channel, my cpu usage can be anywhere from 40% to near 100%.


6. Record TV First, copy the channels file to ~/.mplayer/. Record with:

mencoder dvb://"NBC" -o Chuck.avi -oac copy -ovc copy


If this is a second adapter, then it will be 'dvb://2@"NBC"'. This requires very low CPU usage, but an extreme amount of hard disk space (roughly 1GB every ten minutes). You can try to re-encode the file on the fly, but the CPU usage will be very high and I haven't found a combination of settings that maintains the original quality of the video. But as a starting point:

mencoder dvb://"NBC" -o Chuck.avi -oac mp3lame -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=1500:vhq


You can play back the recorded file using any media player, but with mplayer, it's:

mplayer Chuck.avi -vf pp=fd


Where the "-vf pp=fd" switch turns on de-interlacing.

Friday, November 6, 2009

Prevent a printer from becoming "disabled"

CUPS, by default, disables printout to a printer if the printer stops responding. In order to re-enable printing, you must go into YaST | Hardware | Printers and for the disabled printer, select the "Printout Enabled" checkbox.

To prevent a printer from becoming disabled in the first place, as root, open /etc/cups/printers.conf. For each printer, change the ErrorPolicy line from "stop-printer" to "retry-job". CUPS will now retry the job every 30 seconds.

If you want to change the time it takes before it retries, open /etc/cups/cupsd.conf and add a line (or edit, if it already exists) that says "JobRetryInterval 120", where 120 is the time in seconds.

Thursday, July 30, 2009

How to set the time with NTP as an option

I will show you how to manually update the time and manually use NTP to update the time.

1. Set the current time manually. Before you can use NTP, you need to have the time on your system correct to within a few minutes. If its more than an hour or so, NTP won't have confidence in the time it gets from the server and as a result, it won't update the time. As root:

date "MMDDHHmmYYYY"


where MM is the two digit month, DD is the two digit day, HH is the two digit hour on a 24-hour clock, mm is the two digit minute, and YYYY is the four digit year. For example, 6:30pm on July 30, 2009 would be:

date "073018302009"



2. As root, use YaST to setup the NTP daemon:

yast ntp-client


This YaST module is available under the "Network Services" section of the YaST2 GUI.

You'll need to set the NTP start to automatic before YaST will let add a time server. Add a public server (the tool has a pretty good list of servers you can use). Then, if you don't have a permanent Internet connection (including a laptop or a computer using the Network Manager applet), set NTP to only start manually.


3. From this point on, anytime you want to reset the time on your computer, use, as root:

ntpd -q


This will cause NTP to set the time and then quit.

Saturday, June 27, 2009

Change SSH Port

If you have two or more computers behind a NAT (i.e., router) and you want to access both machines, you need to change the SSH port on at least one of them. To do so, as root, edit /etc/ssh/sshd_config. Add a line that reads "Port PORT_NUMBER". The relevant part of my file looks like this:

Port 2200
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::


You will need to either restart the SSH daemon (or reboot) before the change will take effect. And of course, you'll need to adjust the firewall on your computer to allow the new port to pass through, as well adjust your NAT.

Thursday, June 25, 2009

Freeing disk space

This is just an FYI to anyone who is deleting files off their hard drive and it's probably a pretty familiar concept to those who've been using Linux for awhile.

For any file, Linux maintains a list of handles, that is programs that are currently using the file. When you delete the file, it is only removed from the disk when the number of handles goes to zero.

This becomes important if your drive is almost full and you delete a file, but you find that your disk space has not changed. The most likely reason is that some program still has an open handle.

Make sure that any program that might be using the program is closed and if all else fails, a reboot will clear all of the open file handles.

And if you weren't already aware, the command line tool to find out how much disk space you have left is 'df'.

Tuesday, June 23, 2009

Sound not working in Flash? Here's a possible fix.

Here's the symptom: your sound works fine, except in Flash. You've tried multiple browsers and Flash sound doesn't work in any of them.

The problem is that you're using PulseAudio as a sound engine and sometimes Flash doesn't work well with it.

To fix this, remove the libpulse0 package. As root:

yast --remove libpulse0


When I did this, it ended up removing MPlayer due to dependecies. I reinstalled MPlayer from the repositories which brought back one PulseAudio package, but sound still worked in Flash.

Monday, June 22, 2009

::Updated:: A guide to getting the Digipro T-8000U Tablet to work with OpenSUSE

This post supersedes my previous one. It fixes the bugs that were in that one.

This guide will get the Digipro T-8000U tablet working with OpenSUSE 11.1.

1. Install the necessary prerequisites. As root:

yast -i gcc xorg-x11-devel xorg-x11-server-sdk


2. Get the latest version of the WizardPen driver, which as of now is 0.7.0a2. It is available on a Microsoft website (don't ask me why):

http://cid-43438aff38d34c29.skydrive.live.com/self.aspx/Public/wizardpen/wizardpen-0.7.0-alpha2.tar.gz

Untar, configure, and compile it. Do not install it.

tar xvzf wizardpen-0.7.0-alpha2.tar.gz
cd wizardpen-0.7.0-alpha2
./configure && make


3. The Makefile installs the driver where Xorg won't find it, so you have to install it manually. As root:

cp src/wizardpen_drv.la /usr/lib/xorg/modules/input
cp src/.libs/wizardpen_drv.so /usr/lib/xorg/modules/input


4. You need to find the device associated with the tablet. First, do:

grep Name /proc/bus/input/devices


Look for something that looks like your tablet. Mine was "Aiptek". If nothing sticks out, you can do an:

lsusb


to find the full name of the tablet and compare that with the output from "grep Name /proc/bus/input/devices".

Then do (assuming your tablet's name is Aiptek):

lshal | grep "input.product = 'Aiptek'" -A 20 -B 20 | grep input.device


This should produce a single line with your device. Mine was /dev/input/event7. If this doesn't work, do a "lshal" and search through the output until you find the string.

5. Download and run this calibration routine. The one that is provided with the WizardPen code does not calibrate in the Z-direction, which is critical to getting the pressures to work.

wget http://www.filefactory.com/file/ag8g447/n/wizardpen-calibrate_c wizardpen-0.7.0-alpha2/calibrate/wizardpen-calibrate.c
cd wizardpen-0.7.0-alpha2/calibrate
make
sudo ./wizardpen-calibrate /dev/input/event7


6. As root, edit /etc/X11/xorg.conf. Add a section that reads:

Section "InputDevice"
Driver "wizardpen"
Identifier "WizardTablet"
Option "Device" "/dev/input/event7"
Option "TopX" "0"
Option "TopY" "0"
Option "TopZ" "54"
Option "BottomX" "3000"
Option "BottomY" "2200"
Option "BottomZ" "511"
Option "MaxX" "3000"
Option "MaxY" "2200"
Option "MaxZ" "511"
EndSection


where all of the numbers are from the calibration program in step 5. In the ServerLayout section, add a line that reads:

  InputDevice  "WizardTablet"  "AlwaysCore"


7. Reboot.

The tablet should be working now. To use it in Gimp, you need to go to Edit, Preferences, Input Devices, Configure Extended Input Devices, and for the WizardTablet device, set the Mode to Screen. Be warned, with an extended input device (the tablet) enabled in Gimp the mouse can no longer be used to draw. However, disabling the tablet re-enables the mouse. See this GTK bug for more details.

There is one lingering bug and that is that hotplugging does not work. I'll look at this sometime over the next week. Until then, you'll need to have the devices you want to use plugged in at boot.

---===Very Important===--- If you have a dynamic collection of input devices, where what you have plugged into the computer changes from boot to boot, you'll need to make a couple of small modifications to your xorg.conf. These corrections are necessary if you want to, for example, use a USB mouse occasionally. You need to replace the "/dev/input/DEVICE" lines with either the "/dev/input/by-path/DEVICE_PATH" or "/dev/input/by-id/DEVICE_ID" for your device. You can find out what this are by doing a:

find /dev/input


with and without the device plugged in and then comparing the outputs. There will be two files associated with each device. For mice, use the one without "event" on the end and for the tablet, use the one with "event" on the end.

Sunday, June 21, 2009

Install openSUSE from a USB drive

If you have a netbook or any computer without a cd drive you can easily install openSUSE from just about any USB drive.

You'll need a bootable USB drive (they almost all are and the easiest way to find out if yours will work is just to try it) and a computer that can boot from USB (any computer build in the last five years should be able to).

1. Go to http://software.opensuse.org/ and get a copy of the installer. Make sure you choose the correct type of computer. On AMD/Intel chips, 32-bit will always work and 64-bit will work on most newer computers.

You can choose to use either the DVD or the network install. If you choose the DVD option, you'll need a USB drive with a capacity of at least 4.2GB. Practically, 8GB is the smallest USB drive you'll find with enough space. If you choose the network install, you'll need a network connection during the installation. In my experience, an ethernet connection always works and a wireless connection is somewhat iffy (although the Eee PC wireless card works perfectly).

2. Install the required packages. As root (the rest of the steps require you to be root as well):

yast -i syslinux lilo


3. Mount the installation image:

mkdir MOUNT_POINT
mount openSUSE-11.1-NET-i586.iso MOUNT_POINT -o loop


where MOUNT_POINT is any directory and openSUSE-11.1-NET-i586.iso is the name of the file that you downloaded.

4. The mkbootdisk program copies the files to the USB drive and makes the drive bootable.

mkbootdisk --32 --partition /dev/USB_DRIVE MOUNT_POINT


where USB_DRIVE is the partition you want to copy the files to. On my system, this was /dev/sdc1. If you're installing a 64-bit system, then replace "--32" with "--64".

5. Do a little bit of cleanup:

umount MOUNT_POINT && rmdir MOUNT_POINT



You now have a bootable USB drive with a copy of the openSUSE installation media.

Thursday, June 18, 2009

A *rough* guide to getting the Digipro T-8000U Tablet to work with OpenSUSE

This post has been superseded by this new one. It fixes the bugs that were in this one.

This guide will get the Digipro T-8000U tablet working with OpenSUSE 11.1. However, following this guide will break a couple of things:

-X will fail if the tablet is not plugged in.
-Gimp will not longer allow you to draw with the mouse.

I think I know how to fix both. I'll test and post fixes for both over the weekend. Also, the calibration that I give here is specific to my tablet. I had to write my own code to get the correct calibration (the calibration included with the WizardPen driver doesn't handle pressures correctly). I'll post that as well this weekend.

1. Install the necessary prerequisites. As root:

yast -i gcc xorg-x11-devel xorg-x11-server-sdk


2. Get the latest version of the WizardPen driver, which as of now is 0.7.0a2. It is available on a Microsoft website (don't ask me why):

http://cid-43438aff38d34c29.skydrive.live.com/self.aspx/Public/wizardpen/wizardpen-0.7.0-alpha2.tar.gz

Untar, configure, and compile it. Do not install it.

tar xvzf wizardpen-0.7.0-alpha2.tar.gz
cd wizardpen-0.7.0-alpha2
./configure && make


3. The Makefile installs the driver where Xorg won't find it, so you have to install it manually. As root:

cp src/wizardpen_drv.la /usr/lib/xorg/modules/input
cp src/.libs/wizardpen_drv.so /usr/lib/xorg/modules/input


4. You need to find the device associated with the tablet. First, do:

grep Name /proc/bus/input/devices


And look for something that looks like your tablet. Mine was "Aiptek". Then do:

lshal | grep "input.product = 'Aiptek'" -A 20 -B 20 | grep input.device


This should produce a single line with your device. Mine was /dev/input/event7. If this doesn't work, do a "lshal" and search through the output until you find the string.

5. As root, edit /etc/X11/xorg.conf. Add a section that reads:

Section "InputDevice"
Driver "wizardpen"
Identifier "WizardTablet"
Option "Device" "/dev/input/event7"
Option "TopX" "0"
Option "TopY" "0"
Option "TopZ" "54"
Option "BottomX" "3000"
Option "BottomY" "2200"
Option "BottomZ" "511"
Option "MaxX" "3000"
Option "MaxY" "2200"
Option "MaxZ" "511"
EndSection


And in the ServerLayout section, add a line that reads:

  InputDevice  "WizardTablet"  "AlwaysCore"


---===Very Important===--- My xorg file already had two input device sections, one for my touchpad and another where it had detected the tablet as a mouse. I had to delete that extra mouse section before the tablet would be correctly recognized. Otherwise, Xorg uses the wrong mouse driver and your tablet will only half work.

6. Reboot.


The tablet should be working now. To use it in Gimp, you need to go to Edit, Preferences, Input Devices, Configure Extended Input Devices, and for the WizardTablet device, set the Mode to Screen.

Like I said above, this is a very rough guide. I'll clean it up in a few days and repost then.

Friday, June 12, 2009

TV Antenna Design

I want to mount my TV tuner inside my laptop and I also want to build an internal antenna. In my area, TV stations are in the frequency range from 500MHz to 750MHz (as determined from my channels.conf file) which according to this online calculator puts my ideal antenna length (for a dipole antenna) at between 90 and 150 mm per half.

I wanted to get a better feel for the best antenna length to use, so I rigged an antenna like this:



and I scanned for channels several times, each time cutting 4 mm off of the end of each lead. I recorded the number of channels detected with each setup and then made a plot of the results:



While 128 mm was the best length, there is quite a bit of noise in the plot, so its hard to say for certain that the best length might not be somewhat shorter or longer. There definitely is a tendency for the longer antennas to do better than the shorter ones. Based off of this, I'll probably end up going for a 130 mm antenna, if I can make it fit in the case.

Also interesting: The 128 mm antenna picked up 34 stations, while the high quality antenna that came with the tuner picked up 37 channels. Overall, I think that is pretty good for an antenna made out of a couple spare pieces of wire.

Thursday, June 11, 2009

Creating and using an encrypted hard drive or partition

Considering the dropping cost of laptops and external drives, the most damaging part of having one stolen is not the cost of replacing the hardware, but the cost of the data.

You may have proprietary company information, tax returns, financial information, or worse on there. The best way to protect your data is to encrypt it and encrypt it well. Thankfully, Linux makes this really easy.

The encryption I'll show you how to setup uses a block-encryption algorithm, meaning that the encryption respects the underlying filesystem and in case of a hardware failure, such as a bad block, you'll only lose the data stored at that one location. I once had a Windows encryption package that encrypted using the entire volume. One day, an error cropped up in the middle of the encrypted volume and instantly all of my data was gone.

All of the following should be done as root.

1. The first step is to fill the drive with random data. This serves two purposes:

-If the drive previously contained any unencrypted data, you want it gone and unrecoverable.
-If the drive is not filled with random data, an attacker may be able to determine how much of your encrypted partition is in use.

I feel that the first reason is the more important one, so if your drive is new, then you can skip this step. If you are performing this step via a USB connection, it can be incredibly slow; you can expect this step to take up to a week to complete. If you need it done quick, then hook the drive up via SATA or eSATA.

The command to perform is:

dd /dev/random /dev/DEVICE


/dev/random contains high quality random data. You can replace it with /dev/urandom, which is faster, but uses lower quality random data (i.e., its not quite as random).


2. Create the encrypted partition:

cryptsetup --key-size 256 luksFormat /dev/DEVICE


You will be prompted for your passphrase. LUKS does not use your passphrase to encrypt the data, rather it uses your passphrase to secure the key that encrypts the data. As a result, you can setup multiple passphrases or even revoke passphrases. This way, if you ever feel that your passphrase may have been compromised, you can quickly swap it with another passphrase to maintain data security without having to reencrypt the entire drive.

3. Open the encrypted partition:

cryptsetup luksOpen /dev/DEVICE NAME


"NAME" is the name of the unencrypted device and can be any convenient name for this partition. For example, if you're setting up a backup drive, use "backup" to remind yourself what the drive is for.

4. Create the filesystem:

mkfs.ext3 /dev/mapper/NAME


Of course, you can replace mkfs.ext3 with whatever filesystem you want to use.


You're done! Now how to use your drive. To set it up, use:

cryptsetup luksOpen /dev/DEVICE NAME
mount /dev/mapper/NAME MOUNT_POINT


And to shut it down, use:

umount MOUNT_POINT
cryptsetup luksClose NAME


And if you're going to use this for a partition that needs to be mounted at boot, see my note about getting openSUSE to ask for the password at boot.

Wednesday, June 10, 2009

Backup - Part I

"If there's one thing people know they should do, it's backup their data.
If there's one thing people don't do, it's backup their data."

-Jim Fraser


For me, I resisted making backups for a long time because as a former Windows user, I was scarred by how hard it was. In Windows, your data is all over the place (and not in just one directory), backing up Windows itself is just about impossible, most backed up programs won't run after being restored, and there's no easy, integrated way to make backups.

Proprietary software tends to save your files in volumes, so that you have no access, without the software, to the individual files that comprise the backup. If you lose the install disk, your backup is garbage.

Linux makes backing up data so unbelievably easy though, it's almost stupid not to.

Rsync is the tool and it makes perfect copies between filesystems. It is incremental, so the first backup will take awhile, but each one after will only backup the changes. The command to use is:

rsync -a SOURCE DESTINATION


The "-a" switch puts rsync into archive mode, which activates a number of other switches that are useful for performing backups (such as recursing into subdirectories, for example).

Other switches to consider are:

"-x" keeps rsync from crossing filesystem boundaries. This is useful if you're doing a backup of / and you have /home on a different partition and you don't want the other partition included in the backup.

"-H" preserves hard links. The man page says that this is computationally expensive, but in my experience, it has never been a problem. Hard links occur when two files both point to the same location on the disk and they very rarely occur in personal files. If "-H" is not used, then the backup will have two separate files with two copies of the data, instead of two files pointing to the same data. I generally turn this on when I'm backing up / and leave it off when I'm backing up /home.

"-v" is for verbose mode and results in a listing of all the files that are backed up.

"-z" turns on compression of the data stream. This does not compress the backed up data. Generally, I only turn this on if I'm backing up data across a slow connection, such as across the Internet.

"--delete" deletes files at the destination if they don't exist at the source. Turn this on if you want a one-to-one copy of your data.

Tuesday, June 9, 2009

How to fix inconsistent font sizes

I had a problem after installing OpenSUSE 11.1 with KDE 3.5 where the fonts were normal in some programs and way too small in others. In Firefox, the fonts of webpages were normal, but the fonts of the controls (like the web address, File, Edit, View, etc) were too small. The font used for YaST was barely readable.

The fix is pretty easy. Fire up the KDE control center, either by choosing "Configure Desktop" from the Gecko menu or running kcontrol from the terminal. Choose "Appearance & Themes" from the left hand menu and then choose "Fonts". Set "Force fonts DPI" to either 96 DPI or 120 DPI. On my Eee PC, I chose 120 DPI and now all of my fonts are nice and large and readable.

Sunday, June 7, 2009

Create a movie using Fortran

Recently, I posted a method for making images in Fortran. With a couple of commands, its easy to turn those images into a movie.

I'm assuming that you're following my previous hack and you've got a directory of ppm image files, each one containing a single image of the movie you want to create. First, you'll need to convert each one to jpg:

convert IMAGE.ppm IMAGE.jpg


Then convert the jpg's to an avi file. This requires mencoder, which is included with the MPlayer package, which is part of the Packman repositories.

mencoder "mf://*.jpg" -mf fps=1 -o VIDEO.avi -ovc lavc -lavcopts vcodec=wmv2


The video is encoded using wmv2 which is playable on every Windows and Linux machine I've come across. This plays the video at 1 frame per second. You can increase or decrease the fps switch to set the speed at whatever you like (such as "fps=3.5"). The images are read in alphabetical order, so numbering your images 0001.jpg, 0002.jpg, and so on is not a bad idea.

Here's movie that I recently created as part of my thesis research:

video

It probably doesn't make a lot of sense what's going, but the important thing is that this entire video was created using a couple hundred lines of Fortran, my data files, and a couple of scripts.

Saturday, June 6, 2009

Force processes to a specific processor (processor affinity)

In OpenSUSE 11.1, you can use the taskset command to force a process to a single processor or set of processors. Some race condition bugs in Wine may be helped by fixing Wine to a single processor. To do so, use:

taskset -c 0 PROGRAM


The "0" is the processor to use (starting from zero). Another possibility includes "-c 0,2-3" which would run the program on the first, third, and fourth processors.

Friday, June 5, 2009

Make OpenSUSE 11.1 ask for encrypted volume password during boot

11.1 is the first version of OpenSUSE where I've had to follow these steps, specifically the last part about adjusting the system services. If you have an encrypted filesystem in your /etc/crypttab and /etc/fstab, but don't adjust the system services, then it won't ask for your password on boot and the boot will fail when it tries to mount the encrypted partitions.

If you have a LUKS encrypted filesystem and you want to mount it during boot, then you'll need to add an entry to /etc/fstab like this:

/dev/mapper/DEVICE /MOUNTPOINT ext3 defaults 0 0


An entry to /etc/crypttab like this:

DEVICE /dev/disk/by-id/DISK none none


And turn on the boot.crypto-early service in YaST. Go to YaST, then "System", and then "System Services (Runlevel)".



Turn on expert mode and then for the boot.crypto-early service, enable it for boot, "B".



Hit "Ok" and you're done.

Thursday, June 4, 2009

Fix for depmod gives "Cannot allocate memory" error

I encountered an error I've never seen before while building a kernel. While doing a "make modules_install" I got this error:

DEPMOD 2.6.29.4-0.1-pae
FATAL: Can't read module /lib/modules/2.6.29.4-0.1-pae/kernel/drivers/net/wireless/b43/b43.ko: Cannot allocate memory


At first, I thought it was a symptom of building a kernel on a low memory machine, but I upped the swap space and tried again, only to get stuck in the exact same spot. It turns out the error is easily fixed by turning off the debug option in the /usr/src/linux/.config file. You can do this either with a text editor or "make xconfig" (or "make menuconfig", etc).

You want to replace the line:

CONFIG_DEBUG_KERNEL=yes


with:

#CONFIG_DEBUG_KERNEL is not set


It is near the end of .config.

In the configuration programs, the option to turn off is "Kernel hacking | Kernel debugging".

Tuesday, June 2, 2009

SSH / login welcome message

For as long as I've used it, SUSE has welcomed me as I logged in with a message to "Have a lot of fun". The message is kind of pointless, but its easy to change. It is stored in /etc/motd (message of the day). On shared systems, I use it to post important system updates, like kernel and library updates that might break other's code.

On my own systems, I've found it to be useful to use cron to replace the message with important system stats like the amount of free hard disk space, current usage levels, CPU temperatures, etc.

Sunday, May 31, 2009

Quick and dirty image creation in Fortran

Well, maybe not dirty image creation. Let's make that "quick and easy" image creation instead.

When you're analyzing tons of data, Gnuplot is an indispensable tool. But sometimes you just want to create an image quickly and easily. Gnuplot is also very fussy about the format of its data for 3D plots and its not always practical to reformat your data to fit Gnuplot's needs. The PPM format is in ASCII, which makes reading and writing it extremely straightforward. An example PPM file might be:

P3
2 3
255
255 0 0
0 0 0
0 255 0
128 128 128
0 0 255
255 255 255


-The first line sets the format of the PPM file to be ASCII.
-The second line sets the image to be 2 columns wide and 3 rows tall.
-The third line sets the largest color to 255.
-Each line after is an RGB triplet defining one pixel.

And it looks something like this:



I've blown this image up by a factor of 100 in each direction.

A quick Fortran program which creates this image is:

        program view

open(unit=8, file="image.ppm")

write(8,100) "P3"
write(8,100) "2 3"
write(8,100) "255"
write(8,101) 255, 0, 0
write(8,101) 0, 0, 0
write(8,101) 0, 255, 0
write(8,101) 128, 128, 128
write(8,101) 0, 0, 255
write(8,101) 255, 255, 255

close(8)

100 FORMAT(a)
101 FORMAT(3(i3,1x))

end program


One other quick note: I used Image Magick's "convert" command to convert from the PPM format to GIFs. To simply convert from PPM to GIF (or JPG, etc), do this:

convert image.ppm image.gif


To hard scale it:

convert -scale 200x300 image.ppm image.gif


And you can also soft scale it:

convert -geometry 200x300 image.ppm image.gif


Which produces a final product that looks like this:

Tunneling VNC through SSH

VNC is great for accessing remote machines, but it is inherently insecure. By tunneling through SSH, you can get the security of SSH with almost all of the speed of VNC.

1. Create the tunnel:

ssh -L 5901:localhost:5901 USER@SERVER_ADDRESS -C


This will open up a new ssh session. Use this session to start the VNC server. The first 5901 is the port to use on the local machine and the second 5901 is the port to use on the remote machine. The -C switch turns on compression; without it, the VNC session is intolerably slow.

2. Start the VNC viewer:

vncviewer localhost:1


You're telling the VNC client to connect to your own machine. But the VNC port on your machine has been forwarded to the remote server, so the connection is made.

Saturday, May 30, 2009

Use VNC to view remote desktops

Normally, SSH is a great tool to administer a remote computer. However, if you need access to X programs, you have to use the -X switch which is terribly slow, even over broadband.

VNC isn't super-fast, but it is usable. It can be sped up somewhat by minimizing the size and color depth of the desktop window you use.

Fortunately, openSUSE 11.1 includes a VNC server and client under Applications | System | Remote Access. Unfortunately, the VNC server is severely buggy and incorrectly reproduces the server's desktop. It is bad enough to be unusable.

However, it is easy to start and connect to a VNC server from the console. For this walkthrough, the server is the remote computer you want to connect to and the client is your computer.

1. Install the necessary software on both machines. In this case, you''ll need TightVNC. It is usually installed by default. Check this using:

rpm -qa | grep tightvnc


If nothing comes up then install it using:

sudo zypper install tightvnc



2. You need to open TCP port 5901 on the server and every router/modem it is behind. For example, your connection might look like this:

client---client's router---cable modem---internet---dsl modem---server's router---server

You need to open 5901/TCP on the dsl modem, the server's router, and the server. To do this on the dsl modem and the router, you'll need to refer to the manuals that came with those products. For the server you can do this through the appropriate YaST module:

sudo /sbin/yast firewall


3. Start the vncserver on the server.

vncserver


The first time you do this you'll be prompted to enter a password (read my note on VNC security at the end first). You'll also be asked if you want to setup a view-only password; choose 'no'.

4. Start the vncviewer on the client. You'll need the IP address of the server. Here, I'm assuming its 192.168.1.50:

vncviewer 192.168.1.50:1


Enter the password and you'll be connected to the server.



A few notes on this:

-VNC does not use encryption by default. If you're going to use the above setup unmodified, then keep in mind that your password should not be the same as your root password. It could be easy for an attacker to capture that password and then connect remotely to the system, so make sure that you shut down the VNC immediately after you're finished using it. Also, start the VNC server not as root.

-You can shutdown the VNC on the server by using:

vncserver -kill :1


-The ":1" is the desktop number. If you start multiple VNC servers, each one will have a different number. vncserver by default starts with 1 and increments up. The port for each desktop is 5900+number. So that's why we needed port 5901 for desktop number 1.

-You can speed things up by sending a smaller desktop across the connection. Try this vncserver command instead of the one in step #3.

vncserver -depth 8 -geometry 800x600

Friday, May 29, 2009

Faster Ubuntu performance on Celeron Eee Pc 700's and 900's

The Netbook remix of Ubuntu 9.04 tends to run a little slow on Celeron-based Eee PCs, which includes almost all Eee PCs released prior to May 2008. On the 700 and the 900, you can speed up performance by installing a custom, optimized kernel.

1. Get the .deb files:

wget http://people.ubuntu.com/~apw/lp349314-jaunty/linux-headers-2.6.28-11-generic_2.6.28-11.43~lp349314apw5_i386.deb
wget http://people.ubuntu.com/~apw/lp349314-jaunty/linux-image-2.6.28-11-generic_2.6.28-11.43~lp349314apw5_i386.deb


2. Install them:

sudo dpkg -i linux-headers-2.6.28-11-generic_2.6.28-11.43~lp349314apw5_i386.deb linux-image-2.6.28-11-generic_2.6.28-11.43~lp349314apw5_i386.deb


3. Reboot.



The system should now feel more responsive. Here's a link to the bug page describing why the default kernel performance is slow:

https://bugs.edge.launchpad.net/linux/+bug/349314

Thursday, May 28, 2009

Sed for Blogger

Blogger/Blogspot has an unfortunate error that if you type a list like this:

test1
test2
test3

directly into a <pre> block, it looks something like this:

test1

test2

test3


As you can see, Blogger likes to mangle the newlines, effectively double-spacing the text. The solution is to remove all of the newlines and then replace them with <br> tags. There are a couple of online tools to do this, but its even easier using this sed command line:

sed 's/</\&lt;/g' INPUT_FILE | sed 's/>/\&gt;/g' | sed ':a;N;$!ba;s/\n/<br>/g'


The first sed converts < to the HTML-equivalent &lt;. The second sed does the same thing for >. The third one removes all newlines and replaces each one with <br>. The result is a single line that can be embedded in a pre and that will show up correctly in Blogger (or anywhere else, newline bug or not):

test1
test2
test3

Wednesday, May 27, 2009

TV Tuner playback for netbooks

Realtime playback of HDTV may not be an option on netbooks because of the large amount of data in the HDTV stream and the low processing power of a netbook. If realtime playback isn't an option, it's fairly easy to watch the program by:

1. Record the broadcast
2. Convert the broadcast to a lower bitrate
3. Watch the conversion

Steps:

1. You'll need two console windows open. In the first console window, change the tuner to the correct channel using:

azap -c channels2.conf -r -a 0 "WGN-DT"

where "0" is the number of your tuner (it's zero, unless you have multiple tuners) and "WGN-DT" is the channel you want to watch. The channel name can be found in the channel.conf file. If you don't have a channels.conf file, generate one now using:

scan /usr/share/dvb/atsc/us-NTSC-center-frequencies-8VSB -o zap -a 0 | tee ~/channels.conf

Again, "-a 0" specifies the tuner you're using. Error messages about "tuning failed" are normal.

2. In the second console window, record the program:

cat /dev/dvb/adapter0/dvr0 > Input.mpg

If you have multiple tuners, replace adapter0 with the appropriate adapter.

3. Convert the video to something reasonable using ffmpeg:

ffmpeg -i Input.mpg -s 720x480 -b 800k -vcodec mpeg2video -ac 2 -ab 128k -acodec libmp3lame Output.mpg

If you get a "could not find codec parameters" error, then install the Packman version of ffmpeg.

4. Play the video back:

mplayer Output.mpg

Although you could get better results by matching the resolution to the aspect ratio, if you get it wrong, mplayer will stretch the video appropriately to compensate.

Tuesday, May 26, 2009

Sabrent TV-DGUSB

Woot recently sold the Sabrent TV-DGUSB USB HDTV (ATSC) Tuner for $30. This card/dongle has horrible support. A Google for it turns up almost no pages and I couldn't even find it on Sabrent's own website. Here's the information I do have on it:

Product: Sabrent TV-DGUSB
ID: 05e1:0480
lsusb says: Syntek Semiconductor
Chips: Auvitek au0828
Chips: Auvitek au8524

A similar card with id 05e1:0400 is supported with code from LinuxTV.org. The cards differ in that this one has the au8524 and the driver supported one has a au8522. It turns out that the difference is minimal enough that we can use the driver from one for the other.

Steps to get it working in openSUSE 11.1:

1. Add the Packman repositories.
2. Get the kernel-source, dvb, and vlc packages.
3. Get http://linuxtv.org/hg/~mkrufky/teledongle/archive/tip.tar.bz2.
4. Unbzip it.
5. Change line 222 of linux/drivers/media/video/au0828/au0828-cards.c from 0x0400 to 0x0480. Build it with "make" and "make install" as root.
6. Reboot (yes, its necessary).
7. Plug in the card. You can verify everything is working by making sure the blue light on the unit is on, the au0828 module is installed ("lsmod | grep au0828"), or "dmesg | tail" should read:


usb 2-6: new high speed USB device using ehci_hcd and address 9
usb 2-6: configuration #1 chosen from 1 choice
Manufacturer ID= 0xff, Chip ID = ffff. It is not a TEA5761
tuner' 9-0042: chip found @ 0x84 (au0828)
tda9887 9-0042: creating new instance
tda9887 9-0042: tda988[5/6/7] found
tuner' 9-0043: chip found @ 0x86 (au0828)
tda9887 9-0043: creating new instance
tda9887 9-0043: tda988[5/6/7] found
tuner' 9-004a: chip found @ 0x94 (au0828)
tda9887 9-004a: creating new instance
tda9887 9-004a: tda988[5/6/7] found
tuner' 9-004b: chip found @ 0x96 (au0828)
tda9887 9-004b: creating new instance
tda9887 9-004b: tda988[5/6/7] found
Chip ID is not zero. It is not a TEA5767
tuner' 9-0060: chip found @ 0xc0 (au0828)
tuner' 9-0061: chip found @ 0xc2 (au0828)
tuner' 9-0062: chip found @ 0xc4 (au0828)
tuner' 9-0063: chip found @ 0xc6 (au0828)
tuner' 9-0064: chip found @ 0xc8 (au0828)
tuner' 9-0065: chip found @ 0xca (au0828)
tuner' 9-0066: chip found @ 0xcc (au0828)
tuner' 9-0067: chip found @ 0xce (au0828)
tuner' 9-0068: chip found @ 0xd0 (au0828)
tuner' 9-0069: chip found @ 0xd2 (au0828)
tuner' 9-006a: chip found @ 0xd4 (au0828)
tuner' 9-006b: chip found @ 0xd6 (au0828)
tuner' 9-006c: chip found @ 0xd8 (au0828)
tuner' 9-006d: chip found @ 0xda (au0828)
tuner' 9-006e: chip found @ 0xdc (au0828)
tuner' 9-006f: chip found @ 0xde (au0828)
au0828: i2c bus registered
tda18271 9-0060: creating new instance
TDA18271HD/C2 detected @ 9-0060
DVB: registering new adapter (au0828)
DVB: registering adapter 1 frontend 0 (Auvitek AU8522 QAM/8VSB Frontend)...
Registered device AU0828 [Syntek Teledongle [EXPERIMENTAL]]
usb 2-6: New USB device found, idVendor=05e1, idProduct=0480
usb 2-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 2-6: Product: USB 2.0 Video Capture Controller
usb 2-6: Manufacturer: Syntek Semiconductor

8. You'll need a list of valid channels to use. In the USA, use (not as root):

scan /usr/share/dvb/atsc/us-NTSC-center-frequencies-8VSB -o zap -a 0 | tee ~/channels.conf

You will get a lot of messages about "tuning failed". Ignore these. It just means that there is no station on that channel. If you have multiple TV tuners in your system (I did), then you'll need to tell the scan command which one to use using the "-a" switch. 0 is the first tuner, 1 is the second and so on.

When this is done, channels.conf should have a few (or many) lines, each one corresponding to a local TV channel.

9. Start VLC:

vlc --color --ttl 12 --dvb-adapter=0 channels.conf

Again, if you have multiple adapters, tell vlc which one to use with the "dvb-adapter" switch. The next and previous buttons change the channels.



If all went well, then you're watching live high definition TV. If the video is blocky or cutting out then either your processor is too slow (more on this in a moment) or your signal is not sufficient. The antenna that comes with the tuner is pretty small. I found that sometimes putting the antenna on its side improves the reception significantly.

On my 2.2GHz 4-year-old processor, watching the broadcast full screen took 25% of the processor time (using top). This is pretty good and better than BlazeVideo (the Windows software that comes with the card) that took closer to 50%. This means HDTV should be doable on just about every recent computer, maybe even netbooks. I'll try that next and report back.

I should note that some channels use more processing power than others. On my laptop (a five-year-old Sempron at 1.8GHz), the processor usage per channel is constant, but from channel to channel to varies from 15% to 80%.

Here are some websites that I found helpful. The device pictured in the first link is the previous chip version of this one, the au8522:

http://www.linuxtv.org/wiki/index.php/Sabrent_TV-USBHD
http://linuxtv.org/hg/~mkrufky/teledongle
http://forum.videolan.org/viewtopic.php?f=13&t=38890
http://forums.opensuse.org/applications/multimedia/405117-sabrent-digital-hdtv-atsc-analog-usb.html
http://linuxtv.org/wiki/index.php/ATSC_USB_Devices

GPS on openSUSE 11.1

Getting a gps to work on Linux is actually fairly easy. The devices usually have an integrated usb-to-serial adapter, so they show up in /dev/ as a serial device. Upon plugging in a BU-353 (which is available on Amazon and I fully recommend), there is a /dev/ttyUSB0.

The steps I used to get the GPS up and fully working were:

1. Install two packages as root:

yast -i gpsd gpsdrive

2. Modify /etc/sysconfig/gpsd. Change GPSD_STARTBYUDEV="no" to "yes".
(If you don't do this, you will have to manually start gpsd every time you plug in the gps. You can do so with "gpsd /dev/ttyUSB0" as root.)

3. Start gpsdrive. Give the GPS a minute or so to achieve a lock. The BU-353 gets a lock usually in 30 seconds and even quicker if it was recently used (it has some onboard short-term memory).