Latest Entries »

I’m heading back to the desert for another deployment with the Army National Guard

I’ve been in the Army National Guard since 2003 and have been lucky enough to serve only one tour in Iraq (2005). In 2008 I moved from Michigan to Texas and in doing so I was forced to switch Army units. The unit that grabbed me up was already deployed and I was happily holding down the fort in the rear detachment. Unfortunately, a quick 3 months later I was scooped up by a different unit 300 miles away for a deployment. I am now in the final stages of my preparation for this deployment and am scrambling to wrap up all my stuff before I leave. Hopefully I will have a lot of time to work on Tera-WURFL when I’m gone; I plan to further abstract the database layer and introduce a MSSQL Database Connector in the coming months. I will let you know once I’m in the sandbox!

Background of my problem

A few days ago my Dell XPS 720 decided to die on me. When I push the power button it turns on for about 1/2 second and then turns off. After much debate, I bought a new power supply for it (non-standard of course [24pin AND 20pin power connectors]). Today I received the power supply and low-and-behold that wasn't the problem. Now I need to buy either a motherboard or a new CPU (the existing one is a Core 2 Quad Extreme 3.0). I decided that buying from Dell was a bad choice and I'll just build myself a Phenom II system. Meanwhile, the XPS 720 shipped with 2x 120GB 10k Raptors in a RAID-0 using the sub-par onboard SATA RAID controller. I would just love to get my data off the system, but I can't get it up and running and I'm not going to sink anymore money into it. I decided to figure out exactly how the data was stored on the array members and try to deinterlace it directly into a raw image that I could then mount via loopback device in Linux and copy all my data back out :) It sounds easy, huh? I found this GREAT little python code snippet that does the heavy lifting for me – thanks Sim

I’ve used UFS2 snapshots on FreeBSD by means of the dump command to make stateful filesystem backups in the past, so I recently wanted to take a similar approach to backup some Linux servers. Most GNU/Linux distributions use ext3 as their root filesystem by default these days which does not natively support snapshots, so I needed to find another way to do it. As it turns out, I used LVM on my main Linux servers to add a layer of virtualization to the storage subsystem. LVM does natively support snapshots, and here’s how to do it.

1. Make room for the snapshot

I ran into a major problem right off the bat – the server I was testing LVM snapshots on had one hard drive with 2 partitions, 1 for /boot and one for LVM. The LVM partition contained one Volume Group called “TERA-VIRTUAL” and this VG had 2 Logical Volumes – “root” (mounted on /) and swap_1 (swap). The root LV had all the PE allocated to it and the filesystem beneath it was using all of the available space. The problem is that since there were no available Physical Extents in the VG (or in my PVs), I couldn’t create a snapshot volume. This is important: when you create a snapshot on LVM you are really creating another Logical Volume that is used to hold the changes to the volume. This volume needs to me in the same Volume Group as the volume you are taking a snapshot of – so you need some extra space in that VG! In my case, I needed to resize the root fs, which was on a Logical Volume – this to me an hour plus to achieve since most LiveCDs either have no support for LVM (gparted-live) or the support is broken (backtrack 3 final usb). I booted ubuntu server 7.10 64bit into rescue mode and started a shell in the installer environment and once at a prompt typed vgmknodes to make the /dev/ entries for my LVs, then used resize2fs to shrink my FS and lvreduce to shrink my LV, leaving me with 23G of available Physical Extents in my VG (seriously, if you don’t know what I’m talking about, don’t try it – you will trash your hard drive!).

2. Create the snapshot and run backup

To create a snapshot you first need to figure out how much space you need – it is critical that you have enough space to hold all the changes to your volume until the snapshot is removed!. For me 10% of my 200G is fine – 20G.
The process is to create a snapshot LV, mount it, do your backup, unmount it, and remove the snapshot:

root@TERA-VIRTUAL:~# lvcreate --size 20G --snapshot --name snap /dev/TERA-VIRTUAL/root
Logical volume "snap" created
root@TERA-VIRTUAL:~# lvs
LV     VG           Attr   LSize   Origin Snap%  Move Log Copy%
root   TERA-VIRTUAL owi-ao 200.00G
snap   TERA-VIRTUAL swi-a-  20.00G root     0.00
swap_1 TERA-VIRTUAL -wi-ao   9.46G
root@TERA-VIRTUAL:~# lvdisplay /dev/TERA-VIRTUAL/snap
--- Logical volume ---
LV Name                /dev/TERA-VIRTUAL/snap
VG Name                TERA-VIRTUAL
LV UUID                fFbaH4-22Hq-s7a2-e1mo-DPov-8wff-dvuAiI
LV Write Access        read/write
LV snapshot status     active destination for /dev/TERA-VIRTUAL/root
LV Status              available
# open                 0
LV Size                200.00 GB
Current LE             51200
COW-table size         20.00 GB
COW-table LE           5120
Allocated to snapshot  0.01%
Snapshot chunk size    8.00 KB
Segments               1
Allocation             inherit
Read ahead sectors     0
Block device           254:2
root@TERA-VIRTUAL:~# mount /dev/TERA-VIRTUAL/snap /backup_snapshot/
root@TERA-VIRTUAL:~# cp -Rf /backup_snapshot/VirtualMachines/TERA-WEBSERVER /backup/
root@TERA-VIRTUAL:~# lvdisplay /dev/TERA-VIRTUAL/snap
--- Logical volume ---
LV Name                /dev/TERA-VIRTUAL/snap
VG Name                TERA-VIRTUAL
LV UUID                fFbaH4-22Hq-s7a2-e1mo-DPov-8wff-dvuAiI
LV Write Access        read/write
LV snapshot status     active destination for /dev/TERA-VIRTUAL/root
LV Status              available
# open                 1
LV Size                200.00 GB
Current LE             51200
COW-table size         20.00 GB
COW-table LE           5120
Allocated to snapshot  1.96%
Snapshot chunk size    8.00 KB
Segments               1
Allocation             inherit
Read ahead sectors     0
Block device           254:2
root@TERA-VIRTUAL:~# umount /backup_snapshot/
root@TERA-VIRTUAL:~# lvremove /dev/TERA-VIRTUAL/snap
Do you really want to remove active logical volume "snap"? [y/n]: y
Logical volume "snap" successfully removed

As you can see from the transcript above, you can keep an eye on your snapshot size by using lvdisplay and looking for “Allocated to snapshot”. As seen above, before the backup the snapshot volume was 0.01% full, but by the time my backup finished, it was up to 1.96%.

Now I’ve got a sane copy of my data without having to dismount my filesystem!

Having done Linux administration for over 10 years now, I use the watch command almost every day. On Linux, watch will run a command every couple seconds. It’s particularly nice for watching files change:

# watch ls -lah

This example will keep showing the output of ls -lah every 2 seconds.

About a year ago I switched a couple servers over to FreeBSD 6.1 because of it’s security features and more stable development model. For the last year I’ve been occasionally using the watch command on FreeBSD and getting an annoying error like this:

[root@devel /backup]# watch ls -lah
watch: fatal: bad device name

Today it finally occured to me to look at the manpage:

WATCH(8)                FreeBSD System Manager's Manual               WATCH(8)

NAME
watch -- snoop on another tty line

SYNOPSIS
watch  -cinotW -f snpdev tty

DESCRIPTION
The watch utility allows the user to examine all data coming through a
specified tty using the snp(4) device.  If the snp(4) device is not
available, watch will attempt to load the module (snp).  The watch utility writes to standard output.

It hardly looks like the watch I know! It turns out that FreeBSD has a different use for watch – snooping on other consoles. On FreeBSD, the GNU watch (like on Linux) is called gnu-watch. If you don’t want to install it, you can easily whip up a poor-man’s replacement:

#!/usr/local/bin/bash
while [ 1 -lt 2 ]
do
clear
date
echo ------------------------------------
eval $@
sleep 2
done

TIP: To replace the default ‘watch’ with this script, save the above script as /root/watch, then make it executable, chmod +x /root/watch and alias it in your shell, alias watch=’/root/watch’. You can save this alias permenantly with the bash shell by adding it to ~/.bash_profile .

Here’s a sample output from the poor-man’s watch:

Thu Jul 10 22:09:27 EDT 2008
------------------------------------
total 17113832
drwxr-xr-x   3 root  wheel   512B Jul 10 21:03 .
drwxr-xr-x  21 root  wheel   512B Jul 10 18:25 ..
-rw-r--r--   1 root  wheel    97M Jul 10 15:41 archive1.tar
-rw-r--r--   1 root  wheel    40M Jul 10 15:41 archive1.tar.bz2
-rw-r--r--   1 root  wheel   308M Jul 10 17:19 archive2.tar
-rw-r--r--   1 root  wheel    93M Jul 10 17:19 archive2.tar.bz2
-rw-r--r--   1 root  wheel   600M Jul 10 17:13 archive2a.tar
-rw-r--r--   1 root  wheel   192M Jul 10 17:13 archive2a.tar.bz2
-rw-r--r--   1 root  wheel   3.3G Jul 10 19:09 archive2b.tar
-rw-r--r--   1 root  wheel   1.4G Jul 10 19:09 archive2b.tar.bz2
-rw-r--r--   1 root  wheel   5.4G Jul 10 19:33 archive2c.tar
-rw-------   1 root  wheel   1.9G Jul 10 22:09 archive2c.tar.bz2
-rw-r--r--   1 root  wheel   2.1G Jul 10 17:44 archive2d.tar
-rw-r--r--   1 root  wheel   886M Jul 10 16:08 archive3.tar
drwxr-xr-x   2 root  wheel   512B Jul 10 19:44 scripts

I use Adobe Creative Suite 3 (CS3) Web Premium every day, Flash, Fireworks, Photoshop, but most of all Dreamweaver. I consider Dreamweaver CS3 to be the best HTML/CSS visual editor available and a it’s a very handy FTP client as well. It also serves as a nice text editor with syntax highlighting for PHP (although PDT for Eclipse is MUCH better). My problem is that it crashes on me almost every day with this really annoying error: Runtime Error! This application has requested the Runtime to terminate it in an unusual way. Here’s what it looks like:

This error makes my blood boil! I’ve done everything to try it including, but not limited to:

Delete the configuration Files

This seems to give me the most success. Dreamweaver stores a file cache and some non-site related settings in C:\Users\[your username]\AppData\Roaming\Adobe\Dreamweaver 9\Configuration. If you delete or rename this directory then Dreamweaver will recreate it when it starts up.

Delete the Dreamweaver Windows Prefetch file

When a program starts up, Windows trys to optimize the startup process by creating a “cache” type file in C:\windows\Prefetch. Next time the program opens it uses this file to speed up the process. You can safely delete everything in the Prefetch folder. Doing this helped me get Dreamweaver working yesterday.

Get rid of Logitech SetPoint

Someone on an Adobe forum said this worked for them – I haven’t had much success

Restart computer with only essential processes enabled

Also from Adobe, this seems to have no impact on my computer – it still crashes.

Maybe it’s the 8KB file bug???

According to many people on Adobe’s forum, if you a working on a file that is exactly 8,192 bytes (8KB) Dreamweaver will crash. Well, my files are taking up 8,192K because the block size on my disk pushes 7+K files up to 8,192 (look at “size on disk”), but I put a huge HTML comment at the end of the file to push it past 10K and it didn’t help.

Doing the exact same thing every time will make Dreamweaver crash every time

Here’s my most effective solution as of this morning: once Dreamweaver crashes, you can’t do the exact same thing when you reopen it or it will just keep crashing. I was trying to add a 4×3 table to a document and everytime I did Insert->Table->OK it immediately crashed (I did this 10+ times while changing other Windows stuff). Then I went to show my business partner and instead of adding a table, I went to a different Site and changed some code around, then when back to the original site and BAM! it works fine now! What the HECK is going on here?

Final thoughts

You know, I’ve been working with Linux and open source software for over 10 years, and I finally break down and by a $2,000+ software package – I expect it to work! If it doesn’t work I expect that I can get support to make it work (and not that “please restart your computer then uninstall/reinstall” BS)! Come on Adobe!!! If I charge my clients >$100/hr and I have to fight with Dreamweaver for 1 hr per day, where’s my $36,500 from Adobe for the last year?

P.S. I think the problem is related to MS Visual Studio 2005′s runtime update and Vista or something.

Although I use AMFPHP RemoteObjects with the Cairngorm Framework everyday, I never had a need for a simple LocalConnection. LocalConnections let you communicate between running SWFs, the only problem is that they are unidirectional. SWF A can make a new LocalConnection to SWF B and invoke it’s methods, but SWF B can’t contact SWF A. The way to get around this is to make another LocalConnection back from SWF B to SWF A. Trying to wrap my head around receiving and sending connections was starting to make me angry! They are annoyingly misleading – the receiving SWF needs to .connect() to a named connection, whereas the sending SWF doesn’t – it just calls .send() with the same named connection. I figured I could do the world a favor and abstract this confusion for you.

Here is the result (demonstrated with a Flex SWF and a Flash SWF):

Flex SWF


Flash SWF



In both the Flex MXML and the Flash FLA, I am including my BiDirLocalConnection class and it does all the hard work for me.

To make this bidirectional concept easier to handle, I introduced something called Roles. There are two roles available to your SWFs: Master and Slave. You must pick one of these roles when you instantiate the BiDirLocalConnection Object. If you have more than one running SWF in a given Role you will get an error on the newest one.

Here’s an example of how to use the class in Flex:

import net.teratechnologies.common.BiDirLocalConnection;

private var connection:BiDirLocalConnection;

private function init():void{
    connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_MASTER,this);
    connection.connect();
}

The constructor takes three arguments (the third is optional):

BiDirLocalConnection(role:String,callbackScope:Object,connectionBaseName:String="BiDirConnection")

- role: The role of this SWF (ROLE_MASTER or ROLE_SLAVE)
- callbackScope: When a SWF connects to this one and tries to invoke a method (call a function), where should it look to find the method? Normally you specify this so the remote SWF has access to the functions in your current scope.
- connectionBaseName: This optional argument is only required if you have more than one BiDirLocalConnection at the same time. You can use any string here and it will serve as the prefix for the two LocalConnection connectionNames that are used.

Here is the complete code from the Slave in Flash CS3 as seen above. There are three things on the stage: TextArea (receiveText), TextInput (sendText) and a Button (sendButton):

import net.teratechnologies.common.BiDirLocalConnection;
           
var connection:BiDirLocalConnection;

function init():void{
    sendButton.addEventListener(MouseEvent.CLICK,send);
    connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_SLAVE,this);
    connection.connect();
}
function send(e:Event):void{
    connection.send('showText',sendText.text);
}
function showText(t:String):void{
    receiveText.text = "Received: "+t+"\n"+receiveText.text;
}

init();
stop();

Now here is the complete code for the Flex MXML file:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" creationComplete="init()">
    <mx:Script>
        <![CDATA[
            import net.teratechnologies.common.BiDirLocalConnection;
           
            private var connection:BiDirLocalConnection;
           
            private function init():void{
                sendButton.addEventListener(MouseEvent.CLICK,send);
                connection = new BiDirLocalConnection(BiDirLocalConnection.ROLE_MASTER,this);
                connection.connect();
            }
            private function send(e:Event):void{
                connection.send('showText',sendText.text);
            }
            public function showText(t:String):void{
                receiveText.text = "Received: "+t+"\n"+receiveText.text;
            }
           
        ]]>
    </mx:Script>
    <mx:Panel width="100%" height="100%" layout="absolute" title="Flex 2/3">
        <mx:TextArea id="receiveText" right="10" bottom="39" left="10" top="10."/>
        <mx:TextInput id="sendText" left="10" right="72" bottom="10"/>
        <mx:Button label="Send" id="sendButton" right="10" bottom="10"/>
    </mx:Panel>
   
</mx:Application>

It really is quite simple, the SWFs are calling each other’s “showText” functions with the text of the TextInput box as an argument. You can pass as many arguments as you want to connection.send(), and they don’t need to be simple Strings.

Finally, here is the ActionScript Code for the BiDirLocalConnection class itself:

package net.teratechnologies.common {
    import flash.net.LocalConnection;
   
    public class BiDirLocalConnection{
       
        public static const ROLE_MASTER:String = "master";
        public static const ROLE_SLAVE:String = "slave";
       
        private var txConnName:String;
        private var rxConnName:String;
        private var txLC:LocalConnection = new LocalConnection();
        private var rxLC:LocalConnection = new LocalConnection();
        private var callbackScope:Object;
       
        public function BiDirLocalConnection(role:String,callbackScope:Object,connectionBaseName:String="BiDirConnection"){
            if(role == ROLE_MASTER){
                txConnName = connectionBaseName + "_TX";
                rxConnName = connectionBaseName + "_RX";
            }else{
                rxConnName = connectionBaseName + "_TX";
                txConnName = connectionBaseName + "_RX";
            }
            this.callbackScope = callbackScope;
        }
        public function connect():void{
            trace(rxConnName);
            rxLC.connect(rxConnName);
            rxLC.client = callbackScope;
        }
        public function send(methodName:String,...rest):void{
            txLC.send(txConnName,methodName,rest);
        }
    }
}

You can also download the Flex 2/3 file, Flash FLA and Class file in a ZIP file.

Quick Linux Tip:

If you’re trying to delete a very large number of files at one time (I deleted a directory with 485,000+ today), you will probably run into this error:

/bin/rm: Argument list too long.

The problem is that when you type something like “rm -rf *”, the “*” is replaced with a list of every matching file, like “rm -rf file1 file2 file3 file4″ and so on. There is a reletively small buffer of memory allocated to storing this list of arguments and if it is filled up, the shell will not execute the program.

To get around this problem, a lot of people will use the find command to find every file and pass them one-by-one to the “rm” command like this:

find . -type f -exec rm -v {} \;

My problem is that I needed to delete 500,000 files and it was taking way too long.

I stumbled upon a much faster way of deleting files – the “find” command has a “-delete” flag built right in! Here’s what I ended up using:

find . -type f -delete

Using this method, I was deleting files at a rate of about 2000 files/second – much faster!

You can also show the filenames as you’re deleting them:

find . -type d -print -delete

…or even show how many files will be deleted, then time how long it takes to delete them:

root@devel# ls -1 | wc -l && time find . -type f -delete
real    0m3.660s
user    0m0.036s
sys     0m0.552s

After battling with Flex 2′s DataGrid ItemRenderer for a few hours, I have finally figured out how they really work.

The Scenario

Let’s say you want to have a DataGrid with a thumbnail in one of the cells. To do this you would assign the DataGridColumn an ItemRenderer. Here’s a quick and dirty MXML ItemRenderer that will work:

<?xml version="1.0" encoding="utf-8"?>
<mx:HBox xmlns:mx="http://www.adobe.com/2006/mxml" width="100%" height="100%" verticalScrollPolicy="off" horizontalScrollPolicy="off">
  <mx:Image id="thumbnail" width="25" height="25"/>
</HBox>

Now, when the DataGrid goes to render the cell the first time, it instantiates the ItemRenderer and sets the “data” property to the data of the current row in the DataGrid. Let’s say you have a property called “thumb_src” in the data. You could just bind the image’s source property to “data.thumb_src” and you probably wouldn’t have any problems, but if you wanted to do anything to the data before you use it you will run into trouble. The problem is that DataGrids only create as many ItemRenderers as are visible in the grid, so if you have 1000 rows in your DataGrid but you can only see 10, only 10 ItemRenderers will be created (more will be created on demand if you resize the DataGrid). Once a cell is scrolled off the top of the grid it is reused at the bottom (and visa-versa). When the ItemRenderer is reused, the DataGrid sets the data property with the new data and that’s it – it is not destroyed re-instantiated.

The Problem

All this becomes a major problem when you use the creationComplete event to setup some stuff on your renderer, since this event is only fired once – then renderer will be used and re-used to display the data from many different cells. When this problem occurs, you will see data in the ItemRenderer cells being thrown around seemingly at random when you scroll the DataGrid – it will probably drive you crazy!

The Solution

You need to setup / clear everything in the renderer when the data property is set. The easiest way to do this is to override the Container’s “data” setter function like this:

public override function set data(value:Object):void{
  // This will fire off the FlexEvent.DATA_CHANGE Event
  super.data = value;
  // if the value is null this cell is empty
  if(value == null){
    // clear all the controls
    return;
  }
  // set the controls with this data
}

You could also add an event listener for FlexEvent.DATA_CHANGE, which is fired off by the Container superclass.

Here’s a great example that I made that shows the problem. You can right click on the application and hit “View Source” to see the code, or go there directly.

So the other day my 20.1″ Widescreen Viewsonic monitor started acting funny and turning off on me (this is my 2nd one that has died), so I replaced it with a 22″ Widescreen Xbrite and the new monitor rocks! I figured I would send the Viewsonic back again and have it replaced again, but I thought I would give it another shot as-is first. It seems to run fine unless I leave it on for 12-24 hours at a time so I decided to take it to my office.

Since I already have 2x 17″ Flat Panels at my office, I was going to need another video card to support more than 2. After a little researching on the net, I jumped on Newegg.com and found myself a couple dual-head PCI video cards to throw in with my ATI Radeon X1650 PCIe. I got 2 because the only Radeon series for under $100 were either 1xDVI+1xVGA or 2xDVI Low Profile. I got one of each. The first one I received was a VisionTek Radeon 7000 64MB PCI ($34.99). I installed the card and plugged in my 20.1″ Viewsonic and turned on the computer – no joy. I installed the drivers and prayed it wouldn’t mess up my X1650. Drivers installed no problem, but still no display. Restarted – no display. Now I could see the other card and the monitors in the dislpay properties, but when I “attached” them they didn’t do anything. I restarted it again and went into the BIOS and changed Init Display First from PCIe to PCI Card. I saved the changes and booted into windows – whalah! 3 monitors – all playing nicely :) . The computer seems like it’s going to die when it starts up because everything slows down to a crawl for a little while, but then it works as expected. This 3-monitor setup seems very efficient to me. I’ll have to report back after I’ve used it for a while.

My Setup
ATI Radeon X1650 PCIe 512MB Dual Head (2x DVI)
ATI Radeon 7000 PCI 64MB Dual Head (1x DVI, 1x VGA)
2x Dell 17″ Flat Panels
1x Viewsonic 20.1″ Widescreen Flat Panel

I happened to be browsing around tonight and noticed that NVIDIA has been giving away 1 prize per day for a year! Here's the catch – you need to visit the site every 7 days to see if you won or your prize will go to the next guy! Stupid! To make matters worse, only way to see if you won is to use this annoying marquee on their site!

http://www.slizone.com/object/slizone_sli_giveaway.html

So naturally I decided to make everyone's life a little easier and pumped out a handly little Flex2 app that shows who won for the last week. It is updated in real time since I'm just grabbing the winners off the same RSS feed that's feeding the NVIDIA website. Check it out – it's cool, it's Flex, and it might just save your butt!

http://devel.teratechnologies.net/nvidia_winners/