Linux

Unfinished Drafts: Useful Utility: tar

This is another article that sat in the drafts folder for far too long- Last edited Feb 21st, 2006.

 

I fear writing about tar, and that is why I’m determined to finish it in this sitting, so it won’t fester and scare me off of this series. Why am I scared of writing about tar? Well, this is their flags list verbatim from the man page:

       [  --atime-preserve  ] [ -b, --blocking-factor N ] [ -B, --read-full-records ] [ --backup BACKUP-TYPE ] [ --block-com-
       press ] [ -C, --directory DIR ] [ --check-links ] [ --checkpoint ] [ -f, --file [HOSTNAME:]F ] [ -F,  --info-script  F
       --new-volume-script F ] [ --force-local   ] [ --format FORMAT ] [ -g, --listed-incremental F ] [ -G, --incremental ] [
       --group GROUP ] [ -h, --dereference ] [ --help ] [ -i, --ignore-zeros ] [ --ignore-case ] [ --ignore-failed-read  ]  [
       --index-file  FILE  ]  [ -j, --bzip2 ] [ -k, --keep-old-files ] [ -K, --starting-file F ] [ --keep-newer-files ] [ -l,
       --one-file-system ] [ -L, --tape-length N ] [ -m, --touch, --modification-time ] [ -M, --multi-volume ] [ --mode  PER-
       MISSIONS  ]  [  -N,  --after-date DATE, --newer DATE ] [ --newer-mtime DATE ] [ --no-anchored ] [ --no-ignore-case ] [
       --no-recursion ] [ --no-same-permissions ] [  --no-wildcards  ]  [  --no-wildcards-match-slash  ]  [  --null      ]  [
       --numeric-owner  ]  [  -o,  --old-archive, --portability, --no-same-owner ] [ -O, --to-stdout ] [ --occurrence NUM ] [
       --overwrite ] [ --overwrite-dir ] [ --owner USER ] [ -p, --same-permissions, --preserve-permissions ]  [  -P,  --abso-
       lute-names  ] [ --pax-option KEYWORD-LIST ] [ --posix ] [ --preserve ] [ -R, --block-number ] [ --record-size SIZE ] [
       --recursion ] [ --recursive-unlink ] [ --remove-files ] [ --rmt-command CMD ] [ --rsh-command  CMD  ]  [  -s,  --same-
       order, --preserve-order ] [ -S, --sparse ] [ --same-owner ] [ --show-defaults ] [ --show-omitted-dirs ] [ --strip-com-
       ponents NUMBER, --strip-path NUMBER (1) ] [ --suffix SUFFIX ] [ -T, --files-from F ] [ --totals   ]  [  -U,  --unlink-
       first ] [ --use-compress-program PROG ] [ --utc ] [ -v, --verbose ] [ -V, --label NAME ] [ --version  ] [ --volno-file
       F ] [ -w, --interactive, --confirmation ] [ -W, --verify ] [ --wildcards ] [  --wildcards-match-slash  ]  [  --exclude
       PATTERN  ]  [  -X,  --exclude-from  FILE  ]  [  -Z,  --compress,  --uncompress  ] [ -z, --gzip, --gunzip, --ungzip ] [
       -[0-7][lmh] ]

So it’s a bit overwhelming. The good news is there are two common uses for tar- creating tarballs and opening tarballs. This will be a majority of your interaction with it. You get all sorts of fun options with tar, such as using different compression libraries, but it’s still pretty straight forward.

Simple Archive

Tar produces tarballs, which in its simplest form is a bunch of data files run together into a larger file. in the following instance, -c means create, and -f means “create the following as a file called foo.tar”

tar -cf foo.tar bar/

This takes the bar directory and throws it all into a single file called foo.tar. Apart from some binary mojo to mark the separators between files, it’s almost as it all of the files were pasted end-to-end inside another file. if foo.tar is copied to another machine or place, you could untar the file with the following command:

tar -xf foo.tar

Again you see the -f flag, but the -c flag has been replaced by the extract flag, -x. This will create a directory called bar/ which will contain the contents identical to the original.

Compressing Archives

You also have the option of compressing tarballs in the process of creating them. There are three types of compression built into the version of tar I’m using: -Z, which uses the compress utility (ancient?); -z which uses gzip (old standard); and -j, which uses b2zip, which is good for compressing binaries (appears to be the new standard).

When creating a tarball that is compressed, it’s generally expected that you label it as such by appending the type to the filename, for example:

tar -cZf foo1.tar.Z bar1/
tar -czf foo2.tar.gz bar2/
tar -cjf foo3.tar.bz2 bar3/

Unless you have a specific reason, you’ll probably want to use bz2. You’ll probably never deal with a tar.Z file, but if you do, you’ll know how to deal with it. To uncompress these puppies, switch out the -c flag for the -x flag like we did in the previous example.

tar -xZf foo1.tar.Z
tar -xzf foo2.tar.gz
tar -xjf foo3.tar.bz2

One last option you may want to look at is -v. It will show you files as they’re being processed, and can be good for troubleshooting.

Unfinished Drafts: Useful Utility: cat

This article was originally written back on Feb 21st, 2006. While never completed, I thought it was worth sharing.

Cat is a very simple utility- so simple I debated added it to this list. There are however three really useful flags. I’ll try to write as much as I can about it so you don’t feel ripped off by this article. hrm… did that last sentence sound like filler? I swear it wasn’t meant to- that’s completely on accident.

So what is cat? Cat is a utility for printing the contents of a file or files to the screen. for example:

morgajel@FCOH1W-8TJRW31 ~/docs $ cat path.txt
paths


database admin

system admin

network admin management
morgajel@FCOH1W-8TJRW31 ~/docs $ 

You can also specify several files as well if you want to train them all together and pipe them to another utility.

morgajel@FCOH1W-8TJRW31 ~/docs $ cat foo.log bar.log baz.log |grep "Invalid user"> invalid_users.txt

Cat Flags

So there are three useful flags for cat. the first one is -n, which adds a linenumber to the output, like so:

morgajel@FCOH1W-8TJRW31 ~/docs $ cat path.txt -n
     1  paths
     2
     3
     4  database admin
     5
     6  system admin
     7
     8  network admin management
morgajel@FCOH1W-8TJRW31 ~/docs $

This can be useful when debugging source files. The next option is somewhat related; the -b option adds a line number, but only to non-blank lines. if you’re wanting to figure out for some reason what the 5th item is, not including blank lines, this is the way to go. Here’s an example of what it would look like:

morgajel@FCOH1W-8TJRW31 ~/docs $ cat path.txt -b
     1  paths


     2  database admin

     3  system admin

     4  network admin management
morgajel@FCOH1W-8TJRW31 ~/docs $

Notice how it only counted to 4? There were only 4 text lines. The final option that may or may not be of use is the -s flag, which smushes (that’s a technical term) blank lines together- it leaves single blank lines alone, but if there’s more than one blank line next to each other, it removes all except one. using our file above, watch what happens between “paths” and “database admin” in our example:

morgajel@FCOH1W-8TJRW31 ~/docs $ cat path.txt -s
paths

database admin

system admin

network admin management
morgajel@FCOH1W-8TJRW31 ~/docs $

Notice how there is only one blank line? That’s what -s does. if you’ve ever had a file where you’ve systematically removed text but not newlines and end up with a 500 line file with 20 lines of text, this can be useful for making it readable on a single page.

Well, that’s all I can really say about cat. If you have anything else to add, do so in the comments.

SOLVED: Halp, “screen” is broken on CentOS on my Linode instance!

So while migrating some content to a Linode instance, I attempted to fire up screen ran face first into a brick wall. For context, this was on an up-to-date CentOS 5.5 instance running screen-4.0.3-3.el5. Steps to reproduce:

  • logged in as my normal user
  • typed in “screen” and hit enter
  • console went blank for 5 seconds
  • previous content returned, along with “[screen is terminating]” and my normal command prompt.

Screen didn’t work for any regular users, however it did work for root.

I reinstalled the package, compared it to another centOS server, compared configs checked strace, etc. I could find no reason for this behavior.

Eventually I noticed I was using ttyp0 rather than pts/0 for my TTY; It was the only difference I could see between the two CentOS servers. Since I’d never seen ttyp (that I recall), it struck me as weird. I wasn’t sure if this was related to the fact that the broken instance was a xen guest or what.

It turns out, when Linode builds their CentOS image, they cut the following line out of /etc/fstab:

/dev/ptmx /dev/pts devpts defaults 0 0

That prevents pts from existing, causing you to fall back to the older ttyp, which is apparently incompatible with screen.

once I got that puppy mounted, everything was good.

Humble Bundle #2

So this year I managed to snag the Humble Bundle package;  If you’re not familiar with it, it’s a pack of Indie Games bundled together. There are five good reasons why you should consider getting it:

  • Name your own price ($5000 or $0.01 – your call)
  • You choose who the money goes to (charities and/or developers)
  • If you pay more than the average donation, you get Humble Bundle #1 from last year.
  • All games work on Windows, Mac and Linux
  • No DRM

Our Christmas funds were already drained, but Jackie and I agreed that we could spare $15.  I’m going through and trying the games now and thought I’d write up my notes on them:

Osmos

You’re a little bubble that propels itself by ejecting some of it’s mass in a given direction; the more mass you eject, the faster you go. As you float around, you see other blobs- some smaller, some larger. When one blob touches another, the smaller of the two is absorbed by the larger, bur as your mass grows, velocity slows. It becomes a balancing act of launching yourself to get blobs slightly smaller than yourself; if you eject too much, when you hit your target, you’ll be smaller and will be absorbed by your target (and thus dying).

There are two different types of play; “become the biggest” “absorb the ___”. Become the Biggest is played in a square or round sandbox, or it may be orbiting a massive blob; each has it’s own challenges.  Absorb the ___ involves trying to absorb moving targets- other AI blobs or repulsers (runs away from you). There may be more modes of gameplay, but I haven’t gotten to them yet.

I’ve only dug into it a bit, but it’s very addictive from what I’ve played. The only issue I’ve had is it occasionally freezes;  I have to alt-tab out of it and kill it when it happens.  It’s only happened twice, so it’s not that bad. I am grateful that the devs decided to package it as a .deb file so installing it is as simple as doing a sudo dpkg -i. Overall it’s an excellent game and easily worth $15 by itself. I could also see it ending up on the Wii due to it’s simplicity and controls.

Braid

Braid is one of those horrible horrible games like Tetris or Columns where the gameplay sinks into your brain and your start envisioning it everywhere you go.  The crux of the game is that your character can rewind time;  while it appears to be a side-scrolling platformer, you can’t actually die; so it’s closer to a puzzle game despite the running and jumping.

Each level you open up unlocks a new “feature” items that sparkle green ignore the reverse flow of time; even though you run backwards, the badguys keep moving, or the key stays where you left it. purple sparkles cause a shadow version of you to re-perform the actions you just performed; if you need to get in a gate but the purple lever is too far away and the gate closes before you get there, you can run to the gate, run to the lever, then pull it; rewind until you’re back to the gate and watch the shadow you run over and pull the lever. while he’s doing that, you can run through the gate.

I haven’t gotten very far, but the storyline is very interesting; we all wish we could rewind time and undo our mistakes, but eventually you will become a prisoner of other people’s expectations. Very fun, and again, easily worth $15. The controls are a little funky on my laptop, but this would be a great xbox arcade game. They just have a bin file which creates a dir on your system; it woulda been nicer if it had a .deb file so I could track it.

World of Goo

I’ve heard a lot about this game, but this is the first time I’ve seen it. It was actually part of Humble Bundle #1, but since I paid over the average ($7.53 at the time I purchased it), I got HB#1 as well. It’s a basic physics/building game; think tinkertoys made out of jello. The goal is to get these little goo balls into a tube; usually this is done by building some type of bridge or tower that the gooballs can climb. when you place a gooball near an existing structure, it forms a goo strut; if it only connects to one, it’ll flop like a joint. if you place it next to two, it’ll form a triangle which is relatively sturdy.

There are many different types of goo; some that drip like snot, some that can only be placed on a structure once, some that can’t form structures, etc. To top it off, if you build an awkward tower out of the gooball, the struts will bend and give way, causing the tower to collapse.

It’s a fun game, broken into many levels and chapters, but not really as enthralling as Braid. I could see this on the Wii for $15 or so by itself.

Revenge of the Titans

Revenge of the Titans is a simple tower defense game based around an alien invasion; the graphics are stylized  but simple, and the gameplay is nothing special. I’ve played probably close to 50 or so tower defense games;  It’s a little more fast paced than most and seems to just be one solid, continual wave.  It’s a well made and decent game, but I’m sorta burned out on the genre.

Samorost2

Samorost2 is a Myst-like puzzle game. Unlike the others so far, this one was flash based. The story revolves around a guy living on a small planet with his dog. When aliens stop to steal his pears and kidnap his dog, he follows them back to their base to rescue him. Each phase of gameplay involves a static screen with minor animations; you identify something on the screen that’s interactive, like a flower or ladder, and when you click on it, it triggers an animation. The gameplay is based on trying different combinations of events and timings. I was able to beat it in around an hour and a half, which makes it the shortest of the games so far. The artwork is great, but in the end it’s “just a flash game.”

Still worth a play for the artwork alone.

Lugaru HD

Lugaru appears to be a single-player  3rd person RPG, similar to WoW. The graphics are a bit rough, but it looks interesting nonetheless. Unfortunately, the controls aren’t great on a laptop, but if I had a mouse I’d probably be more interested in it. Walking through the tutorial, I got a feel for the combat, which seems crazy and fun. I’ll have to spend more time on it later.

So thats just a quick sample of what’s in the pack- I haven’t had time to play the other games. This pack is easily worth $15 several times over, and I sorta feel like I.  If you haven’t heard of the Humble Bundle, now is your chance to check it out. Hurry though, it’s not available after Christmas day.

Raw WinXP Virtualbox Partitions on a Thinkpad

New job, new laptop. Many utilities here are windows only, so it requires a bit of… effort… to get myself up and running efficiently. The solution to the windows problem is VirtualBox. I had set this up on my last laptop with little effort, but this time around required a bit more effort. Hopefully the instructions below will help others get up and running quickly.

Disclaimer– your laptop may catch on fire and explode (or worse) if you attempt this… or something.

We’ll be presuming that you’ve already resized your windows partition and have both a working Windows and Linux partition.

In Windows

Log into XP, grab MergeIDE.zip from Virtualbox’s site, extract and run it. It should be a quick flash and be done. (Note: I am not 100% sure this step is needed)

Create a new hardware profile and name it virtualbox. Make sure to set it as a choice during boot. Try rebooting into native windows once to ensure that it does offer you profile options.

In Linux

You’ll need the following packages installed (May differ for non-ubuntu systems):
mbr, virtualbox-ose, virtualbox-ose-qt

Create a stand-alone mbr file to use for booting (yes, you need the force flag):

install-mbr ~/.VirtualBox/WindowsXP.mbr --force

We’re presuming that your windows partition is /dev/sda1. In the below command, we are defining

  • a vmdk file (WindowsXP.vmdk)
  • which raw disk to read (/dev/sda)
  • which partition (1)
  • the new MBR file we just created

VBoxManage internalcommands createrawvmdk -filename ~/.VirtualBox/WindowsXP.vmdk -rawdisk /dev/sda -partitions 1 -mbr ~/.VirtualBox/WindowsXP.mbr -relative -register

Note that you’ll need read/write access to that drive as your user, so you may want to figure out a cleaner/securer way to implement this, rather than adding your user to the disk group (which is very dumb and insecure). I would, but it’s working and I have more important things to do at the moment.

Another issue- apparently thinkpads report the drive heads and cylinders oddly (T410 for me and T60p in article), so we have to add some vmdk settings before virtualbox creates them incorrectly. Open ~/.VirtualBox/WindowsXP.vmdk and add the following at the bottom:

ddb.geometry.biosCylinders="1024"
ddb.geometry.biosHeads="240"
ddb.geometry.biosSectors="63"

The biosHeads appears to be the magic value- it seems to work if it’s set to 240, but the default is 255 (which fails).

Once you add those, start up virtualbox and check the virtual media manager, your new vmdk should be listed there. Once it’s confirmed, create a new virtual machine. Rather than creating a disk, select your vmdk as an existing disk.

After you finish, go the the VM settings->system and make sure the motherboard tab as io-apic  enabled (I also had PAE/NX enabled under processor and VT-x enabled under Acceleration).

Start the VM

There are several errors that could pop up. I’m sure there are plenty more that I stumbled across, but these were the two big ones:

  • a disk read error occurred, press ctrl+alt+del to restart – Caused by incorrect biosHeads- check and make sure it’s set to 240 (this was the fix for me, results may vary).
  • Complaint about kvm/vmx – Virtualbox does not like kvm. Uninstall qemu-kvm.

If things go well, it should flicker mbr in the corner, then go to the hardware profile selection screen. Select the virtualbox profile, and continue, then log in.

What follows is a half-hour of installing generic drivers and dealing with hardware specific auto start apps complaining that they won’t work on this installation. Windows will warn that the new drivers are not blessed, so be forewarned.

Once completed, at the top of the VM windows select Devices-> Install Guest Additions. This will download and mount an ISO, and windows will pop open a folder with the addition executables. Select the one best for you and run the installer. It will prompt you for video and mouse drivers (and trust me, you want them).

The final step is to shut down the windows VM, then reboot into the native windows partition to make sure it still works.  I did receive a few blue-screens before logging in at the beginning, but they appeared random and haven’t happened since.

And that’s all there is to it- simple, eh? Your windows partition should now run in native mode and vm mode.

Shutup Shutup Shutup!

die you wretched pc speaker beep that is so loud when I use the find bar in firefox or tab complete in the cli and nothing is found!

  rmmod pcspkr
  echo blacklist pcspkr >> /etc/modprobe.d/blacklist

aaah, glorious silence.

Request Tracker 3.6.5 broken after updating Cent OS

Can’t locate object method “seek” via package “File::Temp” at /usr/lib/perl5/site_perl/5.8.8/MIME/Parser.pm line 816.

The underlying problem is perl was updated and overwrote the “correct” version of File::Temp that you probably installed when setting up RT and forgot about. To fix this issue


cpan install File::Temp
/etc/init.d/httpd restart

MAKE SURE TO RESTART APACHE! I didn’t, and it cost me probably 2 hours of screwing around with it.

I’m posting this because
http://www.nabble.com/RT-3.6.5-and-Sendmail-error-and-looks-like-perl-error-td15989015.html
Didn’t really mention what the final working solution was.

28

Whew, it’s been a busy year. Back in 2005 I wrote a list of things I’d like to do eventually that I’d be able to do if I had the time and resources that winning the lotto would provide. I’ve made a positive step towards realizing a couple of those.

Family

The big change this year was the addition of Ian. We didn’t get off too well in the beginning, but we’re doing pretty good now. He’s just now to the point where he’s taking his first steps. He’s a smart kid, very inquisitive, but stubborn. Within a few years he’ll be learning how to read and write- I’m guessing it won’t be long until I’m teaching him how to program. He’s got an eye for any type of electronics, but goes nutty for my blackberry or laptop. Yeah, he’s a geek in training.

Jackie’s been doing fairly well, all things considered. She’s been staying home with Ian rather than going back to work, and it’s driving her a little stir crazy. To help ease the cabin fever, she got a laptop for christmas- Ian wasn’t letting her stay in the office, so now she has a way to keep in contact with people from the living room.

Weddings

This was the year of the wedding… Brad and Erin, Jordan and Beth, Matt and Carrie, William and Charleen, Jackie’s friend Joslyn, Jackie’s sister Lori, and Paul and Kristen. I only made it to 4 of the weddings, but I was standing up in two of them. The ones we went to were all in Grand Rapids, so we had the joy of driving back and forth 155 miles each way, with the added bonus of dropping Ian off somewhere along the way

Work

Work started off pretty rough. I have a habit of reflecting the attitudes of those around me, and my supervisor at the beginning of the year was pretty negative. That coupled with Ian’s birth and all the associated expenses turned me into a pretty miserable person. Fortunately the supervisor left and was replaced with a cautious optimist. Things got better- so much so that I just passed the 1 year mark at my current place and still find it interesting and entertaining.

Technologies

I’ve played with many new technologies this year- Jboss, Netscaler and CentOS being the three foremost. I’ve dug back into ruby with a couple of new projects, as well as LDAP and a plethora of new utilities. I even picked up a bit of python. This trend doesn’t look like it’s gonna let up, so the job should be exciting and interesting for some time to come.

Projects

This year saw me move away from music and go back towards software (although I have been playing harmonica in the car at stoplights on the way to work). I joined the Luma team as head cheerleader and started the QT4 branch. After becoming frustrated with the python, qt3->qt4 transition and the codebase in general, I started an unofficial sister project, Ruma. I still sit in the Luma channel and help when I can, but I’m gonna sit back and wait until we’re ready for QT4- hopefully we’ll scrap the entire codebase and start fresh.

Right now I have two other developers “working” for me on Ruma, Lars and Hiro. They’ve put a lot of effort into bringing this bad boy to life, and I’m thankful I’ve had their help. For those of you not aware, Luma and Ruma are LDAP administration tools. As a System Administrator, I use LDAP a lot, so having specialized tools can be helpful.

Another project I started this year was competing in National Novel Writing Month. I finished the rough draft for my first book, Sinblade, a week early. It’s only 150 pages or so, but not a bad start all things considered. Jackie’s (sorta) working on revising it, and once she’s done I’ll open it up for others to start giving me feedback. I’m excited and nervous about it. If it takes off and people like it, I have 6 or so more books that I’m interested in writing.

The money situation is still pretty tight, but it’s getting better. Hopefully next year will be as much of an improvement over this year as this was over last year.

LDAP+ Sudo +TLS fix

For those of you who can’t get those three to work together, make sure you specify both TLS_CACERT tls_cacertfile- I didn’t and it caused me grief.

Epson Perfection v350 on Ubuntu Feisty Fawn

Ok, I’ve done it twice now, I think I’ve figured it out.
Get the source files from Epson’s linux website; I don’t recall the url but these are the files you need (or newer, no idea what the future holds):

iscan-2.3.0-1.c2.i386.rpm
iscan-plugin-gt-f700-2.0.0-0.c2.i386.rpm  

Use alien to convert them to debs (normally bad, but acceptable this time around).

Install both newly created deb packages with dpkg:
dpkg -i iscan_2.3.0-2_i386.deb iscan-plugin-gt-f700_2.0.0-1_i386.deb
Install sane, sane-utils:

apt-get install sane sane-utils

Edit /etc/sane.d/dll.conf, replacing epson with epkowa (yes, I’m serious).

add the following to /etc/udev/rules.d/45-libsane.rules:

# Epson Perfection v350
SYSFS{idVendor}=="04b8", SYSFS{idProduct}=="012f", MODE="664", GROUP="scanner"

And I think that’s about it. Make sure you’re in the scanner group, and run:

scanimage -L

That should simply list your scanner- if it doesn’t let me know- I might have missed something. From this point on you should be able to access the scanner with either sane or iscan.

The one downside to this driver is it doesn’t do the full 48000 PSI, only 24000- this means a scan of a guitar pick is *only* 16 inches across, rather than 32 inches 🙂 Unfortuantely it’s not the preferred resolution for negatives.

Go to Top