Development

SPoE: Slow but steady.

0

So I have a few user stories; time to start putting the infrastructure together. So What have I decided on so far?

Language: Java
Framework: Spring*
Repository: Subversion
IDE: Eclipse
Continuous Integration/ Deployer: Hudson
Build Automation: Maven2

I’m in the process of getting all my pieces together and in place. I’ve set up a subversion repository and eclipse. I have a very basic .war file setup committed and a maven script to build the war file. Every five minutes, a Hudson job polls the repository and rebuilds the warfile, then deploys it to JBoss. It’s a pretty sweet setup despite it only deploying a “Hello World”.

Now I need to refresh myself on Java and learn Spring.  This leads me to my first task:

Card SPoE-0001
User Creation

As a User, I want to be able to provide a preferred username and email address to create an account.

Status: New
Version: 0.0.1
Component: User account
Original Estimate: ?
Time spent: 0
Time Needed: ?

Going into this, I’m familiar with MVC framework in Rails, so I’m guessing the concepts aren’t much different for Spring MVC I’m still new to this process(Java webapp layout, Agile and Spring), so if you see any mistakes or have suggestions, please let me know. If you’re interested in helping me with development,

*Despite suggestions of both Struts and Spring, I went with Spring after scientific investigation.

starting opensourcecook.com development

0

So jackie wants to take her recipes and build an open cookbook sorta thing. I’m not sure how all of this is gonna play out, but I’ve set up the dev environment for it. I’ll try to make notes about it when I can.

28

2

Whew, it’s been a busy year. Back in 2005 I wrote a list of things I’d like to do eventually that I’d be able to do if I had the time and resources that winning the lotto would provide. I’ve made a positive step towards realizing a couple of those.

Family

The big change this year was the addition of Ian. We didn’t get off too well in the beginning, but we’re doing pretty good now. He’s just now to the point where he’s taking his first steps. He’s a smart kid, very inquisitive, but stubborn. Within a few years he’ll be learning how to read and write- I’m guessing it won’t be long until I’m teaching him how to program. He’s got an eye for any type of electronics, but goes nutty for my blackberry or laptop. Yeah, he’s a geek in training.

Jackie’s been doing fairly well, all things considered. She’s been staying home with Ian rather than going back to work, and it’s driving her a little stir crazy. To help ease the cabin fever, she got a laptop for christmas- Ian wasn’t letting her stay in the office, so now she has a way to keep in contact with people from the living room.

Weddings

This was the year of the wedding… Brad and Erin, Jordan and Beth, Matt and Carrie, William and Charleen, Jackie’s friend Joslyn, Jackie’s sister Lori, and Paul and Kristen. I only made it to 4 of the weddings, but I was standing up in two of them. The ones we went to were all in Grand Rapids, so we had the joy of driving back and forth 155 miles each way, with the added bonus of dropping Ian off somewhere along the way

Work

Work started off pretty rough. I have a habit of reflecting the attitudes of those around me, and my supervisor at the beginning of the year was pretty negative. That coupled with Ian’s birth and all the associated expenses turned me into a pretty miserable person. Fortunately the supervisor left and was replaced with a cautious optimist. Things got better- so much so that I just passed the 1 year mark at my current place and still find it interesting and entertaining.

Technologies

I’ve played with many new technologies this year- Jboss, Netscaler and CentOS being the three foremost. I’ve dug back into ruby with a couple of new projects, as well as LDAP and a plethora of new utilities. I even picked up a bit of python. This trend doesn’t look like it’s gonna let up, so the job should be exciting and interesting for some time to come.

Projects

This year saw me move away from music and go back towards software (although I have been playing harmonica in the car at stoplights on the way to work). I joined the Luma team as head cheerleader and started the QT4 branch. After becoming frustrated with the python, qt3->qt4 transition and the codebase in general, I started an unofficial sister project, Ruma. I still sit in the Luma channel and help when I can, but I’m gonna sit back and wait until we’re ready for QT4- hopefully we’ll scrap the entire codebase and start fresh.

Right now I have two other developers “working” for me on Ruma, Lars and Hiro. They’ve put a lot of effort into bringing this bad boy to life, and I’m thankful I’ve had their help. For those of you not aware, Luma and Ruma are LDAP administration tools. As a System Administrator, I use LDAP a lot, so having specialized tools can be helpful.

Another project I started this year was competing in National Novel Writing Month. I finished the rough draft for my first book, Sinblade, a week early. It’s only 150 pages or so, but not a bad start all things considered. Jackie’s (sorta) working on revising it, and once she’s done I’ll open it up for others to start giving me feedback. I’m excited and nervous about it. If it takes off and people like it, I have 6 or so more books that I’m interested in writing.

The money situation is still pretty tight, but it’s getting better. Hopefully next year will be as much of an improvement over this year as this was over last year.

Introduction to Subversion

3

I was planning on simply republishing my previous svn article, but realized that it sucked compared to what I know now.

Prerequisites

I’ll presume you have the following things.
– a Linux machine
– subversion already installed

Terminology to Know

There are a few terms that get mangled if you’re coming from other types of source control. This is just to clear things up.
Repository: the central storage place on the subversion server where your data is kept.
Checked-out copy: Unlike VSS, saying something is checked out does not imply that the file is locked. Also referred to as a local copy; but bear in mind that it doesn’t contain *all* of the data of the actual Repository.
commit: save the changes you made locally to the repository.

Mental Hurdles

All of the files stored in a subversion repository are stored in a meta-filesystem. Much like an ISO image sitting on your desktop is not simply a folder full of files, a subversion repository is not directly accessible when you open it. Instead, you’ll see the guts of the repository- DB files, hooks, locks, etc. Don’t go digging through there to manually change your files- it’ll break things.

Another important one is the meta-filesystem. The “inside” of your repository is just a big filesystem. Much like /bin and /home exist on a linux machine by convention, there are certain base-level directories you should create for convention’s sake. The first thing you should do with a new repository is create three new base level directories: /tags, /trunk and /branches. Your main development will take place in /trunk. Don’t worry about the other two at the moment.

When I refer to the root of the repository, I’ll often refer to it as / or root. This *IS NOT* your server’s root directory or the physical location of the repository- it’s the part of the meta-filesystem where /trunk, /tags and /branches reside.

Step 1: Creating the Repository

Creating a repository is fairly simple. Anyone can create a repository where ever they have write access. All they must do is run

$ svnadmin create ~/project1/

This should create an empty repository. This will demonstrate what I’m talking about when I say that your repository is a meta-filesystem:

morgajel@unicron ~ $ svn ls file:////home/morgajel/project1
morgajel@unicron ~ $ ls /home/morgajel/project1
conf dav db format hooks locks README.txt
morgajel@unicron ~ $ svn mkdir file:////home/morgajel/project1/trunk -m 'creating trunk'

Committed revision 1.
morgajel@unicron ~ $ svn ls file:////home/morgajel/project1
trunk/
morgajel@unicron ~ $ ls /home/morgajel/project1
conf dav db format hooks locks README.txt

Here you can see the repository was empty, then we created a directory called trunk (using a commit message to describe the change we made), then showing that the directory was in fact created. Do the same thing for /tags and /branches.

We now have a working repository!

Checking out the Repository

Creating a repository is fine, but using it would be much more… useful. Next you should check out a copy of your project. Under normal conditions you’ll only be checking out the /trunk, but you mileage may vary in different situations. Since this is on our local machine, we can use the file:// protocol. Other protocols exist, like http:// (webdav), svn:// (svnserve), and svn+ssh:// (svn over ssh), but you don’t have to worry about them right now.


morgajel@unicron ~ $ svn co file:////home/morgajel/project1/trunk my_project1
Checked out revision 3.

This should check out the empty /trunk directory to a local folder called my_project1. The only thing in this directory is a hidden .svn directory which holds the guts of your local copy repository info. It’s similar in function to the CVS directory in CVS. Unfortunately a nearly-empty directory isn’t much use, so let’s add some content.

Adding Content

So let’s add some content.

morgajel@unicron ~ $ cd my_project1/
morgajel@unicron ~/my_project1 $ mkdir -p lib bin share/docs
morgajel@unicron ~/my_project1 $ touch configure Makefile share/docs/README lib/foo.pm bin/widget.pl
morgajel@unicron ~/my_project1 $ svn status
? configure
? share
? lib
? bin
? Makefile

Here I created a bunch of directories and created some empty files. When I ran svn status, svn told me that there were 5 things it wasn’t versioning. Let’s add them.

morgajel@unicron ~/my_project1 $ svn add configure share lib bin Makefile
A configure
A share
A share/docs
A share/docs/README
A lib
A lib/foo.pm
A bin
A bin/widget.pl
A Makefile
morgajel@unicron ~/my_project1 $ svn status
A configure
A share
A share/docs
A share/docs/README
A lib
A lib/foo.pm
A bin
A bin/widget.pl
A Makefile

As you can see, it recursed down into share, bin and lib and added all the goodies inside of each directory. You can also see svn status shows these as well. Keep in mind they’re just slated to be added to the repository- they’re not added yet. Let’s go ahead and commit them.


morgajel@unicron ~/my_project1 $ svn commit -m "a bunch of empty files and directories"
Adding Makefile
Adding bin
Adding bin/widget.pl
Adding configure
Adding lib
Adding lib/foo.pm
Adding share
Adding share/docs
Adding share/docs/README
Transmitting file data .....
Committed revision 4.

Modifying Data

So suppose you’d like to modify these files, you you decide to move the README to the root of your local copy (~/my_project1/):


morgajel@unicron ~/my_project1 $ svn mv share/docs/README README
A README
D share/docs/README
morgajel@unicron ~/my_project1 $ svn stat
D share/docs/README
M bin/widget.pl
A + README

Notice that I used svn mv to move files rather than regular old mv- That’s to make sure svn is aware of the move and keeps the file history associated with the new file. You can also see bin/widget.pl now include some new info as well, and displays an M[odified] next to it. The + next to README shows that it copied the history over from it’s previous position. So what happens if we move a file without svn mv?

morgajel@unicron ~/my_project1 $ mv configure config
morgajel@unicron ~/my_project1 $ svn status
? config
! configure
D share/docs/README
M bin/widget.pl
A + README
morgajel@unicron ~/my_project1 $ mv config configure

You can see that svn panics(!) that configure has gone missing, and sees this new file called config that it’s currently not revisioning. It doesn’t know that they’re the same file.

Commit Messages

You’ve seen me use the -m flag a couple of times now- I’m using it to keep things flowing. If you don’t use it, you’re prompted in your favorite $EDITOR to create a commit statement, which includes the list of modified files. Using the -m flag is useful if you’re scripting commits (I use this when dumping and committing a nightly config file from our load-balancer).

Most of the time however, you’ll use your Editor. Make sure to keep your commit messages sweet and to the point- other’s will see them.


morgajel@unicron ~/my_project1 $ svn commit
[vim shows up, I enter the following text]
Small changes to demonstrate movements
- moved the README
- added shebang to widget.pl
[save and exit vim]
"svn-commit.tmp" 8L, 190C written
Adding README
Sending bin/widget.pl
Deleting share/docs/README
Transmitting file data .
Committed revision 5.
morgajel@unicron ~/my_project1 $ svn update
At revision 5.
morgajel@unicron ~/my_project1 $ svn log
------------------------------------------------------------------------
r5 | morgajel | 2007-11-20 15:55:21 -0500 (Tue, 20 Nov 2007) | 4 lines

Small changes to demonstrate movements
- moved the README
- added shebang to widget.pl

------------------------------------------------------------------------
r4 | morgajel | 2007-11-20 14:17:04 -0500 (Tue, 20 Nov 2007) | 1 line

a bunch of empty files and directories
------------------------------------------------------------------------
r1 | morgajel | 2007-11-20 13:35:28 -0500 (Tue, 20 Nov 2007) | 1 line

creating trunk
------------------------------------------------------------------------

You’ll notice that revisions 2 and 3 aren’t listed- if you’ll remember correctly, they were used to commit the /tags and /branches directories. svn log only reports changes that affect the current target (in this case, ~/my_project1 which is a local copy of /trunk).

There’s a couple more tips and tricks I could go on about- if there’s any interest in this post maybe I’ll write some more about more advanced topics.

Dissappointing state of LDAP support in Ruby

8

So I’ve been working on a new project called Ruma(more on it later), and I’m beginning to get frustrated with the LDAP support. So far I’ve found
* ruby/ldap, ruby-ldap, ldap-ruby by ian macdonald, last release 8/2006
* net-ldap, Net::LDAP by Francis Cianfrocca, last release 8/2006
* ruby-activeldap, ActiveLDAP-ruby by Will Drewry, last release 5/2007

ldap-ruby appears to be the frontrunner, but hasn’t been updated in a year- the forums appear to think it’s been abandoned. Net::LDAP appears to be a work in progress that isn’t quite complete. ActiveLDAP-ruby appears to be mainly for rails, so I don’t know how useful it is.

When you search for ruby ldap, you can any of those name combinations, and it’s impossible to figure out which is alive, which is stable, and which is useful. At this point I don’t think any of the projects meets all three criteria at the moment.

Supporting Standards vs. Supporting all Testcases

3

So I’ve been working on a side app that connects to remote servers. While writing validation code for inputs, I came across an interesting dilemma. I’d like to validate an address input to be valid- be it a hostname, domainname, or ip address. Unfortunately it’s not as simple as it sounds.

According to the domain name RFC, section 3.5 domain names must start with a letter, end with a letter or number, and can have letters, numbers or hyphens in between. When you add in periods to separate it out it’s messy, but workable.

But not everyone follows the RFC- take fifth/third bank for example- I’m not sure how they got away with it, but it doesn’t really follow this (historically I think 3com is the one to blame).

So that leaves me with two choices: follow the standard, or allow for the outliers. It’s doubtful anyone will be using my app to manage 53.com’s ldap server, and less likely that 53.com IS an ldap server. It is however possible that some one out there has either named or inherited a server named 3app6ldp02.foo.internal.

So do I support the guy? Should I care? should I even bother validating the address?

What I Dislike About Python

1

Since I began working on the Luma project, I’ve been playing a lot with Python, a language that I’ve been around for years but never bothered to learn. Since Luma is written in Python and I’m not on the team, I figured it was time to jump in feet first. Coming from a perl/php/ruby/java background, it wasn’t a big leap to make.

However, the more I read and write, the less I like it. Some are minor complaints, while some are a bit larger. I’m gonna list them here as I think of them, from trivial to more important

* indentation – This is really an annoyance more than anything. If you’re not aware, python is whitespace sensitive- rather than using curly braces, it uses indentation to show scope and what belongs where. Oh, and you better make sure to replace your tabs with spaces, otherwise you’ll ned up with bizarre errors that are a pain in the ass to track down.

* debugging tools – This one is a bit more aggravating- I may not have found the equivalents to Data::Dumper or -w in perl, or .inspect in ruby, but that doesn’t mean they don’t exist. I’m sure I’ll find them in time. And for those of you who suggest using the python interpreter for debugging, that doesn’t really work that well when examining a GUI app.
* self – it seems really redundant that inside a class method, the first parameter you pass is self- in other languages I’ve used, self is so prevalent inside a class that it’s implicitly expected, not explicitly stated.
* non-object old-style objects – When is an object not an object? When it’s an old style python object. Apparently the Python object setup is sorta broken- new-style objects are subclassed from an actual object class, while old-style aren’t. People complain that Perl’s OO was tacked on, but this seems far more kludgy.
* super – related to the above point- new-style objects can use super, which requires being told which parent class it should be looking at (since python supports multiple inheritance). The problem is old-style classes do not, so if you’re subclassing a module you didn’t write which itself is subclassed, you may have to dig to figure out if you can use super or not. Old-style classes use a different method- you call ParentClass.method() and pass it self as the first parameter. I think this probably ties into the kludgy way self is handled. For more on super, check out Python’s Super being Harmful (which goes way over my head).

* a line fails but it doesn’t say why, it just stops– I’ve run into this problem while trying to debug code- take the following code for example:

print "in workerthreadadd- will it add? 1"
searchResult = self.ldapServerObject.add_s(self.dn, self.modlist)
print "in workerthreadadd- will it add? 2"

The first line will print- the second one won’t. I don’t know what’s going on in that ldapServerObject- I didn’t write it, but there’s not stackdump or errors spewing out- it just stops. Now I have to go through and debug the object I’m calling to figure out why it’s dying. (This isn’t the actual code that gave me this problem. I can’t remember what was doing it previously.)

That said I’m still gonna keep at it- I didn’t learn perl overnight, nor ruby. I suspect python will take a while to grow on me. When I learned php it was great compared to java because it didn’t need to be compiled; when I learned perl it handled text and gui interfaces better than php; when I learned ruby it’s OO abilities were far greater than perl’s. I just have to figure out what it’s good at that ruby and perl aren’t.

update:
It looks like the old vs. new style objects will be gone in python 3.0- they’re planning on switching to new-style only. I think it’ll be a positive step overall, but after talking to the guys in #python, python 3.0 is much like Perl 6- it should be here sometime within the next decade, but I probably shouldn’t even narrow it down that much 🙂

BusyBusy

0

So I just realized that I’ve become uber busy as of late. My current side projects look like this:
– Warhammer Army builder for ChrisP written in Ruby (on Rails)
– reviving radio.morgajel.com and rebuilding my stream (RoR, again)
– SecretWord module and Logging for Ziggy (Perl + Po::Co::IRC)
– ImportLDIF plugin for Luma

This isn’t including the miniature painting, work, or Ian. The weird part is it feels nice to be busy- but only when it’s busy on my terms 🙂

Giving Back

0

I’ve been using and promoting open source for about 7 years now- it started back in 2000 when I began writing php and playing with apache. In 2003 I made the full switch to Linux. Since then I’ve produced several small little projects and put them under the GPL in hopes that it would help someone else. I’ve also written many articles and how-tos for my site to help spread what I’ve learned.

This past week has been a major milestone for me as I’ve joined the Luma team. For those of you not familiar, Luma is an LDAP browser and administration tool written in Python and QT. My capacity is limited at the moment, since I don’t really know python and am only remotely familiar with QT development- I have, however, been using luma for 2 years now for various ldap based projects.

I’ve already cleaned up and helped organize the bugtracker, and am currently working on an LDIF Importer plugin to feel my way around Luma’s API and python in general. I finally feel like I am giving back to the community.

If you have access to an LDAP server (be it AD, OpenLDAP, Domino, Novell’s, etc), give Luma a try and give us some feedback. We’re looking for beta-testers and python developers- Heck, if you wanted to just comment the code, that’d be great as well. Feel free to stop by #luma on freenode- we’re always minimally paying attention and can probably answer any questions you may have.

27

4

Doing another annual recap this year like I did last year.

2006 was a pretty rough year- move from DC Metro back to Grand Rapids, got a crappy job, found out jackie was pregnant, got laid off from crappy job, spent 3 months unemployed, and at the end of November was offered an awesome new job in Troy, MI (the other side of the state). Other than Ian, and the new job, the best good news is we finally have a decent couch- two of them, actually.

Employment

I enjoyed working at CSX a lot, but Virginia started getting to me- a guy threw himself in front of the orange line and it was on the news down there, and I realized I could sorta understand why he did it… that was my hint that I needed to get out of there. The result was taking the first job that could get me back to Michigan.

I ended up taking a contract-to-hire job back in Grand Rapids that paid poorly, treated me like crap, and generally made me come home and curl up in the fetal position. Part of the agreement to even get close to the payrange I was looking for was I had to work 55 hours a week. The problem was I spent all 55 hours each week for 3 months putting out fires and stressed myself so badly that the doctor told me I had to cut back. Five months(August) into the 3 month contract (oh, it was open ended, didn’t you know that?) I was laid off. No severance, just a boot out the door. I wasn’t the only one laid off since the company is apparently not doing as well as they thought they were, but there’s still a lot of bitterness there.

The recruiter flat out lied when he said the Grand Rapids job market had improved- the unemployment rate is still 7% and companies are closing down left and right. I spent 3 months searching for a job in the area when our friend Chris P. offered to hand my resume off to a friend of his who was looking for a linux admin. Three interviews later, they offered me the job for a decent amount of money and benefits. The down side was it’s in Troy, so we’re moving again. This place seems great tho, and I don’t have any of the doubts or negative feelings I had initially about the job in GR.

Family

In July, Jackie and I found out we were going to have a child. It’s a boy, and he’s due on March 10th. We’ve decided on Ian Hawthorn for a name. We’re really excited about it, but to be honest it didn’t help the stress level when I was laid off a month after finding out. We’re looking forward to it, but the new job is complicating issues like lamaz classes and doctor changes.

My brother Jamie is off being a park ranger/professional bum, Brian is still in school, parents are doing good, inlaws are good, grandparents are all doing well and the cats still haven’t been set on fire- can’t ask for more than that.

Hobbies

I finally finished my chainmail shirt. The age difference between rings resulted in some bad discoloration which I tried to fix by treating with vinegar- Unfortunately that completely removed the zinc coating on some resulting in rusting- once I get the cash I plan on rolling it in a bucket of sand to remove the rust, oiling it, and then putting it in a tupperware container.

I played with my guitar quite a bit early in the year, but slacked off later on. I also realized that learning to play the keyboard is going to require more than 25 keys, so I’ve put it off until I have the time and money to get a real keyboard and take lessons. I did manage to pick up a trumpet and restring my violin- I can play a scale on the violin (barely) and can annoy the neighbors with the trumpet. I also bought some new reeds for Jackie’s clarinet and learned how to play a scale.

I got back into ruby and finally checked out this whole rails thing- it’s very cool. I wish I would have kept up with ruby way back when and ignored the nay-sayers.

I’ve also taken up walking- yes, walking. I’ve been walking about 3 miles every other day since we moved back to Grand Rapids- once around the block. Now that we’ve moved, that’ll probably go by the wayside due to the lack of walking partners and Jackie’s general roundness (from Ian).

Hopefully 2007 will see us getting back to where we were last year this time.

Go to Top