Get the Drivers

Since my last installation, things have changed. a bit. Many different distros are now supported, so we don’t have to shoehorn the RPMs onto the system with Alien. Here are the packages I grabbed off their site:

iscan-data_1.0.1-1_all.deb
iscan_2.25.0-1.ltdl7_i386.deb
iscan-plugin-gt-f700_2.1.0-3_i386.deb


Once you have them, you can install them along with sane and sane utils:

sudo dpkg -i iscan_2.25.0-1.ltdl7_i386.deb iscan-data_1.0.1-1_all.deb iscan-plugin-gt-f700_2.1.0-3_i386.deb
sudo apt-get install sane sane-utils


Edit the Configs

Edit /etc/sane.d/dll.conf, adding epkowa to the list.

Create /etc/udev/rules.d/45-libsane.rules, adding the following:

# Epson Perfection v350
SYSFS{idVendor}=="04b8", SYSFS{idProduct}=="012f", MODE="664", GROUP="scanner"

I use the group “scanner” above; You could go the easy route and use a group that you’re already part of, such as “users” or “admin”, otherwise you’ll have to create the scanner group, add yourself to it, then log out and log back in.

Finishing Up

Afterwards, connect your scanner and run the following command:

scanimage -L


The scanner should fire up now, which means you can use your newly installed iscan app to scan pictures.

Your browser is not supported.

We do not support your browser yet…

If you are on a Windows machine please use:
* Internet Explorer 6 and up
* Firefox 1.5 and up
* Safari Beta 3 and up

If you are on a Mac please use:
* Safari
* Firefox 1.5 and up

Your current browser is Mozilla 1.9.2.5

There are several problems with this:

1. I am using firefox 1.5 and up, specifically, "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100501 Ubuntu/9.10 (karmic) Namoroka/3.6.5pre" I can only presume that since it doesn’t say “firefox”, it fails.
2. You’re parsing my connection string wrong-1.9.2.5 is related to my profile version, not browser version. Don’t do that.
3. Even after I change my connection string to Firefox/1.5, it STILL fails. The error message says Windows or Mac; maybe it’s failing because it is looking for one or the other.
4. The site is completely useless if that page comes up- no contact info to report a problem, nothing. I can’t even tell you that you’re accidentally blocking me.

I end up having to use the following connection string:
"Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8"
and the sad part was YOUR SITE LOOKED FINE. This kind of ungraceful degradation irritates the hell out of many people. If you don’t support my browser, I get it- you don’t want to give people a bad experience. You know what you do rather than make the site completely useless? You pop up a little window saying:

Sorry, we do not support your browser yet…

For better browsing experience, we suggest you try the following:
If you are on a Windows machine please use:
* Internet Explorer 6 and up
* Firefox 1.5 and up
* Safari Beta 3 and up

If you are on a Mac please use:
* Safari
* Firefox 1.5 and up

Your current browser is Mozilla 1.9.2.5
[x] continue anyways…

Note that the little [x] is a close button- VERY important on a warning like that. This allows the users to continue on viewing your website on the off chance that you accidentally mislabeled them.

This is 2010. This is the future. An issue like this was annoying back in 1998- there’s no reason a pretty site like yours should be doing this. Get with the program.

Sincerely,
- Jesse

1. Port Expansion: I mentioned there were 7 ports we use; originally there were only 6, but I enabled SNMP to better trend JVM usage (24 JVMs use a lot of RAM). As we add more features, more ports could be needed.
2. Application Expansion: We originally started out with 9 instances, and after 5 months added another three. More will be possible in the future.

The required flexibility makes numbering conventions difficult. The solution we came up with is to key the information and use it to build the port number. In order to do that, we had to make a few assumptions:

1. Environments are sequestered to their own hardware, however they’re on the same networks- i.e. dev and qa instances will never be on the same physical server, but will share multicast ranges
2. There will not be more than 5 instances in a cluster- any more than that would put theoretical ports out the 65536 range; besides, replication shouldn’t be past 4 nodes, right?
3. Ports below 1024 are off limits
4. We are constrained by IP addresses; one IP per server.

From here we could allow the following constraints:

1. Environments can use the same ports, EXCEPT for multicast- we don’t want QA and prod replicating data!
2. All multicast should use 1 for their Instance ID

With that in mind, port numbers will be laid out with the following structure: XYYZZ

X = Instance ID (e.g. 1-4)
YY = application ID (Helium,Hydrogen,Lithium, etc )
ZZ = Port Type (jmx, ajp, snmp, etc)


1 Instance1
2 Instance2
3 Instance3
4 Instance4



01 Hydrogen
02 Helium
03 Lithium
04 Beryllium
05 Boron
06 Carbon
07 Nitrogen
08 Oxygen
09 Fluorine
10 Neon
11 Sodium
12 Magnesium



01 Shutdown
02 JMX
03 HTTP
04 AJP
05 SNMP
06 REPL
07 MCASTDEV
08 MCASTQA
09 MCASTUAT
10 MCASTPREPROD
11 MCASTPROD


This allows us to define the following:

• Hydrogen AJP for instance 1 is 10104 (1+01+04)
• Oxygen SNMP for instance 4 is 40805 (4+08+05)
• Sodium Multicast for Prod 3 is 11111 (1+11+11) (remember, all multicast use 1 for their instance)

Other than the Multicast port kludge, this way seems to work pretty well. Numbers are easy to parse visually and can be maintained with a centralized key on the wiki. Anyone have any suggestions or examples of how they’ve solved this problem?

Let me know if you find it useful.

 
#!/usr/bin/perl
# read your ldap repo and generation nagios config files for contacts
# and contact_groups
use strict;
use Net::LDAP;
use Data::Dumper;
 
# Set all your custom configs here:
my $ldaphostname='ldap.example.int';
my $groupdn='ou=People,ou=Groups,dc=example,dc=int';
my $peopledn='ou=People,ou=active,ou=Users,dc=example,dc=int';
my $contactdir='/etc/nagios/contacts';
 
my $ldap = new Net::LDAP ($ldaphostname);
$ldap->bind() || die "failed to bind";
 
#search for groups
my $mesg = $ldap->search(
        base => $groupdn,
        filter => "(objectclass=posixgroup)",
        attrs => ['cn','memberuid'],
        );
 
#loop through each entry
while (my $group= $mesg->pop_entry() ){
 
    #set the name of the group
    my $cn=$group->get_value('cn');
 
    #create a comma-separated list of group members
    my $members=$group->get_value('memberuid', asref => 1);
    $members= join( ',', @$members);
 
    # create our configuration block
    my $contactinfo="
        # We only have one contact in this simple configuration file,
        # so there is no need to create more than one contact group.
 
        define contactgroup{
            contactgroup_name       $cn
            alias                   $cn
            members                 $members
            }\n";
 
    # write this block to a file named after the group cn
    print `echo "$contactinfo" > $contactdir/group_$cn.cfg`;
}
 
#search for users
my $mesg = $ldap->search(
            base => $peopledn,
            filter => "(objectclass=person)",
            attrs => ['uid','cn','mail','description'],
           );
 
#loop through each entry
while (my $user= $mesg->pop_entry() ){
 
    my ($pager,$addr);
    # grab the four fields we're interested in
    my $uid=$user->get_value('uid');
    my $cn=$user->get_value('cn');
    my $mail=$user->get_value('mail');
    my $desc=$user->get_value('description');
 
# Note that I store the user's SMS address and Instant Messenger
# addresses in the description field in the following format:
# SMS:1234445555@some.url;
# IM:morgajel@yahoo,morgajel@gtalk;
# for those interested, you can send email to an sms address 
# with help from <a href="http://sms411.net/how-to-send-email-to-a-phone">http://sms411.net/how-to-send-email-to-a-phone</a>
    #strip pager info out of desc and format it
    #(note it's in sms format):
    if ($desc =~/SMS:([^;]+);?/){
        $pager= "        pager           $1\n";
    }
    # strip addresses out of desc and format them:
    if ($desc =~/IM:([^;]+);?/){
        my @multiple_addr=split(/,/,$1);
        my $count=1;
        # break each address out into an addressX line
        # (for informational purposes);
        foreach my $addr (@multiple_addr){
            $addr=$addr."        address$count        $addr\n";
            $count=$count+1 ;
        }
    }
 
    #generate the first half of the contact info
    my $contactinfo="
    define contact{
        contact_name    $uid              ; Short name of user
        use             generic-contact
        alias           $cn               ; Full name of user
        email           $mail
$pager
$addr
    }";
 
    # write this block to a file named after the user uid
    `echo "$contactinfo" > $contactdir/$uid.cfg`;
}

(11:33:42 AM) Jeff: so many gone
(11:34:10 AM) Jeff: jesse
matt
carrie
lisa
stacey
kurt
roger
jim
naveen
brandon
lee
bink
tristan
jason
jen
sean
drew
radha
VJ
jackie
andy
noelle
whip
reedy
jeremy
keith
ryan
pankaj
nirupa
asad
santhi
rick
limer
meads
mick
(11:34:42 AM) Jesse: you should post that as a facebook status update. one comma separated list.
(11:39:06 AM) Jeff: lol
(11:39:12 AM) Jeff: those are just people i knew and worked with in IT
(11:39:18 AM) Jeff: i’m not counting others I didn’t work with or know in IT

SUBJECT: [#24413540] Suspension Notice CPU Limits Reached (1st Warning)
Hello,

Due to the amount of CPU and/or memory resources used by your account, we were forced into suspending the account. Please understand that we only suspend an account as a last resort and want to help you track down the cause as quickly as possible.

Shared servers (MegaPhase, Prophase, and ANHosting accounts) are only allotted 10% of the server’s resources (CPU and Memory) at any given time. This
restriction is put in place to keep balance on the server. The first step is to
try to isolate what content on your website is causing the overage. If this
seems unexpected, it could be a recent change to your site (such as installing
a new wordpress plugin, mailing list application, etc). Any information you can
provide about recent changes can help us to isolate the problem. However, we
were able to retrieve the following files and/or scripts that likely caused the
suspension:

————————

literaryescapism.com 2.33 1.38 0.1
Top Process %CPU 34.0 [php]
Top Process %CPU 25.0 public_html/index.php
Top Process %CPU 23.0 public_html/index.php

————————

This was the beginning of an email I received from our hosting company, telling me that Jackie’s site was suspended due to usage. I’m not disappointed so much that they suspended it (which was the proper course of action), but in how they responded to me. Here’s my response to the warning email:

I’m trying to figure out if this is new behavior from a wordpress exploit or if it’s honestly just grown to this point; do you have any trending data for my cpu usage?

Side note- the access-logs linked in my home directory appear to go nowhere:
access-logs -> /usr/local/apache/domlogs/user
that path doesn’t exist.

I would like to help in any way I can. As a side note, I’ve noticed quite a few “errors connecting to mysql” in the apache logs; could there be an underlying issue with mysql that’s causing processes to back up and chew cpu time?

I’m also seeing some defunct php processes from other users, leading me to wonder if this is more of a systemic problem than one with just my account.

(sorry, I’m an apache/linux admin myself- let me know if there’s anything I can do to assist. I can be reached on gtalk, aim or yahoo.)

And their response:

Hello,

Unfortunately we do not have any further data except for what was given in the e-mail:

literaryescapism.com 2.33 1.38 0.1
Top Process %CPU 34.0 [php]
Top Process %CPU 25.0 public_html/index.php
Top Process %CPU 23.0 public_html/index.php

Your account has been unsuspended so you can troubleshoot this issue. Please let us know if you have any further questions.
–
Best regards,

xxxxxxxxxxxxxx
Technical Support Representative
Hosting Services, Inc.

It’s sort of dumbfounding to be completely shut down like that from a TSR. I would have hoped that he would have seen that I A) know what I’m talking about and B) would have booted me up to someone who could better assist me. Instead he unlocks the account (which, btw, was never locked or suspended as far as I could tell from the time I got the email and checked until he sent the notice that it was unsuspended) and says “sorry, we have no data to help you figure out what happened.” Note that he didn’t address my questions regarding the broken symlink, mysql availability, or defunct processes.

Flustered, I responded with this:

Is it still using such high CPU load? I noticed several processes on the server that were having issues when I looked; many of which were not mine- The load on the server was 114 when I checked it last night. Regardless of server, that’s high, and I wasn’t one of the high processes listed.

I suspect this was a systemic issue- perhaps mysql went down causing multiple sites to freak out. If that’s the case (and there was a problem with the system itself), troubleshooting from my end is near impossible, especially since you have no trending data on usage. To determine if this was a one-time issue or a building issue.

I’d like to help you guys, but ya gotta work with me here. This is what I do for a living- I’m the Apache/Linux administrator. My wife’s site has been running for 6 months with no major changes (other than simple wordpress upgrades, none of which
were in the last week) and no notable increase in traffic or bandwidth usage over the past week.

If you’re incapable of providing details as to whether or not there was a system issue (or even if other clients had similar issues at the same time) my only course of action would be to take our sites to another provider. I don’t want to do that; I’ve been happy with you guys so far.

Please, check and see if there were multiple clients on this server who received warnings as well; I suspect I wasn’t the only one. From where I’m standing the server does not look very healthy- I’m seeing multiple defunct processes:

nobody 6857 0.0 0.0 0 0 ? Z 08:46 0:00 [httpd]
1027 10565 15.0 0.0 0 0 ? Z 08:48 0:00 [php]
32003 10573 0.0 0.0 0 0 ? Zs 08:48 0:00 [cpsrvd-ssl]

Please let me know if there is a sysadmin I can work with to resolve this issue. Like I said, I don’t think this is a problem with my site necessarily.

We’ll see how they respond. Hopefully they’ll be useful- I just switched to them in October, so I really don’t want to have to find another host.

Obvious questions aside (why bother setting up two if they’re going to be hosting the same content on the same box?), lets look at what happens when you misconfigure this relatively simple idea. The tomcat instances each had a webapps directory where their application was deployed, for example,  A1/webapps/Apple/ or B2/webapps/Breakfast/. The goal was to create a common deployment location for A1 and A2, B1 and B2, however a shortcut was taken and the webapps directory was symlinked rather than Apple and Breakfast, then BOTH Apple and Breakfast were placed in the shared directory. This resulted in 2 things:
* a unified place to deploy code
* every single tomcat instance loading BOTH Apple and Breakfast

This means B1/webapps/ now includes Apple/. While this may not seem like a problem (B2′s context root is set to Breakfast/, for example), it can lead to complications down the road:
* each instance now needs to load and compile things for both applications, resulting in wasted resources and startup time.
* If Breakfast/ happens to include it’s OWN Apple/ (because apples are part of a complete breakfast,) B2/webapps/Breakfast/Apple/ is now ignored in favor of B2/webapps/Apple/.

Fortunately I’d stumbled across this misconfiguration the day before others did, so I was able to quickly deduce what was wrong. Why didn’t I fix it? Because I wasn’t sure I was right, and I didn’t want to break someone elses apparently broken configuration any further. Besides, I’m trying to fight this obsessive feeling that “if I don’t do it myself, I won’t know if it’s right,” and doublechecking everyone’s work feeds that.

Fortunately the fix was quick and simple. It may not seem like that big of a deal, but you must remember this is a simplified view of the situation. the REAL setup had 10 applications, with some weighing in at 4-8 gigs each. While most of that is content (images and flash that tomcat doesn’t have to process), sorting through an extra 600k files *per project* will slow the system down.

Now, this behavior may seem dumb until you think of it from a tomcat point of view. “I know you have this /register/ under your context root, but you also have this application named register/, so I’m gonna go ahead and presume you want to run the full application rather thank this dinky directory. Besides, it’s probably a ‘sorry, registration is currently out of order’ in case the full application is taken offline.”

1. Everyone can read and write all projects under / (/Project1, /Project2) EXCEPT:
   1. Only a certain team of devs (and I) can write to /Project1/trunk.
   2. Only a certain team of qa can write to /Project1/branches/release.

Specifically, I should be able to commit to trunk but not release, however that doesn’t seem to be the case. Here’s an abbreviated version of my vhost:

<VirtualHost 10.0.0.5:443>
 blah blah blah snip...
<Location />
     DAV svn
     SVNParentPath /var/svn/
     SVNPathAuthz off
     AuthName  "SVN Access"
     AuthType  Basic
     AuthLDAPUrl     "ldap://ldap.example.int:389/ou=Users,dc=example,dc=int?uid"
     AuthBasicProvider ldap
     AuthzLDAPAuthoritative off
     AuthLDAPGroupAttribute   "memberUid"
     <LimitExcept none>
         Require valid-user
     </LimitExcept>
 </Location>

 <Location /Project1/trunk>
     # Everyone can read, but only devs (and I) can change.
     <LimitExcept REPORT GET OPTIONS PROPFIND>
         Require ldap-group cn=devs,ou=Groups,dc=example,dc=int
         Require ldap-user morgajel
         Satisfy any
    </LimitExcept>

 </Location>

 <Location /Project1/branches/release>
    # Everyone can read, but only QA can change.
    <LimitExcept REPORT GET OPTIONS PROPFIND>
        Require ldap-group cn=qa,ou=Groups,dc=example,dc=int
        Satisfy any
    </LimitExcept>
 </Location>

</VirtualHost>

Any thoughts as to why I’m still able to write to release? and no, I’m not in the QA group; I suspect it has something to do with the Locations essentially being nested. Since we’re managing users and groups with LDAP, simple SVN ACLs won’t work, and I’m not really sure how to accomplish what I need to do.

Thoughts?

Side note: REPORT GET OPTIONS PROPFIND are the only methods needed for read-only svn webdav access. Fun fact, huh?

UPDATE:

I was overthinking the situation- Apache config is not programming. There is no inheritance between locations. There is no nesting. Once you create a new location, you need to set up perms for that, so setting the base / with read/write for everyone, I then define sublocations

 <Location /Project1/trunk>
    # If you want to write to trunk, you need to be one of the required people. You can still read it.
    <LimitExcept PROPFIND OPTIONS GET REPORT>
         Require ldap-group cn=devs,ou=People,ou=Groups,dc=mrm,dc=int
    </LimitExcept>
    <Limit PROPFIND OPTIONS GET REPORT>
         Require valid-user
    </Limit>
 </Location>

 <Location /Project1/branches/release>
    # Everyone can read, but only QA can change.
    <LimitExcept REPORT GET OPTIONS PROPFIND>
        Require ldap-group cn=qa,ou=Groups,dc=example,dc=int
        Satisfy any
    </LimitExcept>
    <Limit PROPFIND OPTIONS GET REPORT>
         Require valid-user
    </Limit>
 </Location>

What I was missing was the second half of the limits, thinking it would inherit from /. It doesn’t.  Without that require valid-user, it was allowing unauthenticated users to read the files (which was no good). Life is good (until I find out where this is broken).

UPDATE 2:

So the above didn’t work when I retested it, so I tried one last time with good results (note the goals have changed, but the principles still apply):

<VirtualHost 10.0.0.5:443>
    SSLEngine on
    SSLCertificateFile      /etc/pki/certs/example.int.crt
    SSLCertificateKeyFile   /etc/pki/certs/example.int.key
    DocumentRoot      /var/www/svn.example.int/docs
    ServerName        svn.example.int
    ServerAlias       svn.exampleco.com
    ErrorLog logs/svn.example.int-error_log
    CustomLog logs/svn.example.int-access_log common

    <Location />
        DAV svn
        SVNParentPath /var/svn/
        SVNPathAuthz off
        AuthName  "SVN Access"
        AuthType  Basic
        AuthLDAPUrl             "ldap://ldap.example.int:389    /ou=Users,dc=example,dc=int?uid"
        AuthBasicProvider ldap
        AuthzLDAPAuthoritative on
        AuthLDAPGroupAttribute   "memberUid"
        AuthLDAPGroupAttributeIsDN off
        order Deny,Allow
        Deny from all
        Satisfy any

        # access
        <LimitExcept NONE>
            Require valid-user
            Satisfy any
        </LimitExcept>
    </Location>

    <Location /ProjectA/branches/releases//>
        order Deny,Allow
        Deny from all
        Satisfy any

        #read-only access    
        <Limit GET PROPFIND OPTIONS REPORT>
            Require valid-user
            Satisfy any
        </Limit>
        # write access
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
            Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
           #            Require ldap-user jmorgan
            Satisfy any
        </LimitExcept>
    </Location>

    <Location /ProjectB/branches/releases//>
        order Deny,Allow
        Deny from all
        Satisfy any

        #read-only access    
        <Limit GET PROPFIND OPTIONS REPORT>
            Require valid-user
            Satisfy any
        </Limit>
        # write access
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
            Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
           #            Require ldap-user jmorgan
            Satisfy any
        </LimitExcept>
    </Location>

    <Location /ProjectC/branches/releases//>
        order Deny,Allow
        Deny from all
        Satisfy any

        #read-only access
        <Limit GET PROPFIND OPTIONS REPORT>
            Require valid-user
            Satisfy any
        </Limit>
        # write access
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
            Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
           #            Require ldap-user jmorgan
            Satisfy any
        </LimitExcept>
    </Location>

</VirtualHost>

I am SURE this one has some redundancy, but quite honestly, I don’t want to deal with it anymore- I have far more important things.

• Dollhouse (Finished)
• Defying Gravity (Finished)
• Fringe (Caught up as of Feb 12th)
• Tosh.0 (Currently Follow)
• Caprica (Currently Follow)
• Lost (Caught up as of Mar 21th)
• Stargate Universe (Caught up as of Feb 12th)
• Heroes (Caught up as of Mar 21th)
• CSI (Followed)
• Scrubs (Followed)
• House (haven’t followed but like it)
• Chuck (haven’t followed but like it)
• Bones (haven’t followed but like it)
• Dexter (haven’t followed but like it)
• Big Bang Theory (haven’t followed but like it)
• Dirty Jobs (haven’t followed but like it)
• Mythbusters (haven’t followed but like it)
• CSI: Miami (haven’t followed but like it)
• Doctor Who (Caught up as of Jan 1st)
• Legend of the Seeker (Binge)
• Supernatural (Binge)

Seems like a lot, but I never really watched them all at the same time. Shows like Chuck and House I’ve caught bits of over the years and like, where as Supernatural and Doctor Who I watched episodes back-to-back, devouring entire seasons over the course of a few days. So how did I get so far behind on the others? Combination of “christmas break” in programming combined with new job and crazy schedule didn’t help, but the standard “show stopped and I have no idea when it’ll be back on” yearly routine was what really killed it.

Anyways, now that the 2009/2010 season is picking back up and I need to figure out what I’ve missed, and how to catch up. I wish that the networks would better utilize Hulu, too- I can’t afford boxed sets, so that gives me the option of watching it on Hulu with commercials, or finding it elsewhere and them not getting a cent. My budget dictates what I watch- I’d like to watch the TrueBlood, but I can’t afford HBO. If they put their shows on Hulu behind a paywall, I won’t watch them, either.

OH, and a special shoutout to CW. They have flash all over their site, unless you try to watch their shows- they they want to install their own little EXE, which means their either too stupid to use one of the bazillion flash video players out there, or they want/need functionality that flash doesn’t provide- something lower level, like spyware would need. To CW, I say “thank you for Supernatural, lose the shitty website and maybe I’ll use it.”