Updated to 3.0
June 18th, 2010
Now lets see what breaks…
Some day I hope you’ll join us in the future, where websites don’t fail miserably for having the wrong user-agent string. Here’s the problem- I go to check my home value at cyberhomes only to receive the following message on every single page:
Your browser is not supported.
We do not support your browser yet…
If you are on a Windows machine please use:
* Internet Explorer 6 and up
* Firefox 1.5 and up
* Safari Beta 3 and upIf you are on a Mac please use:
* Safari
* Firefox 1.5 and upYour current browser is Mozilla 1.9.2.5
There are several problems with this:
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.5pre) Gecko/20100501 Ubuntu/9.10 (karmic) Namoroka/3.6.5pre" I can only presume that since it doesn’t say “firefox”, it fails.I end up having to use the following connection string:
"Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.8) Gecko/20061025 Firefox/1.5.0.8"
and the sad part was YOUR SITE LOOKED FINE. This kind of ungraceful degradation irritates the hell out of many people. If you don’t support my browser, I get it- you don’t want to give people a bad experience. You know what you do rather than make the site completely useless? You pop up a little window saying:
Sorry, we do not support your browser yet…
For better browsing experience, we suggest you try the following:
If you are on a Windows machine please use:
* Internet Explorer 6 and up
* Firefox 1.5 and up
* Safari Beta 3 and upIf you are on a Mac please use:
* Safari
* Firefox 1.5 and upYour current browser is Mozilla 1.9.2.5
[x] continue anyways…
Note that the little [x] is a close button- VERY important on a warning like that. This allows the users to continue on viewing your website on the off chance that you accidentally mislabeled them.
This is 2010. This is the future. An issue like this was annoying back in 1998- there’s no reason a pretty site like yours should be doing this. Get with the program.
Sincerely,
- Jesse
Over the past few years I’ve been doing a lot of work with JBoss and tomcat. One issue we’ve always had is bringing some level of sanity to ports that are in use. My current situation is somewhat abnormal; we have 12 applications, each with two instances, each with 7 ports- that’s a total of 168 ports that we need to keep track of. Now, multiply that by tomcat, apache and nagios’ configurations. Now multiply that by Dev, QA, ITL, PRJ, Preprod and Prod environments on segregated hardware. Even though a LOT of these ports can overlap, that’s a LOT to keep track of in a flat file or spreadsheet. We also require a certain level of flexibility:
The required flexibility makes numbering conventions difficult. The solution we came up with is to key the information and use it to build the port number. In order to do that, we had to make a few assumptions:
From here we could allow the following constraints:
With that in mind, port numbers will be laid out with the following structure: XYYZZ
X = Instance ID (e.g. 1-4)
YY = application ID (Helium,Hydrogen,Lithium, etc )
ZZ = Port Type (jmx, ajp, snmp, etc)
1 Instance1
2 Instance2
3 Instance3
4 Instance4
01 Hydrogen
02 Helium
03 Lithium
04 Beryllium
05 Boron
06 Carbon
07 Nitrogen
08 Oxygen
09 Fluorine
10 Neon
11 Sodium
12 Magnesium
01 Shutdown
02 JMX
03 HTTP
04 AJP
05 SNMP
06 REPL
07 MCASTDEV
08 MCASTQA
09 MCASTUAT
10 MCASTPREPROD
11 MCASTPROD
This allows us to define the following:
Other than the Multicast port kludge, this way seems to work pretty well. Numbers are easy to parse visually and can be maintained with a centralized key on the wiki. Anyone have any suggestions or examples of how they’ve solved this problem?
The Following is an ugly little perl script I’ve whipped up for generating contact entries for Nagios; each file is read via a cfg_dir command in the nagios.cfg. This script is set to run nightly, capturing any new users and updating users and groups automagically.
Let me know if you find it useful.
#!/usr/bin/perl # read your ldap repo and generation nagios config files for contacts # and contact_groups use strict; use Net::LDAP; use Data::Dumper; # Set all your custom configs here: my $ldaphostname='ldap.example.int'; my $groupdn='ou=People,ou=Groups,dc=example,dc=int'; my $peopledn='ou=People,ou=active,ou=Users,dc=example,dc=int'; my $contactdir='/etc/nagios/contacts'; my $ldap = new Net::LDAP ($ldaphostname); $ldap->bind() || die "failed to bind"; #search for groups my $mesg = $ldap->search( base => $groupdn, filter => "(objectclass=posixgroup)", attrs => ['cn','memberuid'], ); #loop through each entry while (my $group= $mesg->pop_entry() ){ #set the name of the group my $cn=$group->get_value('cn'); #create a comma-separated list of group members my $members=$group->get_value('memberuid', asref => 1); $members= join( ',', @$members); # create our configuration block my $contactinfo=" # We only have one contact in this simple configuration file, # so there is no need to create more than one contact group. define contactgroup{ contactgroup_name $cn alias $cn members $members }\n"; # write this block to a file named after the group cn print `echo "$contactinfo" > $contactdir/group_$cn.cfg`; } #search for users my $mesg = $ldap->search( base => $peopledn, filter => "(objectclass=person)", attrs => ['uid','cn','mail','description'], ); #loop through each entry while (my $user= $mesg->pop_entry() ){ my ($pager,$addr); # grab the four fields we're interested in my $uid=$user->get_value('uid'); my $cn=$user->get_value('cn'); my $mail=$user->get_value('mail'); my $desc=$user->get_value('description'); # Note that I store the user's SMS address and Instant Messenger # addresses in the description field in the following format: # SMS:1234445555@some.url; # IM:morgajel@yahoo,morgajel@gtalk; # for those interested, you can send email to an sms address # with help from <a href="http://sms411.net/how-to-send-email-to-a-phone">http://sms411.net/how-to-send-email-to-a-phone</a> #strip pager info out of desc and format it #(note it's in sms format): if ($desc =~/SMS:([^;]+);?/){ $pager= " pager $1\n"; } # strip addresses out of desc and format them: if ($desc =~/IM:([^;]+);?/){ my @multiple_addr=split(/,/,$1); my $count=1; # break each address out into an addressX line # (for informational purposes); foreach my $addr (@multiple_addr){ $addr=$addr." address$count $addr\n"; $count=$count+1 ; } } #generate the first half of the contact info my $contactinfo=" define contact{ contact_name $uid ; Short name of user use generic-contact alias $cn ; Full name of user email $mail $pager $addr }"; # write this block to a file named after the user uid `echo "$contactinfo" > $contactdir/$uid.cfg`; }
This sorta puts it in perspective… One of my few remaining coworkers at EPI made the following comment:
(11:33:42 AM) Jeff: so many gone
(11:34:10 AM) Jeff: jesse
matt
carrie
lisa
stacey
kurt
roger
jim
naveen
brandon
lee
bink
tristan
jason
jen
sean
drew
radha
VJ
jackie
andy
noelle
whip
reedy
jeremy
keith
ryan
pankaj
nirupa
asad
santhi
rick
limer
meads
mick
(11:34:42 AM) Jesse: you should post that as a facebook status update. one comma separated list.
(11:39:06 AM) Jeff: lol
(11:39:12 AM) Jeff: those are just people i knew and worked with in IT
(11:39:18 AM) Jeff: i’m not counting others I didn’t work with or know in IT
SUBJECT: [#24413540] Suspension Notice CPU Limits Reached (1st Warning)
Hello,Due to the amount of CPU and/or memory resources used by your account, we were forced into suspending the account. Please understand that we only suspend an account as a last resort and want to help you track down the cause as quickly as possible.
Shared servers (MegaPhase, Prophase, and ANHosting accounts) are only allotted 10% of the server’s resources (CPU and Memory) at any given time. This
restriction is put in place to keep balance on the server. The first step is to
try to isolate what content on your website is causing the overage. If this
seems unexpected, it could be a recent change to your site (such as installing
a new wordpress plugin, mailing list application, etc). Any information you can
provide about recent changes can help us to isolate the problem. However, we
were able to retrieve the following files and/or scripts that likely caused the
suspension:————————
literaryescapism.com 2.33 1.38 0.1
Top Process %CPU 34.0 [php]
Top Process %CPU 25.0 public_html/index.php
Top Process %CPU 23.0 public_html/index.php————————
This was the beginning of an email I received from our hosting company, telling me that Jackie’s site was suspended due to usage. I’m not disappointed so much that they suspended it (which was the proper course of action), but in how they responded to me. Here’s my response to the warning email:
I’m trying to figure out if this is new behavior from a wordpress exploit or if it’s honestly just grown to this point; do you have any trending data for my cpu usage?
Side note- the access-logs linked in my home directory appear to go nowhere:
access-logs -> /usr/local/apache/domlogs/user
that path doesn’t exist.I would like to help in any way I can. As a side note, I’ve noticed quite a few “errors connecting to mysql” in the apache logs; could there be an underlying issue with mysql that’s causing processes to back up and chew cpu time?
I’m also seeing some defunct php processes from other users, leading me to wonder if this is more of a systemic problem than one with just my account.
(sorry, I’m an apache/linux admin myself- let me know if there’s anything I can do to assist. I can be reached on gtalk, aim or yahoo.)
And their response:
Hello,
Unfortunately we do not have any further data except for what was given in the e-mail:
literaryescapism.com 2.33 1.38 0.1
Top Process %CPU 34.0 [php]
Top Process %CPU 25.0 public_html/index.php
Top Process %CPU 23.0 public_html/index.phpYour account has been unsuspended so you can troubleshoot this issue. Please let us know if you have any further questions.
–
Best regards,xxxxxxxxxxxxxx
Technical Support Representative
Hosting Services, Inc.
It’s sort of dumbfounding to be completely shut down like that from a TSR. I would have hoped that he would have seen that I A) know what I’m talking about and B) would have booted me up to someone who could better assist me. Instead he unlocks the account (which, btw, was never locked or suspended as far as I could tell from the time I got the email and checked until he sent the notice that it was unsuspended) and says “sorry, we have no data to help you figure out what happened.” Note that he didn’t address my questions regarding the broken symlink, mysql availability, or defunct processes.
Flustered, I responded with this:
Is it still using such high CPU load? I noticed several processes on the server that were having issues when I looked; many of which were not mine- The load on the server was 114 when I checked it last night. Regardless of server, that’s high, and I wasn’t one of the high processes listed.
I suspect this was a systemic issue- perhaps mysql went down causing multiple sites to freak out. If that’s the case (and there was a problem with the system itself), troubleshooting from my end is near impossible, especially since you have no trending data on usage. To determine if this was a one-time issue or a building issue.
I’d like to help you guys, but ya gotta work with me here. This is what I do for a living- I’m the Apache/Linux administrator. My wife’s site has been running for 6 months with no major changes (other than simple wordpress upgrades, none of which
were in the last week) and no notable increase in traffic or bandwidth usage over the past week.If you’re incapable of providing details as to whether or not there was a system issue (or even if other clients had similar issues at the same time) my only course of action would be to take our sites to another provider. I don’t want to do that; I’ve been happy with you guys so far.
Please, check and see if there were multiple clients on this server who received warnings as well; I suspect I wasn’t the only one. From where I’m standing the server does not look very healthy- I’m seeing multiple defunct processes:
nobody 6857 0.0 0.0 0 0 ? Z 08:46 0:00 [httpd]
1027 10565 15.0 0.0 0 0 ? Z 08:48 0:00 [php]
32003 10573 0.0 0.0 0 0 ? Zs 08:48 0:00 [cpsrvd-ssl]Please let me know if there is a sysadmin I can work with to resolve this issue. Like I said, I don’t think this is a problem with my site necessarily.
We’ll see how they respond. Hopefully they’ll be useful- I just switched to them in October, so I really don’t want to have to find another host.
Here’s how I stumbled across this (names changed to protect the guilty):
Suppose you have 4 tomcat instances, A1, A2, B1 and B2. A1 and A2 run application Apple/. B1 and B2 run application Breakfast. Someone decided “Hey, we can save deployment time if we symlink A1 and A2 to the same directory.”
Obvious questions aside (why bother setting up two if they’re going to be hosting the same content on the same box?), lets look at what happens when you misconfigure this relatively simple idea. The tomcat instances each had a webapps directory where their application was deployed, for example, A1/webapps/Apple/ or B2/webapps/Breakfast/. The goal was to create a common deployment location for A1 and A2, B1 and B2, however a shortcut was taken and the webapps directory was symlinked rather than Apple and Breakfast, then BOTH Apple and Breakfast were placed in the shared directory. This resulted in 2 things:
* a unified place to deploy code
* every single tomcat instance loading BOTH Apple and Breakfast
This means B1/webapps/ now includes Apple/. While this may not seem like a problem (B2′s context root is set to Breakfast/, for example), it can lead to complications down the road:
* each instance now needs to load and compile things for both applications, resulting in wasted resources and startup time.
* If Breakfast/ happens to include it’s OWN Apple/ (because apples are part of a complete breakfast,) B2/webapps/Breakfast/Apple/ is now ignored in favor of B2/webapps/Apple/.
Fortunately I’d stumbled across this misconfiguration the day before others did, so I was able to quickly deduce what was wrong. Why didn’t I fix it? Because I wasn’t sure I was right, and I didn’t want to break someone elses apparently broken configuration any further. Besides, I’m trying to fight this obsessive feeling that “if I don’t do it myself, I won’t know if it’s right,” and doublechecking everyone’s work feeds that.
Fortunately the fix was quick and simple. It may not seem like that big of a deal, but you must remember this is a simplified view of the situation. the REAL setup had 10 applications, with some weighing in at 4-8 gigs each. While most of that is content (images and flash that tomcat doesn’t have to process), sorting through an extra 600k files *per project* will slow the system down.
Now, this behavior may seem dumb until you think of it from a tomcat point of view. “I know you have this /register/ under your context root, but you also have this application named register/, so I’m gonna go ahead and presume you want to run the full application rather thank this dinky directory. Besides, it’s probably a ‘sorry, registration is currently out of order’ in case the full application is taken offline.”
So the problem I’m having is with limiting LDAP users access to WebDAV directories; specifically, how do I keep devs from committing to the release branch. The setup is each of the large Projects (Project1, Project2) has a trunk and release branch; however some pesky devs try to ninja changes into the release branch, circumventing the entire process. That’s bad. The access should go like this (note this is a subset of the mess I’m dealing with):
Specifically, I should be able to commit to trunk but not release, however that doesn’t seem to be the case. Here’s an abbreviated version of my vhost:
<VirtualHost 10.0.0.5:443>
blah blah blah snip...
<Location />
DAV svn
SVNParentPath /var/svn/
SVNPathAuthz off
AuthName "SVN Access"
AuthType Basic
AuthLDAPUrl "ldap://ldap.example.int:389/ou=Users,dc=example,dc=int?uid"
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPGroupAttribute "memberUid"
<LimitExcept none>
Require valid-user
</LimitExcept>
</Location>
<Location /Project1/trunk>
# Everyone can read, but only devs (and I) can change.
<LimitExcept REPORT GET OPTIONS PROPFIND>
Require ldap-group cn=devs,ou=Groups,dc=example,dc=int
Require ldap-user morgajel
Satisfy any
</LimitExcept>
</Location>
<Location /Project1/branches/release>
# Everyone can read, but only QA can change.
<LimitExcept REPORT GET OPTIONS PROPFIND>
Require ldap-group cn=qa,ou=Groups,dc=example,dc=int
Satisfy any
</LimitExcept>
</Location>
</VirtualHost>
Any thoughts as to why I’m still able to write to release? and no, I’m not in the QA group; I suspect it has something to do with the Locations essentially being nested. Since we’re managing users and groups with LDAP, simple SVN ACLs won’t work, and I’m not really sure how to accomplish what I need to do.
Thoughts?
Side note: REPORT GET OPTIONS PROPFIND are the only methods needed for read-only svn webdav access. Fun fact, huh?
I was overthinking the situation- Apache config is not programming. There is no inheritance between locations. There is no nesting. Once you create a new location, you need to set up perms for that, so setting the base / with read/write for everyone, I then define sublocations
<Location /Project1/trunk> # If you want to write to trunk, you need to be one of the required people. You can still read it. <LimitExcept PROPFIND OPTIONS GET REPORT> Require ldap-group cn=devs,ou=People,ou=Groups,dc=mrm,dc=int </LimitExcept> <Limit PROPFIND OPTIONS GET REPORT> Require valid-user </Limit> </Location>
<Location /Project1/branches/release>
# Everyone can read, but only QA can change.
<LimitExcept REPORT GET OPTIONS PROPFIND>
Require ldap-group cn=qa,ou=Groups,dc=example,dc=int
Satisfy any
</LimitExcept>
<Limit PROPFIND OPTIONS GET REPORT>
Require valid-user
</Limit>
</Location>
What I was missing was the second half of the limits, thinking it would inherit from /. It doesn’t. Without that require valid-user, it was allowing unauthenticated users to read the files (which was no good). Life is good (until I find out where this is broken).
So the above didn’t work when I retested it, so I tried one last time with good results (note the goals have changed, but the principles still apply):
<VirtualHost 10.0.0.5:443>
SSLEngine on
SSLCertificateFile /etc/pki/certs/example.int.crt
SSLCertificateKeyFile /etc/pki/certs/example.int.key
DocumentRoot /var/www/svn.example.int/docs
ServerName svn.example.int
ServerAlias svn.exampleco.com
ErrorLog logs/svn.example.int-error_log
CustomLog logs/svn.example.int-access_log common
<Location />
DAV svn
SVNParentPath /var/svn/
SVNPathAuthz off
AuthName "SVN Access"
AuthType Basic
AuthLDAPUrl "ldap://ldap.example.int:389 /ou=Users,dc=example,dc=int?uid"
AuthBasicProvider ldap
AuthzLDAPAuthoritative on
AuthLDAPGroupAttribute "memberUid"
AuthLDAPGroupAttributeIsDN off
order Deny,Allow
Deny from all
Satisfy any
# access
<LimitExcept NONE>
Require valid-user
Satisfy any
</LimitExcept>
</Location>
<Location /ProjectA/branches/releases//>
order Deny,Allow
Deny from all
Satisfy any
#read-only access
<Limit GET PROPFIND OPTIONS REPORT>
Require valid-user
Satisfy any
</Limit>
# write access
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
# Require ldap-user jmorgan
Satisfy any
</LimitExcept>
</Location>
<Location /ProjectB/branches/releases//>
order Deny,Allow
Deny from all
Satisfy any
#read-only access
<Limit GET PROPFIND OPTIONS REPORT>
Require valid-user
Satisfy any
</Limit>
# write access
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
# Require ldap-user jmorgan
Satisfy any
</LimitExcept>
</Location>
<Location /ProjectC/branches/releases//>
order Deny,Allow
Deny from all
Satisfy any
#read-only access
<Limit GET PROPFIND OPTIONS REPORT>
Require valid-user
Satisfy any
</Limit>
# write access
<LimitExcept GET PROPFIND OPTIONS REPORT>
Require ldap-group cn=Administrators,ou=People,ou=Groups,dc=example,dc=int
Require ldap-group cn=Team Leads,ou=People,ou=Groups,dc=example,dc=int
# Require ldap-user jmorgan
Satisfy any
</LimitExcept>
</Location>
</VirtualHost>
I am SURE this one has some redundancy, but quite honestly, I don’t want to deal with it anymore- I have far more important things.
I watch a lot of TV… well, I should qualify that; I do a lot of stuff with the TV on. I pay attention to the shows, but I’ll be doing other things. Since about halfway through November I started losing track of shows- mainly because of the stupid seasonal holiday crap. Here’s the stuff that I’ve watched at one time and lost track of:
Seems like a lot, but I never really watched them all at the same time. Shows like Chuck and House I’ve caught bits of over the years and like, where as Supernatural and Doctor Who I watched episodes back-to-back, devouring entire seasons over the course of a few days. So how did I get so far behind on the others? Combination of “christmas break” in programming combined with new job and crazy schedule didn’t help, but the standard “show stopped and I have no idea when it’ll be back on” yearly routine was what really killed it.
Anyways, now that the 2009/2010 season is picking back up and I need to figure out what I’ve missed, and how to catch up. I wish that the networks would better utilize Hulu, too- I can’t afford boxed sets, so that gives me the option of watching it on Hulu with commercials, or finding it elsewhere and them not getting a cent. My budget dictates what I watch- I’d like to watch the TrueBlood, but I can’t afford HBO. If they put their shows on Hulu behind a paywall, I won’t watch them, either.
OH, and a special shoutout to CW. They have flash all over their site, unless you try to watch their shows- they they want to install their own little EXE, which means their either too stupid to use one of the bazillion flash video players out there, or they want/need functionality that flash doesn’t provide- something lower level, like spyware would need. To CW, I say “thank you for Supernatural, lose the shitty website and maybe I’ll use it.”
So I’ve mentioned before that I picked up a motorola droid recently for my new job and have been playing with it relentlessly. One of the many things I like to do with a moment of downtime is browse the android marketplace- lots of interesting things in there (I browse just the free section currently).
Anyways, in the marketplace is an app that looks like a star trek tricorder. When I first saw it, I downloaded it thinking it would be a neat little image display that I could taunt my friend The General with (he’s a big trek fan) when I see him later this year.
I didn’t expect the damn thing to actually *WORK*. Here’s a list of functionality from it’s site:
So, there you have it- I have a working tricorder, beeps and all. It’s not that useful, really, but does it matter? HELLO! touch screen tricorder! Nerd, right?
Wait, it gets better. Jackie and I are looking at getting a bigger vehicle, and this weekend we plan on going for some test drives. One thing car commercials always say is “it’s such a smooth ride…” Well, now I have a way to MEASURE how the smoothness of the ride.
To repeat, I have a working tricorder that I plan on using for real world decisions.
This image? That was just me picking up the phone- it shows how much it was jostled. If I set that on the console as Jackie drives around the same area, it will give me an idea of how bumpy the ride will be. This is an actual *useful* application of the motion sensor if it was used in an app built for this purpose. Sadly, I don’t have the time to write it myself, so when we go for a test drive, I’ll use the tricorder app.
Now, picture it- Jackie and I are sitting in a new car with a salesman telling me how awesome his car is, and I’m sitting there, WITH A TRICORDER, going “BEEE-LOOP BOOPBEEP” while he’s trying to talk, monitoring how rough the ride is while he tells me it’s smooth.
So should I be stoked or embarrassed?