Don Don’s Garage

Smog isn't a result of people modifying their cars and putting big engines in them. It's a result of the people who are disinterested in their cars and don't take care of them until they break down.

It irks me to no end to see some old jalopy billowing smoke out it's tail end, and I'm not allowed to bore and stroke out my excellently tuned 1998 truck, just because this jerks car is 25 years older and grandfathered in.  You can bet my next vehicle will be 1975 or older.  (The CA emissions smog cutoff)  Where I might have once been satisfied with a 454, I will put in at least a 527 just to spite CARB.

CARB stiffles innovation preventing any but the largest of companies from even attempting to bring something new to the aftermarket car parts arena.  They are preventing major advances in making the performance aftermarket environmentally friendly. (ok, less hostile)

Now they are going to far, they want to ban Black Paint.  Not because of any chemicals in the paint, or that producing said paint is environmentally unfriendly, but because of its color. http://www.thetruthaboutcars.com/carb-so-crazy/

CARB should be allowed to set new car emissions standards as they have been. Those vehicles should not be allowed to exceed those emissions standards when smogged.  What CARB SHOULDN'T be allowed to do is dictate which products can be sold, installed, or painted onto said vehicles.  If I buy some combination of parts that exceed my vehicles emissions, I won't pass smog, I have to go fix it until it does. Simple.  If they did that, the market for high performance but emissions friendly car parts would flourish.  Innovation would ensue, breakthroughs would be made, jobs would be created (or at least maintained).

What next, everybody has to buy a white Prius?

That's a serious problem, because I'm not sure how I'm going to fit that 527 under the Prius' hood.  (There's a smog exemption for CNG motors, and you can convert a 527)

My truck is a 1998 Jeep Grand Cherokee 5.9 Limited.  I didn't know when I bought it that it was a rare truck. I was fascinated by it on the first test drive in 2009 after moving to the Bay area from Washington State.

From wikipedia:

The 5.9 Limited was a Jeep Grand Cherokee produced only for the 1998 model year, having more luxury and performance than that of the regular Limited. Chrysler manufactured nearly a quarter million Grand Cherokees in 1998. Of those, less than fifteen thousand were 5.9s. It housed a Magnum 5.9 LV8 engine with an output of 245 hp (183 kW) and 345 lb·ft (468 N·m) of torque, going from zero to 60 mph (100 km/h)in only 7.3 seconds[6] (Motor Trend measured this at a slightly faster 6.8 seconds[7]), making it the quickest SUV available that year. The performance of the 1998 5.9 L V8 has been surpassed by Jeep only with the introduction of the 2006 Jeep Grand Cherokee SRT8, which housed a 6.1 L Hemi Motor.[8] The 5.9 Jeep Grand Cherokee was named the 1998 four-wheel drive vehicle of the year by Peterson's 4-wheel & Off-Road.[9]

It's stock configuration:

* Engine: 5.9-Liter, OHV, SMPI V-8
* Horsepower 255 @ 4000 RPM
* Torque (lb-ft) 350 @ 3200 RPM
* 0-60 MPH 7.0 seconds
* Brakes (F/R) Disc (ABS)/disc (ABS)
* Drive Train Front-engine/all-wheel-drive
* Transmission Four-speed automatic

From: North American Grand Cherokee Association (NAGCA)

The 5.9 Limited was advertised as the "Worlds fastest sport utility vehicle" in advertisements that Jeep ran during 1998, and that was borne out by testing. A 1998 press release by Chrysler stated the following:

The following is a summary of changes to Chrysler's engines for 1998:

In the 5.9L V-8 engine (Grand Cherokee, Dakota and Ram pick-up, and Ram van), a new camshaft adds up to 15 bhp and the torque curve is widened to provide more responsive performance in the normal driving range for all models. In addition, specific to the Grand Cherokee, 15 degrees more spark advance (which requires premium fuel) and a 25 percent reduction in backpressure makes a total increase of 25 bhp over the current 5.2L engine. The cooling fan motor is now electric, eliminating a power drain of up to 20 bhp.

The performance increases of the greater spark advance and electric fan are some of the main reasons the 5.9 Limited performed so much better than the other Chrysler trucks with 5.9L engines. The magazine test results were impressive:

Differences between the 5.9 Limited and the 5.2 Limited:

* A stronger 46RE transmission than the 5.2 L with a heavier output shaft
* Quadratrac heavy duty NV249 transfer case
* Standard trac-lock rear differential
* An electric fan
* A high-output 150A alternator

1998 Jeep 5.9 Limited (Motor Trend Jan/98):
* 0-50 - 4.90 sec.
* 0-60 - 6.80 sec.
* 0-70 - 9.20 sec.
* 60-0 Braking - 126.00 ft.
* 1/4 Mile - 15.20 sec.
* Slalom - 58.40 mph
* Skidpad - 0.74 g

http://www.nagca.com/grandtech/52and59comparison.htm

Things I've done to the truck so far:

* Tune up with E3 Spark Plugs
* Replacement Mopar High Performance spark plug wires
* Replacement Rotor and Distributor Cap from Mopar Performance
* K&N FIPK Cold Air Intake
* Doug Thorley Headers (aka Mike Leach headers)
* Flowmaster SUV 50 Muffler
* MSD High Performance Ignition Coil
* Overhauled the rear end replacing axles and differential bearings

Big props to Craig Hill and his crew at Top of the Hill Performance Center http://topofthehillpc.com/ who straightened out the botched exhaust install from another shop and installed the headers for me. Fantastic professional job.

Doug Thorley Headers also gets some props for going above and beyond the call of duty to support their customers.  http://dougthorleyheaders.com/

Next on my todo list:

* Get it Chassis Dyno'd http://www.dynocom.net/
* Take it to Infineon Wednesday Night Drags http://www.infineonraceway.com/raceway/race/wednesday_night_drags/
* Go to Bonneville, and get it into the 130mph club. http://www.bonnevilleracing.com/130-mph-club.asp

WebROaR http://webroar.in/, a Ruby web application server designed to support Rails and Rack based applications recently came to my attention. My currently preferred Rails stack is Apache2 http://httpd.apache.org/, Ruby Enterprise 1.87 http://www.rubyenterpriseedition.com/, and Passenger http://www.modrails.com/, using NewRelic RPM http://www.newrelic.com/features.html for notifications and metrics.  I wanted to see how WebROaR stacked up as a replacement.

I began my investigation the way I do any application. I start by writing an installation script.  My tool of choice for deployments right now is Capistrano http://www.capify.org, and I've recently been interested in Deprec http://deprec.failmode.com/, a framework of recipes built on top of Capistrano specifically designed to install on ubuntu, which by chance, is also my preferred linux distribution. So I decided to extend Deprec with a WebROaR installation recipe.

WebROaR began showing it's relative youth immediately.  I expect to be able to install a production application non-interactively. That is to say, I should be able to script the entire affair by passing values to the installer, configure, and make scripts via the command line.  The current version of WebROaR's install will prompt you for up to four different values and provides no mechanism to create a non-interactive install.

With Capistrano a non-interactive install can be worked around by examining output and responding as a user would with stored answers. Other deployment systems based on shell scripts will find installing WebROaR challenging.

The next issue to deal with is provisioning the application.  There was no mechanism, save rewriting the entire configuration file, to create a new application from Capistrano.  For this reason I left any configuration of the server or applications out of the deprec recipe.

The problem with re-creating the entire configuration via Capistrano is a simple one, when you install an application with Capistrano it doesn't normally have any concept of any other applications that may be hosted by your server.  So a rewrite of the entire config would end up wiping out all other applications unless pains were taken to make it tolerant.

Another sign that WebROaR is fairly young is that when I deployed a simple test application, the deploy scripts hadn't properly used the bundle command to unpack all of the application's dependent gems. It left the application in a really sorry state.  WebROaR's user interface didn't expose the error condition, nor was it able to serve the configured application.  It wasn't until I connected to the box via ssh and tried to run script/server in the applications deploy folder before I got any clue as to what the problem might be.  WebROaR should have done what it could to raise the problem to my attention within its exceptions section. I dealt with several errors where the WebROaR UI wasn't any help which gives me the impression there are many areas where WebROaR is rough around the reliability edges.

It's great that WebROaR's concept is built around interactive management, Apache, Passenger, Mongrel, and Thin don't really provide any instrumentation without something like NewRelic's RPM.

The exception reporting was definately useful, and presented nicely, if not exactly timely.  After triggering a few errors it still took several refreshes of WebROaR's UI before the information was ready to be examined.  Similarly there was a delay from when the first requests were generated before the graphs in the analytics could be reviewed.

Things like email and sms notifications are not in the product yet. I would have expected the ability to automatically send notifications and have some control over the process when metrics go above or beyond certain thresholds, or when certain types of exceptions occur.

The admin interface is protected by only a single username and password, multiple users aren't possible. Obviously role based access controls aren't in the product yet either.

The documentation is rather good for what features there are. But there aren't that many features.

I didn't do any kind of exhaustive performance benchmarks or stress tests because my curiosity had already been sated by getting my test application up. WebROaR doesn't compare to the configurability, stability, and breadth of options that I could even remotely consider it as a replacement for my current Rails Stack, even if all of it's claims about performance are true.

If it's going to become a production ready application, it's going to have to make it effecient for sysadmins to install and manage it from the command line and report error conditions better.  I can absolutely see it's alure to hosting companies as a great way to host rails apps if they can delegate control to end users and use an API to control the configuration. The project is very young, announced November 25th, 2009 according to their Blog, and it's incredibly promising, I'll definately check it out again when they get near a 1.0 release.

Deprec WebROaR recipe: http://github.com/donnoman/deprec/blob/2cdac7ed5c322d41512673a2e199707b3e47de...

Related changes to deprec: http://github.com/donnoman/deprec/commit/2cdac7ed5c322d41512673a2e199707b3e47...

Any fixes will be comitted here: http://github.com/donnoman/deprec/tree/webroar

Sample App Deploy Script using webroar: http://github.com/donnoman/flitten_deploy/commit/76e0ae350d75127fb43e7cc181f0...

Why on earth would you want to do this?

For a very specific use case.

1. You do development of a web application locally that needs a wildcard domain name.  ie: where you want 192.168.1.1 to answer any http request for *.example.com ( example.com, www.example.com, blah.example.com, ridiculously.long.example.com) without specifically configuring each name in a hosts file.
2. You have previously setup your workstation to use Google's public DNS and don't want to lose the benefits by setting up your own nameserver ( Also applies if you forward to ANY upstream DNS servers, just swap the Google IP's out with the ones you want.)
3. You are using Windows XP (This should work on Vista). You can apply the same configuration files here for any version of BIND. ( OSX Users should look at a utility called DNSEnabler that provides a dead simple graphical user interface to manipulate BIND on OSX ), but the instructions steps here are specific to Windows.

Why Google's Public DNS?

It's faster than your ISP's default DNS which will make browsing faster.  You can also use these instructions for other DNS networks like OpenDNS, or even forward them back to your ISP if you so wish. http://code.google.com/speed/public-dns/

Why Windows XP?

Because it's what I have at home. I use OSX and Linux at work where I use DNS Enabler http://cutedgesystems.com/software/DNSEnabler/ on OSX and I've previously posted how to configure BIND on Linux to use Google's Public DNS in an office environment, which pretty much covers all my bases.

Why BIND?

It is the defacto DNS implementation, it's well worn and battle tested; and they make a Windows distribution of the software that's freely available.  While I like Microsoft's DNS server, it's only available on their server products, so regrettably it's not an option for Windows XP. 

There are other Windows based DNS servers, but most of them are commercial and are not as clean, cheap, or as easy as DNSEnabler, so it wasn't worth the time to research any of them. I also have experience configuring BIND so I figured I could just share my configuration and hopefully the 2 other people on the internet that have the same needs as I can benefit from my experience.

For more information about BIND: https://www.isc.org/software/bind . A great reference about DNS in general as well as BIND is the venerable book "DNS for Rocket Scientists" http://www.zytrax.com/books/dns/

What are the steps?

1. Download latest BIND zip for windows: https://www.isc.org/software/bind
2. Unpack and run: BINDInstall.exe
3. View: C:\windows\system32\dns\bin\readme1st.txt
4. Start->Run: C:\windows\system32\dns\bin\rndc-confgen -a
5. Use Explorer, navigate to c:\windows\system32\dns, right click "etc", properties, security, add, "named", click "full control", OK, OK
6. Download ftp://ftp.internic.net./domain/named.root and save it to C:\windows\system32\dns\etc\named.root to seed BINDs root hints.
7. Use Windows Services to start/stop the Named Service
8. Start -> Right-Click "My Computer", manage , click "Services", Look for "ISC BIND", Right-Click start,stop, or restart.
9. Start -> Right-Click "My Network Places", properties, Right-Click your active "Local area connection", properties, click "Internet Protocol (TCP/IP)", properties, Use following dns server: 127.0.0.1
10. For any other system on your network that you want to use this nameserver you would need to use your hosts real IP address for the local network (It may change if your on dhcp, remember to check if you have "internet" problems.) and repeat only step 9 on that system, assuming its Windows.  If it's a Unix platform you would edit /etc/resolv.conf.  If you want to get fancy you can edit dhclient.conf to prevent DHCP from overwriting your custom nameserver selection.
11. If you customized hosts in your hosts file that will be covered by the wildcard you must remove them: C:\windows\system32\drivers\etc\hosts
12. If you intend on using the dns server from workstations other than the localhost and you are running any kind of firewall you will need to open up port 53 both UDP and TCP.  For Windows' included firewall Start -> Right-Click "My Network Places", properties, Right-Click your active "Local area connection", Select the "Advanced" tab, settings, Select the "Exceptions" tab, Add Port ( name: DNS, port: 53, type: TCP), repeat last step except for UDP this time.

C:\windows\system32\dns\etc\named.conf is as follows:

options {
  // version statement - inhibited for security
  // (avoids hacking any known weaknesses)
  version "get lost";
  // optional - disables all transfers
  // slaves allowed in zone clauses
  allow-transfer {"none";};
  forwarders {8.8.8.8; 8.8.4.4;}; //GOOGLE Public DNS
  directory "C:\WINDOWS\system32\dns\etc";
};

view "trusted" {
  match-clients { 192.168.0.0/16; 127.0.0.1; }; // any private class c and localhost
  recursion yes;
  // required zone for recursive queries
  // retrieve from: ftp://ftp.internic.net./domain/named.root
  zone "." {
    type hint;
    file "named.root";
  };
  // basic localhost support
  zone "localhost" in{
    type master;
    file "master.localhost";
  };
  // basic localhost support
  zone "0.0.127.in-addr.arpa" in{
    type master;
    file "localhost.rev";
  };
  // this is the wildcard zone
  zone "dev.example.com" {
    // Don't forward queries for this zone.
    forwarders {};
    type master;
    // The final extension of .txt simply is so that Windows doesn't
    // think the file is executable (.com), and will open the file with your
    // systems designated text editor without any fuss.
    file "dev.example.com.txt";
  };
};

view "badguys" {
  match-clients {"any"; }; // all others hosts
  // recursion not supported
  recursion no;
};

C:\windows\system32\dns\etc\dev.example.com.txt is as follows:

$TTL 2d    ; 172800 secs default TTL for zone
$ORIGIN dev.example.com.
@             IN      SOA   ns1.dev.example.com. hostmaster.dev.example.com. (
                        2009120500 ; se = serial number
                        12h        ; ref = refresh
                        15m        ; ret = update retry
                        3w         ; ex = expiry
                        3h         ; min = minimum
                        )
              IN      NS      ns1.dev.example.com.

                              ; Retrieve the IP for the target of the wildcard
                              ; Linux: ifconfig
                              ; Windows: ipconfig
@             IN      A       192.168.1.1

www           IN      A       192.168.1.1

                            ; Retrieve the IP for the DNS server
ns1           IN      A       192.168.1.70

*             IN      CNAME   www

C:\windows\system32\dns\etc\localhost.rev is as follows:

 

$TTL    86400 ;
; could use $ORIGIN 0.0.127.IN-ADDR.ARPA.
@       IN      SOA     localhost. root.localhost.  (
                        1997022700 ; Serial
                        3h      ; Refresh
                        15      ; Retry
                        1w      ; Expire
                        3h )    ; Minimum
        IN      NS      localhost.
1       IN      PTR     localhost.

C:\windows\system32\dns\etc\master.localhost is as follows:

 

$TTL    86400 ; 24 hours could have been written as 24h
$ORIGIN localhost.
; line below = localhost 1D IN SOA localhost root.localhost
@  1D  IN     SOA @    root (
                  2002022401 ; serial
                  3H ; refresh
                  15 ; retry
                  1w ; expire
                  3h ; minimum
                 )
@  1D  IN  NS @
   1D  IN  A  127.0.0.1

To test your new caching dns server for resolving the local wildcard domain: Start->Run: "cmd", then type "nslookup whatever.dev.example.com" at the command prompt.

(If it doesn't work and you can't browse the web anymore, Look at step 9 and set dns back to "obtain DNS server address automatically", or to 8.8.8.8 and 8.8.4.4, to go direct to Google's DNS. You should also look at your Event Viewer to discover any errors, See step 8, except select the "Event Viewer" instead of "Services", then look at the "Application" log.)

The command should return:

Server:  localhost
Address:  127.0.0.1

Name:    www.dev.example.com
Address:  192.168.1.1
Aliases:  whatever.dev.example.com

You can use nslookup to check a few more sites like www.disney.com, or www.yahoo.com; to make sure that the forwarding is occuring.

Your done, enjoy.

After using tail for a long time, I've only recently had a need to became familiar with tail's ability to watch multiple files.

you can easily watch a single file, thats the tail we all know and love.

tail -f /var/log/syslog

But I've got another rsyslog directory that concentrates logs from a bunch of different servers with specific naming conventions that I can match by filespec.

For each cluster there are multiple app, database, loadbalancers and memcache servers. trying to debug a problem, I needed to tail all of the app servers at the same time.

it's dead simple, particularly if you are in the directory all the files you want to tail reside.

tail -f *production-app*

Where it matches any filenames that contain "production-app".

If I need to watch the mysql servers of my testing cluster

tail -f *testing-mysql*

Incidentally you can also tail multiple files without using a filespec

tail -f /var/log/apache2/access.log /var/log/apache2/error.log

it tails the files that match the filespec and interleaves the output with markers so that you know which log file you are looking at. Beautiful.

Other tail tricks:

combine with grep to watch for your needle before it gets buried in the haystack:

tail -f /var/log/syslog | grep "my needle"

combine with grep to exclude a bunch of annoying messages that you don't need.

tail -f /var/log/syslog | grep -v "annoying message I don't want to see"

Here's one that I commonly use to cut the cruft out of watching logs on my EC2 instances, it eliminates any lines with Connection OR Kernel in them.

tail -f /var/log/syslog | grep -v 'Connection\|kernel'

http://code.google.com/speed/public-dns/

No forwarders, not previously cached: 259ms.

; > DiG 9.3.4-P1 > disney.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
;disney.com. IN A

;; ANSWER SECTION:
disney.com. 900 IN A 199.181.132.250

;; AUTHORITY SECTION:
disney.com. 86400 IN NS huey.disney.com.
disney.com. 86400 IN NS huey11.disney.com.

;; Query time: 259 msec
;; SERVER: 192.168.250.220#53(192.168.250.220)
;; WHEN: Thu Dec 3 10:52:19 2009
;; MSG SIZE rcvd: 84

Using Google's Public DNS, not previously cached: 120ms.

; > DiG 9.3.4-P1 > disney.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 0

;; QUESTION SECTION:
;disney.com. IN A

;; ANSWER SECTION:
disney.com. 900 IN A 199.181.132.250

;; AUTHORITY SECTION:
. 52428 IN NS M.ROOT-SERVERS.NET.
. 52428 IN NS A.ROOT-SERVERS.NET.
. 52428 IN NS B.ROOT-SERVERS.NET.
. 52428 IN NS C.ROOT-SERVERS.NET.
. 52428 IN NS D.ROOT-SERVERS.NET.
. 52428 IN NS E.ROOT-SERVERS.NET.
. 52428 IN NS F.ROOT-SERVERS.NET.
. 52428 IN NS G.ROOT-SERVERS.NET.
. 52428 IN NS H.ROOT-SERVERS.NET.
. 52428 IN NS I.ROOT-SERVERS.NET.
. 52428 IN NS J.ROOT-SERVERS.NET.
. 52428 IN NS K.ROOT-SERVERS.NET.
. 52428 IN NS L.ROOT-SERVERS.NET.

;; Query time: 120 msec
;; SERVER: 192.168.250.220#53(192.168.250.220)
;; WHEN: Thu Dec 3 10:53:45 2009
;; MSG SIZE rcvd: 255

In this trivial and far from scientifically accurate test it appears Google's DNS is considerably faster, and as long as they can continue to maintain this level of performance the use of their servers will be greatly beneficial to our office network.

We use ISC-dhcpd and BIND on linux servers and configure them on the boxes using vi. There's no pretty Web Based interface on a broadband router here.

If you have a broadband router, these instructions will not do you any good. Instead, your broadband router probably has a barely-usable web interface, you should RTFM.

Our DHCP hands out the addresses for two of our local servers that run BIND because we host several domains internally.

Making the change:

Assume root status on your name-server

sudo -i

Create a time-stamped backup copy of your /etc/named.conf

cp /etc/named.conf /etc/named.conf.`date +%s`

Edit the BIND configuration file called named.conf.

vi /etc/named.conf

Add the following inside the options {...} section

forwarders { 8.8.8.8; 8.8.4.4; }; //Google Public DNS

If you host zones you should exclude them from forwarding

zone "somedomain.com" IN {
type master;
forwarders { }; //don't forward
file "somedomain.internal.db";
allow-transfer {
192.168.0.215;
};
notify yes;
};

Test the new configurations

/etc/init.d/named configtest

Restart Named

/etc/init.d/named restart

Rinse and repeat for each of the name-servers that your DHCP server hands out to your clients.

Get out of root before you screw something else up

exit

Let your office enjoy.

I've been considering moving several of my shared hosting sites to a single inexpensive VPS. Linode looks like it should be my first candidate.

http://journal.uggedal.com/vps-performance-comparison

I found and reported a knife on BART today. I'm probably not the only one who saw it, but I am the only one that called and had it picked up. Bart was there and removed it in less than 3 minutes.