Recently I found BGP Expert site, which is a nice Internet routing reference. Once a month the site gathers a list of all the allocated IP addresses around the world from the respective authorities and dumps them into a database. The database is queryable via the Addresses Per Country page.

...the number of IP addresses used in every country of the world. The numbers are generated from information published by the Regional Internet Registries (AfriNIC, APNIC, ARIN, LACNIC and RIPE NCC) published on their FTP servers.

Using this query page, I saved the results from 1990, 2000 and 2009 to a text file, imported into MS Excel, imported again into Google Docs, and finally added a Google Docs Map widget. Here are the results:

(Note: The colors are relative to each individual map.)

China is on the rise!

Add a comment

Posted by Stu Thompson on 29 August 2010 11:39:00 CEST #

Here is the quick-n-dirty on how to configure HP ProCurve series switch ports for dynamic LACP. My direct experience so far is with the HP ProCurve 2510G-24 and the HP ProCurve 2910al-24, but it should translate to similar switches. From other examples I've seen around the Intertubes, the differences you may encounter will be with the port names.

In the below example I am configuring a 4-port LACP trunk. That said, this exact configuration would work for two 2-port trunks.

Here we go: First, log into your switch. Then follow these series of commands:

 HP2910al> 
 HP2910al> config
 HP2910al(config)# int 1-4 disable
 HP2910al(config)# int 1-4 lacp passive
 HP2910al(config)# int 1-4 enable
 HP2910al(config)# 

Next, check the status of the ports. If you have no cables connected then it should look like this.

 HP2910al(config)# show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   1      Passive   1         Down      No        Success
   2      Passive   2         Down      No        Success
   3      Passive   3         Down      No        Success
   4      Passive   4         Down      No        Success

 HP2910al(config)# 

If you do have cables connected, and the attached network node is properly configured, then this is what you should see after executing the show lacp command.:

 HP2910al(config)# show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   1      Passive   Dyn1      Up        Yes       Success
   2      Passive   Dyn1      Up        Yes       Success
   3      Passive   Dyn1      Up        Yes       Success
   4      Passive   Dyn1      Up        Yes       Success

 HP2910al(config)# 

Dynamic LACP configuration can be done only in the ProCurve command line interface. Static LACP trunks can be configured at the CLI or the ProCurve's menu system. For an example of this read How to configure Ethernet Bonding with LACP on Debian 5 Linux.

Add a comment

Posted by Stu Thompson on 25 August 2010 11:09:00 CEST #

Multiple Tomcat instances on Debian Linux? Yup, it can be done with a little bit of configuration magic. The below example assumes that you have multiple IPs already configured on your server and want to run each Tomcat instance on a single unique IP over port 80.

• Don't install a Tomcat package via apt-get or whatever. Besides the packages having a long list of problems with anything but the most trivial setup, it does not help you get multiple instances up and running. Instead download the tar files and unpack into instance specific directories. I personally run each instance with its own user in said user's home directory. E.g.:

  /home/tomcat0
  /home/tomcat0/apache-tomcat-7.0.0
  /home/tomcat1
  /home/tomcat1/apache-tomcat-7.0.0
  /home/tomcat2
  /home/tomcat2/apache-tomcat-7.0.0
  
  

• In conf/server.xml, configure the Server's port attribute to an instance specific value. I use a simple equation: the last for bytes of the IP address plue 8000. E.g.: 10.20.30.40 would use port 8040. The port is used by the bin/shutdown.sh script to signal Tomcat to shutdown. It only listens to the localhost and not the specific IP configured elsewhere.

      <Server port="8058" shutdown="SHUTDOWN"/>

• Also in conf/server.xml is the Connector, which will need configuring. In this example we have the default HTTP Connector. You'll want a unique IP for each instance, which is configured with the address atribute. And if doing port redirection then set the proxyPort attribute too. (See bottom of post for more on this.)

      <Connector port="8080" 
                 proxyPort="80" 
                 address="10.0.0.1"
                 protocol="HTTP/1.1"
                 redirectPort="8443"
               />

I've not bothered with setting up instance specific init.d scripts. For better and worse, I run my instances with manual start up and shutdown. Why? We have 'hot stand-by' servers and prefer our fail-over process to be manually instigated. If anyone one there writes some instance specific scripts I would be glad to post them here.

Regardless, the above configuration will get you up and running with multiple Tomcats on a single box!

(If you are reading this, then you may very well be interested in Port redirection for multiple Tomcat instances on linux 2.6 with iptables too.)

Add a comment

Posted by Stu Thompson on 12 August 2010 22:29:00 CEST #

"The" wife and I went on holiday to Hvar, Croatia earlier this month. We stayed at a quant little flat in town halfway up the karst hillside with an amazing view. So, having brought my camera, tripod and Arduino-based intervalometer, I setup my kit and made a few short clips.

This is my best results yet! That said, I did learn (and relearn) a few lessons for next time:

• Must put the camera into full manual mode. Any automatic settings (speed, aperture, etc.) result in flickering in the video. Semi-automatic modes are not good enough.
• Manually focus once and leave it. The auto focus takes a variable amount of time (leading to inconsistant intervals) and different focal lengths from frame to frame.
• The tripod, a cheap aluminum model, with vertical extension is too wobbly. The movement shows up when zoomed in with the 135mm lens. Keep the vertical extension collapsed.
• Stop using the 'small & coarse' JPEG image quality setting. The compression leaves unsightly artifacts in large, low contrast areas of the photos, like sky & cloud backgrounds.

One thing that kind of bugs me about the Canon is the camera's system of storing photos in sub folders with only 100 images each. This is probably because the image file name number maxes out at 9999, another irritating problem because movies can easily be the composite of 10,000+ frames. To resolve both of these issues, I've written a little Bash script that copies the images into a single target folder and renames them starting from 00001 (a requirement for FFmpeg).

#!/bin/bash
set -x
id=0
for dir in raw/*CANON
do
    echo "Processing directory $dir"
    for file in $dir/*
    do
        echo "Processing file $file"
        id=$(($id+1))
        name="$(printf "%04d" $id)" 
        
        cp -v $file "orig/$name.jpg"
    done
done

After some processing (resizing, namely) I ran the below FFmpeg command used to compose the movie from the images. It uses h264, my favorite video codec, and decent quality settings.

manoa:orig stu$ /opt/local/bin/ffmpeg -f image2 -i %05d.jpg -vcodec libx264 -f mp4
   -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8
   -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -flags2 +mixed_refs -coder 0 -me_range 16 
   -g 75 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4  -r 25 -an 
   -b 1280kb -y hvar.avi
FFmpeg version UNKNOWN, Copyright (c) 2000-2009 Fabrice Bellard, et al.
  configuration: --enable-pthreads --enable-libx264 --enable-libmp3lame --enable-libfaad
    --enable-libfaac --disable-ffplay --disable-ffserver --disable-static --enable-shared 
   --enable-nonfree --enable-gpl --enable-postproc --enable-hwaccel=mpeg2_vaapi 
   --enable-hwaccel=mpeg4_vaapi --enable-hwaccel=h263_vaapi --enable-hwaccel=wmv3_vaapi 
   --prefix=/opt/local --extra-cflags=-I/opt/local/include 
   --extra-ldflags=-L/opt/local/lib
  libavutil     50. 3. 0 / 50. 3. 0
  libavcodec    52.35. 0 / 52.35. 0
  libavformat   52.38. 0 / 52.38. 0
  libavdevice   52. 2. 0 / 52. 2. 0
  libswscale     0. 7. 1 /  0. 7. 1
  libpostproc   51. 2. 0 / 51. 2. 0
  built on Dec 15 2009 17:01:07, gcc: 4.0.1 (Apple Inc. build 5493)
Input #0, image2, from '%05d.jpg':
  Duration: 00:07:05.96, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj420p, 1280x720 [PAR 72:72 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
[libx264 @ 0x1028600]using SAR=1/1
[libx264 @ 0x1028600]using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
[libx264 @ 0x1028600]profile Baseline, level 3.1
Output #0, mp4, to 'hvar.avi':
    Stream #0.0: Video: libx264, yuv420p, 1280x720 [PAR 1:1 DAR 16:9], q=10-51, 1280 kb/s, 25 tbn, 25 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame=10649 fps=  5 q=16938494.0 Lsize=   73318kB time=425.96 bitrate=1410.0kbits/s    
video:73233kB audio:0kB global headers:0kB muxing overhead 0.115394%
[libx264 @ 0x1028600]frame I:162   Avg QP:28.94  size: 82562
[libx264 @ 0x1028600]frame P:10487 Avg QP:31.99  size:  5875
[libx264 @ 0x1028600]mb I  I16..4: 31.0%  0.0% 69.0%
[libx264 @ 0x1028600]mb P  I16..4:  2.4%  0.0%  0.7%  P16..4: 32.4%  5.3%  2.2%  0.0%  0.0%    skip:56.7%
[libx264 @ 0x1028600]final ratefactor: 29.67
[libx264 @ 0x1028600]coded y,uvDC,uvAC intra:36.9% 27.2% 12.7% inter:6.6% 2.1% 0.1%
[libx264 @ 0x1028600]ref P L0  48.7% 18.1% 16.8%  7.9%  8.4%
[libx264 @ 0x1028600]kb/s:1408.4
manoa:orig stu$ 

Once the video was uploaded to youtube.com, I added the sound with YouTube's AudioSwap functionality. That feature is very convenient for adding audio to these movies. It sorts the available audio clips into a logical set of categories, has a search function and a "find sound tracks about the same length as my video" option.

Responses[1]

Posted by Stu Thompson on 11 August 2010 20:50:00 CEST #

Over the past two months my company (xtendx AG) moved our servers to a new data center at green.ch. One of the primary motivations for this move was to gain access to a multiple gigabit per second Internet link. Each of our production streaming servers have either 2 or 4 channels ethernet bonding configuration with LACP. Once they were configured I set out to test their capacity and validate the entire design and configuration.

To that end, I installed iperf onto each of our servers. One box was configured as a server. Two others were configured as clients. Because of the method LACP uses to split up the traffic, it is impossible or merely very difficult to setup a server-to-server that uses more than 1Gbps. Generally, LACP accomplish multiple link speeds by splitting the individual client-server traffic over the two links on a client-server address pair basis.

Our setup here is fairly simple: three servers, each with either a 2-channel or 4-channel LACP ethernet bonding setup connected to a lone HP ProCurve 2510G-24 switch. The switch was manually configured to place the ports into a dynamic LACP bond. The bonds themselves are configured with the kernel module parameters mode=4 miimon=100 max_bonds=4 xmit_hash_policy=1

Running iperf in server mode. Note that iperf uses port 5001 by default, so adjust your firewalling solution if necessary.

iperf -s -i 2

Running iperf in client mode. This was done on two physically separate machines.

cat /dev/zero |  iperf -c svr.example.com -t 2400 -i 2

Yup, looks good. There was the possibility that both clients would have come in on the same link. This is possible because the decision about which channel to use is based upon a the source and destination addresses. It is also by design--don't fret! Simply using a different server for one of the clients would resolve the issues.

Add a comment

Posted by Stu Thompson on 09 August 2010 14:31:00 CEST #