20100927

FTBFS Fixing / More news on the Sun Java 6 PPA

Let's start with the Java things:

First of all, thanks to all of you who were commenting on the last article about the Sun Java6 Community PPA. I took the input and this is the outcome:


  1. Packages for Ubuntu 10.04 LTS: Done
  2. Package for Ubuntu 8.04 LTS: Done
  3. Asked Doko to push Sun Java packages to the partner archive: Done
Also thanks to Petar Velkovski, who was so kind to write an email, that I should tell you that you eventually need to update your alternatives and your java alternatives with 
"update-alternatives --config java" and "update-java-alternatives -s java-6-sun".

The second work today way to try to fix some FTBFS packages (FTBFS == Failed To Build From Source). 

Thanks to Lucas Nussbaum, who rebuilt the entire Ubuntu archive and provided a list of all packages which are FTBFSing.
I tried to fix most of the Universe/Multiverse packages, at least I'm trying to do some packaging work again, next to my daily business.

Sadly I touched one package which gives me now nightmares. It's using an old auto*foo (version 1.6) which is not anymore in our archives. I wonder if I can fix it really.

20100926

Sun Java6 Packages [updated]

Update: The packages are working like a charm.

Today I needed sun-java6 packages for Maverick.
But since Ubuntu 10.04 LTS the sun-java6 packages are not anymore in the standard archives of Ubuntu. Instead the packages are distributed via the partner archive of Canonical.

So far so good, until I found out, there are no packages inside the partner repository for Maverick.

So, I took the last available package, checked the rules file, and started to create a new version of this package.

You can find the result on a new team PPA which you can find here: https://launchpad.net/~sun-java-community-team/+archive/sun-java6

Right now, I just can say, they are building under maverick on amd64 and i386. Inside the PPA buildservers this will take another 11 hours or so.

I didn't test the installation on my system, but will do so today.

But I really wonder, why we don't provide updated packages until now.

@Doko, you can take the packages from the PPA and take them for the Partner Archive.

During the next days, I'll push the new version of the package as well for Lucid to the PPA.

Have a nice weekend :)

20100917

Thoughts about a loadbalanced DNS setup

During the last days, we had a discussion about how we provide a simple, but always available DNS Server setup.

Right now, most of the domains are served by public and internal DNS servers of our office IT.
So, to not fiddle around with zone files, I thought it would be a good idea to have at least two DNS slave servers, which are filled by the master servers of our office IT.

These two slave DNS servers will serve all hosts and zones for our DCs (hence, there is just more then this setup), behind a IVPS/LVS loadbalancer.

This is the inkscape graph I came up with:

The IPVS/LVS Loadbalancer itself is an active/passive pacemaker cluster.
So I sat down and created a proof of concept on a testing VMWare ESX Server.

I created four machines, 2 DNS, 2 IPVS.

The used software for the two IPVS/LVS machines are:

  1. Ubuntu 10.04 (aka Lucid Lynx)
  2. pacemaker
  3. corosync
  4. ipvsadm
  5. ldirectord
The used software for the two DNS Servers are:

  1. Ubuntu 10.04 (aka Lucid Lynx)
  2. Bind9
The network is setup like this:

  • DNS-01: 192.168.10.10
  • DNS-02: 192.168.10.11
  • LB-DNS-01: 192.168.10.20
  • LB-DNS-02: 192.168.10.21
  • DNS-Server VIP: 192.168.10.100


Starting with the easy part of this setup:

Installation of the two DNS Server:

  1. Writing a Puppet recipe for the two DNS Servers, for setting up the production network, install the DNS software, deployment of the bind9 configurations, especially to provide the slave zones.
  2. Provisioning of the two DNS Server machines in (DC)²
  3. Deployment via FAI, Puppet runs inside the FAI install run
So, from preparation over provisioning to the running production system this took round about 30 minutes.

Now for the IPVS/LVS Loadbalancers.

This wasn't trivial, because of pacemaker and the strange configuration of it.
Therefore I decided to do the default deployment via FAI and Puppet but left the special Pacemaker part out of the test setup.

So here it goes:


  • Corosync configuration
    I just used the default config and just adjusted the totem/interface section:
    • LB-DNS-01:
      • interface {
                        # The following values need to be set based on your environment
                        ringnumber: 0
                        bindnetaddr: 192.168.10.0
                        mcastaddr: 226.94.1.1
                        mcastport: 5405
                }
    • LB-DNS-02:
      • interface {
                        # The following values need to be set based on your environment
                        ringnumber: 0
                        bindnetaddr: 192.168.10.0
                        mcastaddr: 226.94.1.1
                        mcastport: 5405
                }
Now you edit /etc/default/corosync and set the environment variable "START" to "yes".

When everything is ok (you should just check /var/log/syslog), you have a clean pacemaker cluster. If you are using some special Cisco or Juniper Network devices, don't forget to enable multicast on the connected ports (as I have two ESX Machines under control, I had to do that on the host access port of our Cisco switch).

Now for the pacemaker CIB setup.
I don't use any stonith resources, to make that clear before you ask, I don't trust the external/vmware stonith agent. And regarding the test setup I'm working on, this is also not necessary.
To avoid no failover without stonith agents, you have to tell pacemaker to 
  1. stonith-enabled: false
  2. no-quorum-policy: ignore
Without those settings, you won't see a failover while a node is disappearing.

So, here is my excerpt of the CIB:
<crm_config>
        <cluster_property_set id="cib-bootstrap-options">
        <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.0.8-042548a451fce8400660f6031f4da6f0223dd5dd"/>
        <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="openais"/>
        <nvpair id="cib-bootstrap-options-expected-quorum-votes" name="expected-quorum-votes" value="2"/>
        <nvpair id="cib-bootstrap-options-stonith-disable" name="stonith-enabled" value="false"/>
        <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore"/>        
      </cluster_property_set>
</crm_config>
<resources>
        <clone id="resDNSClone">
                <meta_attributes id="resDNSClone_Meta">
                        <nvpair id="resDNSClone_meta_max" name="clone-max" value="2"/>
                        <nvpair id="resDNSClone_meta_node_max" name="clone_node_max" value="1"/>
                </meta_attributes>
                <primitive id="resDNSClone_ldir" class="ocf" provider="heartbeat" type="ldirectord">
                        <operations>
                                <op id="resDNSClone_ldir_op" name="monitor" interval="30s" timeout="10s"/>
                        </operations>
                        <meta_attributes id="resDNSClone_ldir_meta">
                                <nvpair id="resDNSClone_ldir_meta_threshold" name="migration-threshold" value="10"/>
                                <nvpair id="resDNSClone_ldir_meta_target" name="target-role" value="Started"/>
                                <nvpair id="resDNSClone_ldir_meta_fail" name="on-fail" value="standby"/>
                                <nvpair id="resDNSClone_ldir_meta_quorum" name="required" value="quorum"/>
                        </meta_attributes>
                        <instance_attributes id="resDNSClone_ldir_attribs">
                                <nvpair id="resDNSClone_ldir_config" name="configfile" value="/etc/ha.d/ldirectord.cf"/>
                        </instance_attributes>
                </primitive>
        </clone>
        <clone id="resDNSPingd">
                <primitive id="resDNSClone_Pingd" class="ocf" provider="pacemaker" type="pingd">
                        <instance_attributes id="resDNSClone_Pingd_attribs">
                                <nvpair id="resDNSClone_Pingd_attribs_hostlist" name="host_list" value="172.24.24.1 172.24.24.20 172.24.24.21"></nvpair>
                                <nvpair id="resDNSClone_Pingd_attribs_dampen" name="dampen" value="5s"></nvpair>
                                <nvpair id="resDNSClone_Pingd_attribs_multiplier" name="multiplier" value="100"></nvpair>
                                <nvpair id="resDNSClone_Pingd_attribs_interval" name="interval" value="2s"></nvpair></instance_attributes></primitive>
        </clone>
        <group id="resDNSLB">
                <meta_attributes id="resDNSLB_meta">
                        <nvpair id="resDNSLB_order" name="ordered" value="false"/>
                </meta_attributes>
                <primitive id="resDNSIP" class="ocf" provider="heartbeat" type="IPaddr2">
                        <operations>
                                <op id="resDNSIP_failover" name="monitor" interval="10s"/>
                        </operations>
                        <meta_attributes id="resDNSIP_meta">
                                <nvpair id="resDNSIP_meta_fail" name="on-fail" value="standby"/>
                        </meta_attributes>
                        <instance_attributes id="resDNSIP_attribs">
                                <nvpair id="resDNSIP_ip" name="ip" value="172.24.24.100"/>
                                <nvpair id="resDNSIP_nic" name="nic" value="eth0"/>
                                <nvpair id="resDNSip_cidr" name="cidr-netmask" value="24"/>
                        </instance_attributes>
                </primitive>
        </group>
</resources>
<constraints>
        <rsc_colocation id="resDNSIP_colo" rsc="resDNSLB" with-rsc="resDNSClone" score="INFINITY"/>
</constraints>

So, this CIB xml you save somehow under cluster.xml and then just do a:

sudo cibadmin --replace -x cluster.xml

on one cluster node and now your cluster should be running, or?

What's missing?

Ah yes, the ldirectord setup, I put this config file under /etc/ha.d/ldirectord.cf (as you can read above), because in past times, this was the location where you found the ldirectord configuration.

ldirectord configuration:

checktimeout=10
checkinterval=15
failurecount=3
negotiatetimeout=10
autoreload=yes
logfile="local0"
quiescent=yes


virtual=192.168.10.100:53
        real=192.168.10.10:53 gate
        real=192.168.10.11:53 gate
        protocol=tcp
        scheduler=rr
        request="testhost.internal.zone.tld"
        receive="172.24.24.10"
        service=dns
        
virtual=192.168.10.100:53
        real=192.168.10.10:53 gate
        real=192.168.10.11:53 gate
        protocol=udp
        scheduler=rr
        request="testhost.internal.zone.tld"
        receive="172.24.24.10"
        service=dns

Now, just reboot both boxes, and ssh after reboot into your boxes.
Start the pacemaker monitoring tool via "sudo crm_mon" and just playaround.

20100915

Enter Flattr

I just added Flattr to my latest web 3.0 things to enjoy.

I think, I'll test this flattr thing for a couple of months, and see how far it goes.

I submitted my (DC)² Project to it, to eventually participate in flattr.

Furthermore, I spent the monthly flattr ammount to the debian dpkg developers, phpmyadmin, gajim and KeepPassX.

Debian DPKG Developers, because DPKG is the base of my daily work and as well Ubuntus base.
PhpMyAdmin, because it helps not so mysql cli affine developers to do their work. We are using it in our company a lot, that's why I want to support this project.
Gajim, Oh yes, it's one of the coolest xmpp clients written in python. I love it.
KeepPassX, a nifty tool for distributed password storage and sharing in a team. A must have.

20100913

SysAdmin tasks made easy.

Sometimes you write simple scripts or shell functions without sharing them, because you forget how important they are in your daily work.

During the last weeks, while I followed the FAI mailinglist, I saw, that some people do have problems doing simple Debian/Ubuntu SysAdmin tasks like setting up an Debian/Ubuntu archive mirror.

Therefore I catched up with my little helper scripts, and thought: Heck, let's make a small shell library from it and push it to the public.

So, during the next days/weeks I'll present some nifty helper scripts/helper functions which could make your life a bit easier.

This is also important, because (DC)² and FAI are needing some of those standard admin tasks to be done before you could even start with the automation of your OS Deployment in your datacenter.

Furthermore I'll invite you to share your nifty and tricky shell/perl/python/ruby scripts of your daily SysAdmin work.

20100912

Oh why....

Yesterday , September 11th, I saw a post, IMHO from Steve Stalcup, about "I'll never forget..."

Sadly this post he removed, whysoever. I really appreciate those posts, because it will give us a reminder of  human kinds darkest past and it shows the reader, that even Geeks are Human.

I'm Sorry For The New York People Who Lost Loved Ones...

I'm Sorry For People Who Were Losing Loved Ones During German Darkest History.

I'm Sorry For People Who Were Losing Loved Ones During Tiananmen Square Protests, 1989.

I Hope That People Will Learn From Those Killings And That Human Kind Will Never Ever Do This Again.

Dear Politicians, Learn From Those Happenings. Especially US Government, German Government and Chinese Government You Won't Solve Problems With Millitary Force.


I Still Remember My Feelings, When I Heard About That Incident On 9/11. I Just Called People From Companies I Worked With And Those People Were Just Some Blocks Away From Ground Zero.

But, I'm Not Hating Muslims, I'm Not Hating Chinese, I'm not Hating Germans, I'm Hating Politicians, It Doesn't Matter What Religion They Have, Or What Country They Are Coming From.

Never Forget!

And Yes, This Style Of This Post Is With Purpose

20100902

Just in case you are a HP BL4* G6/7 user

and you have an HP NC 511i Emulex Dual Port 10GB Ethernet/ISCSI/FoE Adapter on board,
it works with Ubuntu Lucid and Maverick (Daily from Yesterday).

The only bug, which is now known as LP bug #628776, is that in the installer kernel module udeb packages the be2{net,scsi} modules are missing.

When you continue the installation without network Ubuntu Lucid/Maverick comes up and detects this card and loads the kernel modules.

20100901

News on the (DC)² Front

During the last days I spent some time on playing around with Mono and C#.
The result is, that I started to write a desktop client for the (DC)² Project.
It's far from complete, less views then the web client, but hey, it's a start.

You can find the sources on Launchpad.net.

And no, I don't want to hear any "Mono is Evil and you know the patents" trolling. For me, it's a good start to learn a new language, and to tackle sometimes with our local developers, who are using C# / .NET and of course Mono, too. I could have used Python or Perl, but both I know, and I could have used Java as well, but we all know what Oracle is up to.

You can directly startup your Monodevelop and check the projects source files.
Furthermore, I'm using the XMLRPC C# Library from Cook Computing