Category Archives: Ilmu

RhodeCode As FastCGI Program With Lighttpd In Debian

So one day I want to have a github-esque push box, somewhere I can push to collect my projects in one place that is NOT GitHub or BitBucket, or whatever. As always, my internet connection sucks and so I prefer to mirror interesting projects with incremental updates through daily/weekly/whenever-I-feels-like pull or fetch instead downloading each release.

My old setup was two separate hgweb and gitweb running as fastcgi applications connected to a lighttpd server through sockets each serving as mercurial and git web gui. In status quo, everything works and I like it.

Why Change A Perfectly Working System?

Because I can. Besides I want to get my hands on this RhodeCode thing I have put on hold for quite some time. It can handle both mercurial and git in one application, so that’s a good selling point that I would like to try.

RhodeCode Installation

My opinion on the overall installation, it was painless. First, I create a special user and group for this program:


    $ sudo adduser --disabled-login --no-create-home --uid 500 rhodecode
    

Next, I set /opt/rhodecode/ so everything about this RhodeCode stay inside that directory. Furthermore to make things easier on installation I chown that directory to my regular user & group, for now:


    $ sudo mkdir -p /opt/rhodecode
    $ sudo chown -R ariel:ariel /opt/rhodecode
    

Next, I create a new virtualenv instance:


    $ virtualenv --no-site-packages /opt/rhodecode/venv/2.7
    $ cd /opt/rhodecode/
    

This will initialize new virtualenv inside venv/2.7, the “2.7” part is because I use Debian Wheezy and its Python is at 2.7.3.

Okay now, I begin using the new virtualenv to download rhodecode and its dependencies:


    $ source venv/2.7/bin/activate
    (2.7)$ pip install rhodecode
    

Oh joy, I get to wait for something. So off to making tea then …

For some strange reason RhodeCode fail when downloading Mercurial (2.6.2). Now my pushbox is on Wheezy Stable, its Mercurial was at 2.2.2. So, while I’m at it anyway why not just use 2.6.2 directly in this virtualenv?

Mercurial Installation

I set to build a fresh Mercurial straight from upstream:


    (2.7)$ cd /tmp/
    (2.7)$ wget http://selenic.com/hg/archive/cceaf7af4c9e.tar.bz2
    (2.7)$ tar xf cceaf7af4c9e.tar.bz
    

The cceaf7af4c9e is Mercurial tag for version 2.6.2, the version that RhodeCode 1.7.1 wants. So I build that:


    (2.7)$ cd Mercurial-cceaf7af4c9e/
    (2.7)$ python setup.py build
    (2.7)$ python setup.py install
    

I’m still in my virtualenv environment as you can see in my prompt string, it says “(2.7)$“. So I know when I do an “install” the new Mercurial version will be installed to /opt/rhodecode/venv/2.7/lib/python/site-packages.

Not surprisingly Mercurial installed successfully. I have no further use for the Mercurial source code, so I remove them and get back to rhodecode directory:


    (2.7)$ cd /opt/rhodecode/
    (2.7)$ rm -rf /tmp/Mercurial-cceaf7af4c9e/
    

RhodeCode Installation Part 2

Mercurial should not be a problem anymore, continue with the installation:


    (2.7)$ pip install rhodecode
    

After waiting a while for downloading and compiling (this push-box is a re-purposed 2005-ish desktop, not exactly what you’ll call as “fast”) we are done with the downloads.

Now, according to RhodeCode documentation I should create a configuration file. I’m new, so I’ll just follow that:


    (2.7)$ paster make-config RhodeCode pushbox.ini
    (2.7)$ paster setup-rhodecode pushbox.ini
    

It then ask for “Do I want to delete database?”. Of course I say yes, I don’t have one yet!

Next, it prompts me for my repository location. I guess for my first try I should create a new directory, so I stop the job (Ctrl + Z) and do a quick mkdir:


    (2.7)$ sudo mkdir -p /srv/repos/
    (2.7)$ sudo chown ariel:ariel /srv/repos/
    (2.7)$ fg
    

Continuing the job, I put “/srv/repos/” here. Of course I don’t have anything in that directory yet, you think I’ll use my REAL repository location?

It then asks for admin user name, password, and email address, you know, the standard stuff.

Maiden Flight Of RhodeCode

Alright, let’s run this thing:


    (2.7)$ paster serve pushbox.ini 
    
RhodeCode first run

RhodeCode first run

Well, that was easy.. so I kill the RhodeCode program by pressing Ctrl + C.

What About My Needs?

Now that the “default configuration” RhodeCode is working, its time for adapting it to my requirements:

  1. First, I do not want to run another web server on port 5000. So it has to work with my current webserver: lighttpd.
  2. Web access, push and pull, must use SSL not standard http.
  3. I want it to start automatically at startup like any other fastcgi apps.

RhodeCode As A FastCGI Program

After a while browsing about Paste, I found out to make it run as fastcgi program is to specify/use flup (surprise! surprise!) on the server section. So I get flup on this virtualenv too:


    (2.7)$ pip install flup
    

To make RhodeCode use flup I put this on my /opt/rhodecode/pushbox.ini file:


    [server:main]

    ** more lines here, I commented as I don't use waitress, gunicorn, paste
       http server, etc. **

    ## BEGIN: USE FLUP FASTCGI FOR LIGHTTPD ##
    use = egg:PasteScript#flup_fcgi_thread
    socket = %(here)s/socket
    umask = 000
    ## END: USE FLUP FASTCGI FOR LIGHTTPD ##

    ** some more irrelevant lines **
    

The main idea is for flup to make RhodeCode running as threaded fastcgi program communicating to outside world using UNIX socket at /opt/rhodecode/socket, and that socket’s permission is 777.

On the web server side, I create new config file at /etc/lighttpd/conf-enabled/45-rhodecode.conf:


    # BEGIN /etc/lighttpd/conf-enabled/45-rhodecode.conf
    $SERVER["socket"] == ":443" {
        $HTTP["url"] =~ "^/repos" {
            fastcgi.server = ("/repos" => (("check-local" => "disable",
                                            "socket"      => "/opt/rhodecode/socket"))
                             )
        }
    }

    $HTTP["url"] =~ "^/repos" {
        $SERVER["socket"] != ":443" {
            $HTTP["host"] =~ "(.*)" {
                url.redirect += ( "^/(.*)" => "https://%1/$1" )
            }
        }

    }
    # EOF /etc/lighttpd/conf-enabled/45-rhodecode.conf
    

Using this configuration, lighttpd will:

To test the new configuration, I restart the lighttpd web server and try to run the RhodeCode program again:


    (2.7)$ sudo service lighttpd restart
    (2.7)$ paster serve pushbox.ini 
    

** Checks http://pushbox.home/repos/ on the browser **

This time it should work as before, using a cute little socket instead of running as http server on some port.

Debian-ish Setup

Now that basically the program is done, it’s time to make it run as daemon.

In my case I wrote my own /etc/init.d/rhodecode script. But I probably didn’t have to, because later I found out that the developer also have a nice init script for Debian:

https://secure.rhodecode.org/rhodecode/files/433d6385b216da52f68fa871ed1ff99f8d618613/init.d/rhodecode-daemon2

But for completeness sake I will post my init script too. My script is pretty much the skeleton file modified to execute “paster serve” in daemon mode under the privilege of rhodecode user and group:


    #! /bin/sh
    ### BEGIN INIT INFO
    # Provides:          rhodecode
    # Required-Start:    $remote_fs $syslog
    # Required-Stop:     $remote_fs $syslog
    # Default-Start:     2 3 4 5
    # Default-Stop:      0 1 6
    # Short-Description: rhodecode repository application service
    ### END INIT INFO

    # Author: ariel 

    # Do NOT "set -e"

    # PATH should only include /usr/* if it runs after the mountnfs.sh script
    PATH=/sbin:/usr/sbin:/bin:/usr/bin
    DESC="RhodeCode Repositories Service"
    NAME=rhodecode
    SCRIPTNAME=/etc/init.d/$NAME
    RUNUSER=rhodecode
    RUNGROUP=rhodecode
    HOME=/opt/rhodecode
    PIDFILE=$HOME/paster.pid
    LOGFILE=$HOME/paster.log
    CONFIG=$HOME/pushbox.ini
    PYTHON_PATH=$HOME/venv/2.7/
    PASTER=$PYTHON_PATH/bin/paster
    PASTER_ARGS="serve --daemon --user=$RUNUSER --group=$RUNGROUP --pid-file=$PIDFILE --log-file=$LOGFILE $CONFIG"

    # Exit if the package is not installed
    [ -x "$PASTER" ] || exit 0
    [ -f "$CONFIG" ] || exit 0

    # Load the VERBOSE setting and other rcS variables
    . /lib/init/vars.sh

    # Define LSB log_* functions.
    # Depend on lsb-base (>= 3.2-14) to ensure that this file is present
    # and status_of_proc is working.
    . /lib/lsb/init-functions

    #
    # Function that starts the daemon/service
    #
    do_start()
    {
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
        start-stop-daemon --start --quiet --chdir $HOME --chuid $RUNUSER:$RUNGROUP --pidfile $PIDFILE --exec $PASTER -- $PASTER_ARGS start > /dev/null 2>&1 || return 2
    }

    #
    # Function that stops the daemon/service
    #
    do_stop()
    {
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --start --quiet --chdir $HOME --chuid $RUNUSER:$RUNGROUP --pidfile $PIDFILE --exec $PASTER -- $PASTER_ARGS stop > /dev/null 2>&1 || return 1

        if [ -f $PIDFILE ]; then
            rm $PIDFILE
        fi

        return 0
    }

    #
    # Function that sends a SIGHUP to the daemon/service
    #
    do_reload() {
        #
        # If the daemon can reload its configuration without
        # restarting (for example, when it is sent a SIGHUP),
        # then implement that here.
        #
        start-stop-daemon --start --quiet --chdir $HOME --chuid $RUNUSER:$RUNGROUP --pidfile $PIDFILE --exec $PASTER -- $PASTER_ARGS --reload > /dev/null 2>&1 
        return 0
    }

    case "$1" in
      start)
        [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
        do_start
        case "$?" in
            0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
            2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
      stop)
        [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
        do_stop
        case "$?" in
            0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
            2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
      status)
        $PASTER $PASTER_ARGS status > /dev/null
        exit $?
        ;;
      reload)
        #
        # If do_reload() is not implemented then leave this commented out
        # and leave 'force-reload' as an alias for 'restart'.
        #
        log_daemon_msg "Reloading $DESC" "$NAME"
        do_reload
        log_end_msg $?
        ;;
      restart)
        #
        # If the "reload" option is implemented then remove the
        # 'force-reload' alias
        #
        log_daemon_msg "Restarting $DESC" "$NAME"
        do_stop
        case "$?" in
          0|1)
            do_start
            case "$?" in
                0) log_end_msg 0 ;;
                1) log_end_msg 1 ;; # Old process is still running
                *) log_end_msg 1 ;; # Failed to start
            esac
            ;;
          *)
            # Failed to stop
            log_end_msg 1
            ;;
        esac
        ;;
      *)
        echo "Usage: $SCRIPTNAME {start|stop|status|restart|reload}" >&2
        exit 3
        ;;
    esac

    :
    

Using the usual command to make it run automatically on startup:


    (2.7)$ sudo update-rc.d rhodecode defaults
    

Now that I have program set, init script and all, it’s time to clean up.

For starters, everything under /opt/rhodecode/ should be owned by user and group rhodecode:


    (2.7)$ sudo chown -R rhodecode:rhodecode /opt/rhodecode/
    

The same thing should be applied to /srv/repos/, but in my case I own it to user and group repo. Why? My current repositories that is stored elsewhere, also owned by this user/group, so I’m just being consistent here:


    (2.7)$ sudo chown -R repo:repo /srv/repos/
    

Because the rhodecode program executed under user rhodecode I also add user rhodecode to group repo, and make sure anyone in repo group can write to /srv/repos/:


    (2.7)$ sudo adduser rhodecode repo
    (2.7)$ sudo chmod -R g+w /srv/repos/
    

Some Gotchas Along The Way

  1. My first gotcha was about file permission of socket, it seems flup defaults to create socket that is not writable to others. And since my web server runs under user lighttpd (not www-data, rhodecode, or ariel) the web server returns “503 Service Unavailable”. So by adding “umask = 000” in my pushbox.ini flup will create a socket with 777 permission that is accessible to user running the program (user rhodecode) and the user running the web server (user lighttpd).
  2. Second gotcha came when pushing through rhodecode, the git client program says the server “hung up” and by adding GIT_CURL_VERBOSE=1 environment variable I see curl got “413 Requested Entity To Large” from the server.

    In my case my problem basically caused by /var/cache/lighttpd/ being owned by user www-data. You see, as I said earlier, in my setup my lighttpd program runs as user lighttpd and NOT as www-data which is the default in Debian. So if you’re having your push fail with “hung up” message, check the filesystem permission of the directory pointed by lighttpd as server.upload-dirs.

  3. The third gotcha is on the browser, my browser is Mozilla Firefox with No Script on by default on ALL domains. RhodeCode uses AJAX heavily, so if there is no repository showing on RhodeCode even though you know you already push something to it, check your No Script (temporary) whitelist.

Closing

There are some other things that need to be done such as using my real repositories location, setting proper filesystem permissions, and so on; but they are normal administrator daily work so I’ll leave that out from this post.

Some observations:

  • RhodeCode seems to install some hooks to repositories, which you might not want. There are settings somewere when you login as admin to disable this but I dont know if those would remove hook(s) that RhodeCode installed the first time it found a new repository.
  • From the log it seems, in my setup, it fails on file permission when installing some hooks but other than that I haven’t experienced any problem on pulling, cloning, and pushing to repositories.
  • Its appearance is not as sexy as github or bitbucket, sure. But I’ll manage, I rarely use web gui anyway.

2 Comments

Filed under Ilmu, Orang bego punya kegiatan

The Significance of # For Mercurial

Did you know that you have to encode “#” in your path if you work with Mercurial (hg)? I didn’t.

Let’s say I have a repository named “work#A” and so the usual Mercurial usage:

$ hg clone ~/projects/work#A mybranch
$ hg out ../work#A

wouldn’t work, and anything that requires path to a repository wouldn’t either as long you have a “#” in the path.

Excuse Me, What?

First of all you should read Mercurial’s manual on this, or if you fancy go grab a console and type:

$ hg help urls

The Mercurial guys decided to use “#” delimiter between repository path and revision. Does it sounds like a good feature to have? Can’t tell, I never use it but for me it smells like a legacy thing.

It’s A Feature Alright ..

Let’s say you have a can code in C# and you have projects under directory /media/c#/, so now you want to clone a project.

Unfortunately, if you are using Mercurial, you can’t do this:

$ hg clone /media/c#/mylinuxkernelrewrittenindotnet/ /tmp/dummybranch

to clone mylinuxkernelrewrittenindotnet to /tmp/dummybranch.

It will fail because it cannot find repository in /media/c. And if you DO have repository in /media/c, as long you dont have a tag named /mylinuxkernelrewrittenindotnet/ it will fail which is good because in this case we do not want to clone from /media/c up to tag /mylinuxkernelrewrittenindotnet/.

If you do have BOTH, a repository in /media/c and a tag named /mylinuxkernelrewrittenindotnet/ it will clone just that, successfully, but clearly it is not what you thought it is.

Wait, But I Thought Mercurial Is Intuitive?

Right away you thought you can escape this thing by adding a “\“, but nooo.. it wouldn’t work either.

So the Mercurial solution to this is to url-encode that path and put “file:” in front of the path expression. The end result should look like a URL.

Through research we know that a # url-encoded to %23. So with that knowledge, now you can do this:

$ hg clone file:/media/c%23/mylinuxkernelrewrittenindotnet/ /tmp/dummybranch

you can also use relative path as usual:

$ hg out file:../../../../work/foo%231

or :

$ hg pull file:~/work/foo%232

which I have to admit is such a hassle. But really, the alternatives to this (in Mercurial) are:

  • Just rename that stupid directory, or
  • Avoid it by using symlink without “#” in it, or
  • Not to write full path every time at all. In Mercurial we can define paths in hgrc (or .hgrc) and use them as alias.

Why I Think This # Is Ugly

Because the damned “-r” parameter exists, alive, and working that’s why.

Want to clone up to a revision? Type this:

$ hg clone -r 30 repopath newclone

Want to pull a revision from a remote repository? Type this:

$ hg pull -r c723c2da http://something.com/hg/

The reasoning why they have to treat “#” as revision identifier in a path is beyond me. Honestly I never use that feature and probably never will, I cannot think of a situation where this “feature” will be useful.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan, Pendapat gak penting

PXE: Network Booting Into Debian Setup

I just installed Debian on a 2002-ish desktop: Intel Celeron 1.7GHz, 256 SDRAM, 10 GB HDD, an aging floppy drive alongside a dodgy CD drive; just another you-wont-see-me-using-this-on-a-daily-basis desktop PC. I hostnamed it vindauga.

BIOS Setup

First of all, I have to make vindauga to look forward to booting from network. On BIOS I’ve set its boot order to LAN and then HDD, I also have to set it’s “On-board LAN” to “Enabled”.

Connect It To Network

Luckily, I already have a Debian server up and sharing its internet connection. This server is going to bootstrap the installation for vindauga.

The thing about booting from network, is to make vindauga to join my network. Luckily I already run a dhcp server (package name: isc-dhcp-server) on my Debian server, this is my setup:

# this is an excerpt from my /etc/dhcp/dhcpd.conf
subnet 192.168.77.224 netmask 255.255.255.248 {
    # this subnet is for wired LAN at eth2
    range 192.168.77.226 192.168.77.230;

    option subnet-mask          255.255.255.248;
    # broadcast address to nodes in this subnet
    option broadcast-address    192.168.77.231;

    # set resolver & gateway to this computer/my Debian-server (192.168.77.225)
    option domain-name-servers  192.168.77.225;
    option routers              192.168.77.225;

    host VINDAUGA { # assign fixed IP address for vindauga for now
        hardware ethernet       00:E0:4C:99:DF:E7;
        fixed-address           192.168.77.226;
    }

    # this is the most important of all; with this, dhcp server will provide
    # NOT ONLY network details (address, gateway, etc.) but also a bootstrap
    # program for vindauga to run
    filename "pxelinux.0";
}

The filename "pxelinux.0"; part is to make vindauga to expect a program named pxelinux.0 for booting. As you might already know a dhcp server does not transfer file, basically it just giving the client a hint about the file, the one who actually transfer that file over the wire is another program called tftp-hpa, basically it is a simplified-ftp server. In Debian you can find its package named as “tftpd-hpa“.

Get tftp-hpa To Deliver

This tftp-hpa program —by default— uses UDP on port 69 so I punched a hole in my firewall to allow this traffic.

Now, its configuration file:

# /etc/default/tftpd-hpa

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/srv/tftp"
TFTP_ADDRESS="192.168.77.225:69"
TFTP_OPTIONS="-v -s"

As you can see, on TFTP_ADDRESS I set it to listen on 192.168.77.255 port 69. This is the IP address on my Debian server for eth1, which is connected to vindauga. The TFTP_OPTIONS is to make tftp-hpa to be verbose (the “-v” part), and to serve on a specific root (the “-s”).

The “-s” part is significant because it allows the dhcp server to say just pxelinux.0 and tftp-hpa would resolve it as /srv/tftp/pxelinux.0.

What To Deliver

This (highly-anticipated :)) pxelinux.0 file can be found in the netboot distribution of Debian:

http://kambing.ui.ac.id/debian/dists/stable/main/installer-i386/current/images/netboot/

Since I’m in Indonesia it is the fitting Debian mirror for me, feel free to use another mirror.

I download pxelinux.0, all files under debian-installer/ and pxelinux.cfg/ directory and put them in /srv/tftp/. I skipped the debian-installer/i386/pxelinux.0 because its a symlink to pxelinux.0 and end up with this:

/srv/tftp/
├── pxelinux.cfg
│   └── default
├── pxelinux.0
└── debian-installer
    └── i386
        ├── linux
        ├── initrd.gz
        └── boot-screens
            ├── xfce
            │   ├── txtdt.cfg
            │   ├── txt.cfg
            │   ├── prompt.cfg
            │   ├── prmenu.cfg
            │   ├── menu.cfg
            │   ├── adtxtdt.cfg
            │   └── adtxt.cfg
            ├── vesamenu.c32
            ├── txt.cfg
            ├── syslinux.cfg
            ├── stdmenu.cfg
            ├── splash.png
            ├── rqtxt.cfg
            ├── prompt.cfg
            ├── menu.cfg
            ├── lxde
            │   ├── txtdt.cfg
            │   ├── txt.cfg
            │   ├── prompt.cfg
            │   ├── prmenu.cfg
            │   ├── menu.cfg
            │   ├── adtxtdt.cfg
            │   └── adtxt.cfg
            ├── kde
            │   ├── txtdt.cfg
            │   ├── txt.cfg
            │   ├── prompt.cfg
            │   ├── prmenu.cfg
            │   ├── menu.cfg
            │   ├── adtxtdt.cfg
            │   └── adtxt.cfg
            ├── f9.txt
            ├── f8.txt
            ├── f7.txt
            ├── f6.txt
            ├── f5.txt
            ├── f4.txt
            ├── f3.txt
            ├── f2.txt
            ├── f1.txt
            ├── f10.txt
            ├── exithelp.cfg
            ├── dtmenu.cfg
            └── adtxt.cfg

Boot It Up

I restart my dhcp server to make configuration changes into effect and start tftpd-hpa on my Debian server:

$ sudo /etc/init.d/isc-dhcp-server restart
$ sudo /etc/init.d/tftpd-hpa start

I turn on vindauga and lo, it boot into Debian’s setup menu.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan

TP-LINK TF-3200 On Debian

So I bought a FastEthernet NIC, a cheap TP-LINK PCI card, somewhere around $3,5. Here’s the box:
TP-LINK TF-3200

The model is TF-3200, and I want to install it alongside my D-LINK .. something.

Installation

The hardware installation is boring, I unwrap it, put it on one of the empty PCI slots and fasten its screw to the casing.

Get My Debian To Use It

Strangely at first boot my Debian didn’t recognize it. Yay!! There is only one way to install a PCI card and I could swear even a brick couldn’t get it wrong, so knowing hardwares love-hate relationship with Linux I have to suspect it’s more down to modules and stuff.

Fortunately, for a $3.5 LAN card, it also comes with a “Resource CD”; it’s a driver CD. In short, there is a driver for Linux located in TF-3200/LinuxDriver/ and it comes in source form.

Aww, that’s cute!

I did try to compile but somehow I cant get it to work:

$ make
make -C /lib/modules/2.6.39-2-486/build SUBDIRS=/tmp/LinuxDriver modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.39-2-486'
  CC [M]  /tmp/LinuxDriver/sundance_main.o
/tmp/LinuxDriver/sundance_main.c: In function ‘sundance_probe1’:
/tmp/LinuxDriver/sundance_main.c:712: error: implicit declaration of function ‘SET_MODULE_OWNER’
/tmp/LinuxDriver/sundance_main.c:735: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c:762: error: ‘struct net_device’ has no member named ‘open’
/tmp/LinuxDriver/sundance_main.c:763: error: ‘struct net_device’ has no member named ‘hard_start_xmit’
/tmp/LinuxDriver/sundance_main.c:764: error: ‘struct net_device’ has no member named ‘stop’
/tmp/LinuxDriver/sundance_main.c:765: error: ‘struct net_device’ has no member named ‘get_stats’
/tmp/LinuxDriver/sundance_main.c:766: error: ‘struct net_device’ has no member named ‘set_multicast_list’
/tmp/LinuxDriver/sundance_main.c:767: error: ‘struct net_device’ has no member named ‘set_mac_address’
/tmp/LinuxDriver/sundance_main.c:768: error: ‘struct net_device’ has no member named ‘do_ioctl’
/tmp/LinuxDriver/sundance_main.c:769: error: ‘struct net_device’ has no member named ‘tx_timeout’
/tmp/LinuxDriver/sundance_main.c:771: error: ‘struct net_device’ has no member named ‘change_mtu’
/tmp/LinuxDriver/sundance_main.c: In function ‘mdio_read’:
/tmp/LinuxDriver/sundance_main.c:1029: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘mdio_write’:
/tmp/LinuxDriver/sundance_main.c:1059: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_open’:
/tmp/LinuxDriver/sundance_main.c:1088: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c:1094: error: ‘SA_SHIRQ’ undeclared (first use in this function)
/tmp/LinuxDriver/sundance_main.c:1094: error: (Each undeclared identifier is reported only once
/tmp/LinuxDriver/sundance_main.c:1094: error: for each function it appears in.)
/tmp/LinuxDriver/sundance_main.c:1141: error: ‘SPIN_LOCK_UNLOCKED’ undeclared (first use in this function)
/tmp/LinuxDriver/sundance_main.c: In function ‘check_speed’:
/tmp/LinuxDriver/sundance_main.c:1195: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_timer’:
/tmp/LinuxDriver/sundance_main.c:1242: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘tx_timeout’:
/tmp/LinuxDriver/sundance_main.c:1258: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘init_ring’:
/tmp/LinuxDriver/sundance_main.c:1311: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘tx_poll’:
/tmp/LinuxDriver/sundance_main.c:1354: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘start_tx’:
/tmp/LinuxDriver/sundance_main.c:1381: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘reset_tx’:
/tmp/LinuxDriver/sundance_main.c:1423: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘intr_handler’:
/tmp/LinuxDriver/sundance_main.c:1487: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘rx_poll’:
/tmp/LinuxDriver/sundance_main.c:1621: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c:1677: error: implicit declaration of function ‘eth_copy_and_sum’
/tmp/LinuxDriver/sundance_main.c: In function ‘refill_rx’:
/tmp/LinuxDriver/sundance_main.c:1718: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_error’:
/tmp/LinuxDriver/sundance_main.c:1749: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘get_stats’:
/tmp/LinuxDriver/sundance_main.c:1772: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘set_rx_mode’:
/tmp/LinuxDriver/sundance_main.c:1801: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c:1811: error: ‘struct net_device’ has no member named ‘mc_count’
/tmp/LinuxDriver/sundance_main.c:1816: error: ‘struct net_device’ has no member named ‘mc_count’
/tmp/LinuxDriver/sundance_main.c:1822: error: ‘struct net_device’ has no member named ‘mc_list’
/tmp/LinuxDriver/sundance_main.c:1822: error: ‘struct net_device’ has no member named ‘mc_count’
/tmp/LinuxDriver/sundance_main.c:1823: error: dereferencing pointer to incomplete type
/tmp/LinuxDriver/sundance_main.c:1824: error: dereferencing pointer to incomplete type
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_ethtool_ioctl’:
/tmp/LinuxDriver/sundance_main.c:1869: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_ioctl’:
/tmp/LinuxDriver/sundance_main.c:1953: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘netdev_close’:
/tmp/LinuxDriver/sundance_main.c:2001: error: ‘struct net_device’ has no member named ‘priv’
/tmp/LinuxDriver/sundance_main.c: In function ‘sundance_remove1’:
/tmp/LinuxDriver/sundance_main.c:2118: error: ‘struct net_device’ has no member named ‘priv’
make[4]: *** [/tmp/LinuxDriver/sundance_main.o] Error 1
make[3]: *** [_module_/tmp/LinuxDriver] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.39-2-486'
make: *** [all] Error 2

Anyway, after a quick glance at the Makefile it seems it (supposed to) generate sundance.o and to install it to /lib/modules/$(KernelVersion)/kernel/drivers/net/ as sundance.ko. Fortunately a quick ls -lh /lib/modules/$(uname -r)/kernel/drivers/net/ | grep sundance told me that I already have it. So I hope I can skip it, now that I know my Debian already have its sundance.ko.

About sundance.ko

Of course, the next step is to get to know this sundance.ko thing. I query that driver by running:

$ sudo modinfo /lib/modules/$(uname -r)/kernel/drivers/net/sundance.ko
filename:       /lib/modules/2.6.39-2-486/kernel/drivers/net/sundance.ko
license:        GPL
description:    Sundance Alta Ethernet driver
author:         Donald Becker 
alias:          pci:v000013F0d00000200sv*sd*bc*sc*i*
alias:          pci:v000013F0d00000201sv*sd*bc*sc*i*
alias:          pci:v00001186d00001002sv*sd*bc*sc*i*
alias:          pci:v00001186d00001002sv00001186sd00001040bc*sc*i*
alias:          pci:v00001186d00001002sv00001186sd00001012bc*sc*i*
alias:          pci:v00001186d00001002sv00001186sd00001003bc*sc*i*
alias:          pci:v00001186d00001002sv00001186sd00001002bc*sc*i*
depends:        mii
intree:         Y
vermagic:       2.6.39-2-486 mod_unload modversions 486
parm:           media:array of charp
parm:           debug:Sundance Alta debug level (0-5) (int)
parm:           rx_copybreak:Sundance Alta copy breakpoint for copy-only-tiny-frames (int)
parm:           flowctrl:Sundance Alta flow control [0|1] (int)

I found the author (Donald Becker) match with the information found at sundance_main.c. Another interesting information I found is the dependency of this module, it says it depends on another module named mii, so I ran a check on that one too:

$ sudo modinfo mii
filename:       /lib/modules/2.6.39-2-486/kernel/drivers/net/mii.ko
license:        GPL
description:    MII hardware support library
author:         Jeff Garzik 
depends:
intree:         Y
vermagic:       2.6.39-2-486 mod_unload modversions 486

This sundance depends on mii, where we also have mii.c and mii.h lying on the source directory too, so this DEFINITELY it.

I already have it! So now it should be working after I enable those two modules, okay then:

$ sudo modprobe mii; sudo modprobe sundance;

And then I want to see if anything happened:

$ dmesg | grep -i eth | tail
.. unrelated texts ..
[53251.562523] eth1: IC Plus Corporation IP100A FAST Ethernet Adapter at 00019800, 54:e6:fc:83:12:72, IRQ 19.
[53251.563050] eth1: MII PHY found at address 0, status 0x7849 advertising 01e1.
.. unrelated texts ..

It seems my card finally recognized by Linux! One final check just to make sure :

$ ip link show
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:19:5b:5b:cb:22 brd ff:ff:ff:ff:ff:ff
3: eth1:  mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 54:e6:fc:83:12:72 brd ff:ff:ff:ff:ff:ff
4: wlan0:  mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:11:09:be:fe:0d brd ff:ff:ff:ff:ff:ff

Yep, it’s there.

Okay, Now What?

I put it into use, duh! I update my shorewall configurations and make dhcpd to serve on that interface too, etc.. all is well until I get to reboot.

I forgot that I have to automatically load those two modules at boot, so I add sundance.conf to /etc/modprobe.d:

# /etc/modprobe.d/sundance.conf
alias eth1 sundance
options sundance debug=3

Now everything works as it should be.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan

I Prefer to Start At ~/ Please

For people (me included) who gets annoyed to find they start at ~/Documents every time they use Konsole/Terminal/urxvt/whatever, Vim, Dolphin, etc. here’s what to do to make it start at ~/:

  1. Edit file ~/.config/user-dirs.dirs
  2. Add or edit an entry named XDG_DOCUMENTS_DIR having value of "$HOME/":
    XDG_DOCUMENTS_DIR="$HOME/"

  3. Save it.

Restart any application as needed. You’re welcome.

Leave a comment

Filed under Ilmu

Redo After hgpullsvn Aborted By External Causes

This is yet another case where a power outage giving you trouble. Just in case you haven’t struck by any of it in the last ~ 3 months consider yourself lucky. I know I’m not.

I just noticed —via Mercurial’s web interface— that my fpc, lazarus, and fpcdoc repository is several days old. While I haven’t put any mechanism to update my local repositories automatically on every start-up, but I kept my self to update it everyday whenever I get to use my computer; I put it as one of “after I log in”-rituals.

I’m fully aware that the fpc and lazarus SVN repository are updated daily since both are being actively developed. So I expect that when I update them the last time, exactly yesterday, so it must have pulled some changes (a.k.a “commits”). But seeing that it was last modified “5 days ago” then I strongly suspect something went (quietly) wrong.

Define “wrong”

Something not right.

Okay smart-ass, I’m going to close this tab right here in 3 … 2 …

Anyway, my fpc and lazarus local repository should be no more than 24 hours late. Because I remember doing update yesterday, and the day before that, but I’m getting report that it was updated 5 days ago. I also recall that roughly five days ago we had a power outage (note: I also recall one more outage since that day) for more than an hour or so.

Uh oh..

Exactly. This may have to do with when it was interrupted; and consequently, on how Subversion and Mercurial deal with this sort of accident.

First, I want to confirm into this power outage theory. So I login to my server computer and do a hgpullsvn manually on my local fpc repository:

$ hgpullsvn

it gave me this:

Interrupted, please wait for cleanup!

External program failed (return code 255): hg '--encoding' 'utf-8' 'up' '-C' 'default'
abort: unknown revision 'default'!

Wait, what?

I have to say that after a bit while (10 seconds or so) I left my computer because I ran out of drink. So unless someone (or something) accidentally trip over the keyboard which is very unlikely because a) I don’t have any pet, and b) the computer is in my room; nobody mess with the computers in MY room. So I ran the same thing again, this time patiently waiting in front of the screen and with more output just to make sure:

$ hgpullsvn --verbose --debug

which gave me this:

* hg '--encoding' 'utf-8' 'root'
* hg '--encoding' 'utf-8' 'qapplied'
* svn 'info' '--xml' '.'
Retrieving remote SVN info...
* svn 'info' '--xml' 'http://svn.freepascal.org/svn/fpc/trunk'
* hg '--encoding' 'utf-8' 'branch'
* hg '--encoding' 'utf-8' 'branches'
* hg '--encoding' 'utf-8' 'st' '-mard'
* hg '--encoding' 'utf-8' 'up' '-C' 'trunk'
* hg '--encoding' 'utf-8' 'branch' 'trunk'
* hg '--encoding' 'utf-8' 'parents' '--template' '{tags}'
Fetching SVN log for revisions 16959-16969...
* svn 'log' '--xml' '-v' '-r' '16959:16969' '--limit' '10' '.'
* svn '--version' '-q'
* svn 'up' '--ignore-externals' '--accept' 'postpone' '-r' '16959' '.'

Interrupted, please wait for cleanup!

* svn 'cleanup'
* hg '--encoding' 'utf-8' 'parents' '--template' '{tags}'
* svn 'up' '--ignore-externals' '-r' '16958' '.'
* hg '--encoding' 'utf-8' 'up' '-C' 'default'
External program failed (return code 255): hg '--encoding' 'utf-8' 'up' '-C' 'default'
abort: unknown revision 'default'!

So this thing stopped when downloading for commit #16959, well I just have to re-download it manually then.

Fixing It

Now I have to say there are two ways to deal with this, one is to re-download ONLY the offending commit and the other one is to download all commits since the last one until the latest. What I’m going to do here is to re-download only the offending commit, and then leave the rest for hgpullsvn.

Here’s how to download the last commit:

$ svn update -r 16958 --ignore-externals --accept postpone --force

The --force parameter is to ignore the existing, already downloaded files, and to re-download them all; or something like that. 😛

And… I need the commit-message and the name of the person doing the commit so I can put it as Mercurial commit-message and committer name:

$ svn log -r head

All relevant information duly noted.

After the Subversion part is done, I want it to register on Mercurial as well, so we take any changes and commit it as Mercurial commit. This command will add new files and delete removed files since last Mercurial commit:

$ hg addremove

Let’s commit using message and committer name from subversion.

$ hg ci -m "Do not use TProcess to run the compiler when it is not available" -u joost

Don’t forget to tag it with “svn.${SVN-REVISION-NUMBER}” because hgpullsvn needs it:

$ hg tag -r tip -l svn.16959

After manually brought the offending commit, I can continue using hgpullsvn, as usual, to pull the rest of the commits:

$ hgpullsvn --verbose --debug

Conclusion

It seems like hgpullsvn —and by proxy: Subversion and Mercurial— can not cope with sudden, non-waiting, undetectable abortion such as power outage. To be fair, there is so little any programmer can do either.

So, in the case of hgpullsvn the only thing you can do is to clean up the mess up and do what it do manually.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan

Wait, Iz Dat Arch Linux?

Got my first Arch Linux installed yesterday, a somewhat different experience for a guy used to Debian, but basically it’s all the same.

I’m bored, please entertain me with your Arch Linux endeavor

In general I spent most of the time preparing for the installation, while the installation itself took less than an hour.

Preparation before the install took 3 hours or so. The mirroring partitions off to a remote computer over 1Gb ethernet connection went about 90 minutes, coupled with partition table preparations (moving partitions, resizing, and formatting a dedicated /boot partition) took the other half of it.

Downloading the very basic stuffs compromising a modern Linux desktop is much tolerable when using a mirror nearby —that is, using local connection.

After that, all I have is a working system and ready to reboot, right? Well unfortunately, no.

So, that bastard gave you a hard time then?

In a way, yes. I swear this feels like the installer itself set up a challenge for me to solve before I can join with the *supposedly* elites being Arch Linux user.

The problem was GRUB throwing me this Error 15: File not found thing. Like any rookie, I hit the internet immediately. After browsing many support forums and mailing list, I developed a sense of what was wrong, and set to fix the damn problem.

First, I think it would be better to upgrade it to GRUB2 anyway. So I did just that by chrooting to my new system and made GRUB obsolete in one command:

pacman -S grub2-bios

after that I make sure it work this time by asking it to re-probe and rewrite the old bootloader:

grub-install --recheck /dev/sda

Of course the /dev/sda if for my SATA harddrive. If you are reading this to know how to fix your own problem, make sure to change that to whatever disk you are installing GRUB to.

But after that it’s all roses and ponies, right?

Sort of … well, kindda yes. Everything works, yes that’s a good thing. At that point I just have to install what I wants, and embracing how things work in the Arch Linux.

I mean the first thing that strikes me is this rc.conf file. I kindda like it, in Debian there are many files to edit, but in Arch Linux you just have to edit only one file for modules blacklisting, hostname, locale, daemon startups, network-configuration definition.

Of course being a KDE user myself, the next thing I do is to install KDE:

pacman -S kdebase-workspace


along with many interesting programs, including the non-vetted AUR builds.

So you are not a Debian guy anymore huh?

I am a Debian user, just not on my desktop computer. My server computer still running Debian, in fact I just did a stable-upgrade after Debian 6.0 came out. The stability side of Debian is still the best among Linux distributions in my book.

On the other side, for my day-to-day computer where I can tinker with it I chose to live on-the-edge and go with Arch Linux. 😀

1 Comment

Filed under Ilmu, Orang bego punya kegiatan, Pendapat gak penting

Bookmark Firefox Raib

Dalam dua hari terakhir seolah-olah ada setan di komputer saya. Setiap kali saya menutup Mozilla Firefox, bookmark selalu hilang. Saya restore dari backup tapi setelah restart Mozilla Firefox, hilang lagi.

Saya pakai Mozilla Firefox dengan profile khusus yang saya siapkan untuk dipakai sehari-hari, bisa di-share, dan digunakan untuk beberapa komputer; oh ya, tentu.. ada enkripsinya supaya hanya saya saja yang bisa pakai profile itu. Masalahnya, dengan embel-embel kustomisasi non-standard pabrikan Mozilla, saya bingung ini salah di program atau karena kustomisasi yang saya buat?

Masalah ini muncul setelah saya update Mozilla Firefox 3.6.13, jadi mungkin ada perilaku yang sebelumnya berfungsi sebelum versi 3.6.13 namun setelah versi ini tidak bisa lagi. Namun kustomisasi yang saya gunakan juga tidak terlalu njelimet amat; cuma script kecil supaya Mozilla Firefox memuat profile dari lokasi tertentu tempat volume ter-enkripsi di-mount. Jadi seharusnya tidak ada masalah, kecuali mulai versi 3.6.13 Mozilla Firefox menyimpan bookmark diluar dari lokasi profile.

Solusi

Setelah mencari-cari solusi di internet, sebenarnya tidak ada penjelasan secara pasti apa masalah saya. Tapi dari yang saya baca ada kemungkinan database SQLite yang dipakai untuk menyimpan bookmark sudah corrupt.

Hemmm… kalau begitu artinya SQLite tidak sepenuhnya “journal“… hmmm. Biarkan itu untuk bahan pemikiran lain waktu, namun untuk sekarang kemungkinan itu sangat terbuka.

Jadi pertama-tama saya rename file places.sqlite di dalam direktori profile saya. Setelah menjalankan Mozilla Firefox dengan cara biasa, yaitu dengan menggunakan profile khusus itu, file places.sqlite dibuat ulang dan saya bisa melakukan restore dengan mengambil bookmark dari tanggal sebelum saya update Mozilla Firefox. Hasilnya? Sukses, semua berjalan seperti biasa.

Nampaknya terbukti, file places.sqlite memang benar-benar corrupt dan saya —Mozilla Firefox— harus membuat file tersebut baru lalu melakukan restore dari backup sebelumnya.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan

Fixing PySide UI Code-Generator Error

Today while coding a program I’m working on I got this ImportError exception. It says it cannot find a module. It is one of those modules that supposed to be created automatically every time I hit the magic make button.

So this shift my curiosity to the build process: what went wrong in the building process?

This is the message that got my attention:

An unexpected error occurred.make[1]: *** [../../sekolah/app/qt/inputasalpendidikandlg_ui.py] Error 1
make: *** [qtappres] Error 2

This message must be the clue to the ImportError exception thrown by Python interpreter. So basically Python can’t import it because the build unable to create it in the first place.

Clues

The next logical thing is to try creating the UI module by hand, with debug message, so:

$ pyside-uic -d resources/qt/ui/inputasalpendidikandlg.ui

gave me:

# -*- coding: utf-8 -*-

# Form implementation generated from reading ui file 'resources/qt/ui/inputasalpendidikandlg.ui'
#
# Created: Tue Jan 14 23:33:06 2011
#      by: PySide uic UI code generator
#
# WARNING! All changes made in this file will be lost!

Traceback (most recent call last):
  File "/usr/bin/pyside-uic", line 75, in main
    options.indent)
  File "/usr/bin/pyside-uic", line 39, in generateUi
    pysideuic.compileUi(uifname, pyfile, execute, indent)
  File "/usr/lib/pymodules/python2.6/pysideuic/__init__.py", line 73, in compileUi
    winfo = compiler.UICompiler().compileUi(uifile, pyfile)
  File "/usr/lib/pymodules/python2.6/pysideuic/Compiler/compiler.py", line 37, in __init__
    CompilerCreatorPolicy())
  File "/usr/lib/pymodules/python2.6/pysideuic/uiparser.py", line 104, in __init__
    self.factory = QObjectCreator(creatorPolicy)
  File "/usr/lib/pymodules/python2.6/pysideuic/objcreator.py", line 79, in __init__
    raise WidgetPluginError, "%s: %s" % (e.__class__, str(e))
WidgetPluginError: : 'pluginType'

Hmm.. this is far more complicated than I thought. It seems, the faulty went straight to the PySide code. Because when I tried the same ui file with PyQt4’s pyuic4 it generate the module just fine.

Solving It

After reading the code on /usr/lib/pymodules/python2.6/pysideuic/objcreator.py and comparing it to PyQt4’s I found, semantically, they are doing the same thing. So the code is good, unless I missed something, which is hardly possible because both code are written nicely.

Now I know that this thing load plug-ins, by means of reading Python modules in a certain path. Putting some default variables and taking it back from the plug-in and basically do something against that. Nothing uncanny here, so what’s the big deal?

So I browse to the directory where it load its plug-ins: /usr/lib/pymodules/python2.6/pysideuic/widget-plugins/.

This is what I found:

$ ls
__init__.py	__init__.pyc	qtwebkit.py	qtwebkit.pyc

Aha! The __init__.py module being nothing more than a empty module (with its “compiled” __init__.pyc) is the culprit here. The objcreator.py executes __init__.py causing exception because the loader code expect variable pluginType to be defined by the plug-in, which in this case is an empty file.

So the easiest solution is to remove __init__.py and __init__.pyc from widget-plugins directory. At first I move the two to /tmp/ just in case something break badly I can put it back in, but after seeing make working as intended I delete them immediately.

Another alternative is to modify objcreator.py to remove __init__.py from “plugins” after enumerating files from the widget-plugins directory.

Leave a comment

Filed under Ilmu, Orang bego punya kegiatan

Multiple Items Existence Check in Python

This is new for me, sometimes I want to do this:

aList = [1, 2, 3, 5, 7];

if (2, 3) in aList:   # I want to check whether 2 and 3 is in aList
    doSomethingCool( );

that code right there is a logical error, I thought the interpreter will search aList for items inside the tuple while the truth is that it will look for the existence of a tuple with 2 and 3 in it inside aList.

Thanks to some guy’s code which I happened to stumble upon, now I know that I can use all( ) function to do that. A leaner solution than the tedious enumeration of all items and checking it one by one. Here’s a demonstration:

# instead of this ..
allFound = True;
for num in (2, 3):
    if not (num in aList):
        allFound = False;
        break;
if allFound:
    doSomethingCool( );

# now I can use this ...
if all(num in aList for num in (2, 3)):
    doSomethingCool( );

wow, look at that! It makes perfect sense because most of the gist in that sample is a list comprehension and all( ) made up the invisible AND between the list items. Another similar function to all( ) is the appropriately-named any( ) which having the effect of OR instead of AND.

Leave a comment

Filed under Ilmu