Stuff I do

People dealing with cron, and bash scripts that might take a bit longer than they're supposed to often encounter the following behaviour. Suppose that you launch something from cron in say every hour. The stuff usually completes in 10 minutes, but sometimes, when the load peaks, or network clogs, the process is running much slower. After an hour an other one is launched, further hogging the resources of the machine and possibly messing up data. The solution for this is to pay attention not to run twice of course (and also to fix the underlying problem that causing the slowdowns).

How to write fast programs with numpy and python At first, I wanted to write a post about "how to write fast programs with python and numpy". After writing a few test cases it quickly turned out that the fast, numpy versions of tests are not only fast because of numpy, but because they use numpy "the right way". Let's see what does it mean.

Thrift Thrift is apache's tool. It can generate client / server codes based on a file written in it's own descriptor language. At first I was thrilled how easy it'll be to write a Cassandra client with it: "you just have to generate the C files, #include them, call a few functions and it's done". Yeah. Like anything in the world works like that. And this particular thing is no exception.

I had to write a kernel module recently that implemented a block device. I've carefully read the LDD3 material on every topic I thought might be useful; I concentrated mainly on block devices of course. Linux block drivers can do their job by providing a function that the kernel will call if it needs to do IO on the device. The first method described in LDD3 was the "request" method. According to this, the driver had to provide a function (wich was called the "request" function after the name of the field in a struct that is registered with the kernel at driver initialization). This function is called whenever a so-called request is available in a so-called request queue. The request does not equal to a single IO request that the kernel received from userspace (like when you do a read() on a disk for example). Such user space originating requests are merged and/or splitted to form suitably sized and appropriately queued chunks of IO to do.

Python is a great language but it's far from being a complete tool. It do have got a lot of libraries, many of them already built into the standard distribution. However as you do more and more work with it, chances are that you'll have to do something that does not have a canned solution ala Python. This usually means that you have to interface with C libraries or you even have to write the C code yourself. It's possible and even not too hard to write C libraries that can be loaded by Python and expose Python objects and functions. If you want to rewrite existing Python code to be faster, it's probably the way to go. (I strongly advise you to think twice before you act, such work won't be an easy thing to do.) So if you have to write your own Python things in C, read this . If you haven't done much C before, this gonna hurt. Fortunately there is one particular case when you don't have to write C code at all, even it smells like low-level coding: the case

Whenever you learn a new language, you shall write a program that outputs "Hello World". It seems to be some kind of a law of the Universe. While such a useless piece of software fails to demonstrate almost any properties of the language itself, it's still a way to show how to compile and/or run programs on the platform in question. There are a lot of classical problems that could be used to briefly scratch the surface of a new language. My favorite classical programming problem is drawing (an approximation of) the Mandelbrot set. This exercise is versatile enough to shed some light on the language's philosophy and toolset, while being relatively simple. Last time I stumbled upon Haskell, a really shiny gem among the languages I know. Guess what, I wrote a Mandelbrot set generator: https://github.com/netom/mandelbrot.hs . I have to thank Daniel Pek ( http://blog.pdani.org/ ) for telling me about his experiences with Haskell and infect me with the functional virus.