dm's technical weblog

Learning Pyramid: 1// Installation

nospam@example.com (Dirk Makowski) — Thu, 02 Jun 2011 11:50:13 +0200

Use virtualenv and virtualenvwrapper.

Put this into ~/.bashrc

# virtualenv
export WORKON_HOME=$HOME/myprojects
source /etc/bash_completion.d/virtualenvwrapper

Create a virtual environment with

mkvirtualenv $ENV
workon $ENV
cdvirtualenv

Install Pyramid and other eggs:

pip install --upgrade distribute
pip install pyramid
pip install pyramid_openid pyramid_beaker

Create a Pyramid project

# Traversal, no DB
bin/paster create -t pyramid_starter $PROJECT
cd $PROJECT
which python # check whether virtualenv is correctly installed
which paster
python setup.py develop # create development environment
# Run tests
python setup.py test -q
pip install nose coverage
nosetests --cover-package=$PROJECT --cover-erase --with-coverage

Start server:

cd $PROJECT
paster serve development.ini --reload

Parallel processing batches of images with ImageMagick, reconsidered

nospam@example.com (Dirk Makowski) — Mon, 02 May 2011 13:10:08 +0200

Recently I showed some bash scripts to process images with ImageMagick in parallel. Boy, was I astonished when I read more about GNUs "parallel" command: It does not only scale perfectly on all CPUs and all cores, it also can use several machines via the network.

The task to scale down all photographs can be condensed into this simple command line:

1024$ ls ../all/*.JPG | parallel convert {} -resize 1024x1024 {/.}_preview.jpg

See also the great comments to this blog entry about "parallel".

Parallel processing batches of images with ImageMagick

nospam@example.com (Dirk Makowski) — Fri, 29 Apr 2011 14:07:03 +0200

From my recent holiday trip to Lanzarote I brought back nearly 600 photographs. When I reviewed them, it became quite clear that my picture-taking skills are not that impressive and most of the images, if not all, need some kind of retouching, most notably, adjusting the white-balance.

The Gimp becomes more and more my friend, but processing all of the photographs manually would rather quickly wear out my mouse arm and index finger. Some kind of scripting would be nice here, as scripts also can be kept for later reference on what I did.

Continue reading "Parallel processing batches of images with ImageMagick"

Parenchym 2

nospam@example.com (adm: Dirk Makowski) — Sat, 17 Apr 2010 17:27:51 +0200

Unter diesem Titel werde ich loser Folge über die Neuentwicklung von Parenchym schreiben. Im Grunde könnte Parenchym wie es ist einfach um neue Funktionen erweitert werden. An manchen Stellen hatte ich meine Ideen jedoch nicht konsequent genug umgesetzt, und diese verdienen eine Vereinfachung. Ausschlaggebender ist jedoch, dass ich in Parenchym die neuen Sprachkonstrukte wie z.B. Namespaces einführen möchte. Und um alle Klassen in Namensräumen anzuordnen muss ich ohnehin fast den gesamten Code refaktorieren -- und dies scheint mir ein geeigneter Zeitpunkt, auch die architektonischen Änderungen umzusetzen.

Natürlich wird der bestehende Zweig von Parenchym noch weiter gepflegt und ausgebaut werden. Parenchym 2 ist zum jetzigen Zeitpunkt nicht viel mehr als eine Studie.

Deutsche Ländernamen im PrestaShop

nospam@example.com (adm: Dirk Makowski) — Thu, 25 Feb 2010 15:54:27 +0100

Möchte man den PrestaShop in Deutsch installieren, erhält man bei prestashop-deutschland.de/ und dem angeschlossenen Forum Rat und Hilfe. Auch eine schon eingedeutschte Variante kann man dort herunter laden.

Die Namen der Länder sind jedoch noch nicht übersetzt. Man kann zwar die Namen im Backend einzeln selbst übersetzen, aber das war mir zu mühselig.

In der Tabelle "ps_country" speichert PrestaShop Informationen wie die interne LänderId (id_country) und den ISO Code. Die Tabelle "ps_country_lang" enthält die übersetzten Namen in verschiedenen Sprachen. Und Tabelle "ps_lang" enthält die Namen der Sprachen.

So kann man sich einen Überblick verschaffen:

SELECT * FROM ps_country c, ps_country_lang cl, ps_lang l WHERE c.id_country=cl.id_country AND cl.id_lang=l.id_lang;

Der aktuelle Datenbestand enthält zwar schon Übersetzungen für deutsch (id_lang=3), aber die Namen sind dennoch englisch.

Continue reading "Deutsche Ländernamen im PrestaShop"

Finding stuff in multiple files and opening them in gVIM

nospam@example.com () — Sun, 21 Feb 2010 17:15:44 +0100

Currently I'm working on some DokuWiki template which is based on the arctic template. DokuWiki itself scatters the CSS styles in several files like 'layout.css', 'design.css etc. The arctic template adds its own bunch of files like 'arctic_layout.css', 'arctic_design.css' and so on.

Now I needed to change the color of some page element. But where in this plethora of style files is the matching CSS definition? I called upon several good friends to find it.

Firstly I fired up FireBug which nicely showed which CSS classes were applied to that page element. As it turned out, the element was wrapped in a div tag and formatted by a CSS class named 'meta'.

Fine. Now I need to find the string "div.meta" (or, maybe just ".meta") in the style files. Of course we know the commands 'find' and 'grep' will quickly show us the results, but today -- as an additional sugar -- I wanted to open all found files together in the same instance of gVIM. Ah this is why I love the command line:

find . -name '*.css' -or -name '*.php' | xargs grep div.meta \
|awk -F: '{print $1;}' |sort -u | xargs gvim

Let's split that command up:

The find command lets us only look into the wanted files. And it does that recursively, if we would need that.
With xargs we feed those filenames nicely into grep. And grep prints each line which matches the expression, prepended with the name of the file where the line was found.
We are only interested in the filenames, so awk will get them for us: "split the line at colons and print the first column".
Maybe there are more than one matching lines in the same file. Therefore we build an unique list of 'em.
And lastly, let xargs provide the list as arguments to gVIM

That's it.

Well, not quite. DokuWiki allows to use some kind of variables in the CSS files. Consequently, the file which defines the CSS for 'div.meta' does not contain the sought color information, but the name of a variable. And this, FINALLY, I can change in the file 'style.ini'. Argh.

Repair grub with grml

nospam@example.com () — Wed, 14 Oct 2009 09:05:39 +0200

My development box runs a bleeding edge Debian sid installation. One day an update produced the message that grub 0.97 is now called grub-legacy and that it will chainload into the new (extremely bleeding edge) grub 2. Should that work I could run upgrade-from-grub-legacy. Well, that sounded nice and I researched what is the status of Grub 2. Almost everybody who wrote about Grub 2 noted, that documentation lacks a lot -- so I won't reproduce this statement here Fortunately, in the caveat section the draft of the Grub 2 manual gives the following hint

deb upgrades - In the Debian 5.x and Ubuntu 9.x series, upgrading to GRUB 2 after an an LVM/LVM+crypto installation (under GRUB legacy) will leave you unhappy.

Hmpf. Made a mental note not to touch above command, because I am using lvm. But yesterday evening my mental state was kind of unconscious and out of -- whatever it was, that had driven me -- I entered upgrade-from-grub-legacy. A dialogue asked me which devices Grub 2 shall be installed on and -- @$""§ again without thinking -- I selected both drives. [Deviation] The computer has two different brands of hard drives drives, an old one and a new one, with different sizes and partitioning. The system used to identify them as /dev/hda and /dev/sda. A recent upgrade of the kernel introduced some confusion here, because suddenly the hda was populated as sdb! Researching (many thanks to all the people who write about their knowledge) found that the new kernels see all drives as "sd". So I changed the fstab and everything worked again. [/Deviation] Back to Grub 2. The installation and a reboot finished without errors. Null sweat here. But I noticed that the Vista entry in Grub's menu was missing. So I installed and run os-prober which found the Vista partition and also produced an entry for the splash screen (which I disliked but accepted). Strangely, it did not list my Linux kernels! As Whitesnake had put it:

When I first saw you baby You took my breath away, I knew your name was Trouble But, my heart got in the way I couldn't stop myself from reaching out, I could not turn away

In my case my brain just passed out. I did a reboot. Great, the Grub menu appeared and an entry for Vista was there. -- BUT IT WAS THE ONLY ONE! Uniqueness might be applicable for certain rings, but certainly it was displeasing for my boot menu. Very! Hardened from battling with overwritten Lilo boot records in the past, and in the very comforting situation of having a quite recent backup I did not break into sweat. Browsing through the pile of stuff on my desk produced two things, in that order: a disc of grml Schluchtenscheisser, and a grin on my face.

Linux Virus Shock?

nospam@example.com () — Tue, 23 Jun 2009 18:00:07 +0200

A couple of minutes ago something weird happened on my Linux box: I had been working on some graphics in Gimp and was clicking through the windows when suddenly a song started to play. I was startled, and it lasted only for a few notes and then fell silent again. My first (paranoid) suspicion was, somehow a virus had pirated my machine or, some hacker was trying out commands in a shell... Hmm, a less into /var/log/auth.log showed nothing suspicious. But then, a capable hacker could have concealed her traces. Unfortunately I had no more ideas about how to find out the source of the song and so I finished the graphics. When the desktop reappeared after all of Gimp's windows were closed, I remembered that I had saved that song onto the desktop. And there was another mp3. I wanted to listen to it again and moved the mouse cursor upon its icon, but -- before I could double-click, the song started to play! Aww, I was tricked by the (new?) Gnome (or Nautilus?) behaviour, to play a sound file immediately when the cursor hovers it. That must have happened when I browsed through Gimp's windows earlier on!

Today's favourite documentation egg

nospam@example.com () — Sun, 22 Feb 2009 02:47:01 +0100

:nunmap can also be used outside of a monastery.

Found in VIM's help, describing the various map commands.

What Wikipedia has to do with Star Wars

nospam@example.com () — Sat, 27 Dec 2008 18:11:27 +0100

Hmm, what an interesting sight... [caption id="attachment_101" align="aligncenter" width="300" caption="Vaderpedia"][/caption]

Microsoft Outlook Security -- WTF?

nospam@example.com () — Tue, 16 Dec 2008 08:19:43 +0100

Aus aktuellem Anlass suche ich im Internet nach Informationen ueber die Sicherheit der Emailanzeige in Microsoft Outlook. Obwohl ich schon einiges gewoehnt bin von Microsoft, verschlaegt es mir bei folgender Dreistigkeit doch dermassen die Sprache, dass ich entgegen meiner sonstigen Gewohnheit nicht einmal mehr faehig bin, in Englisch zu schreiben. Doch der Reihe nach. Zunaechst schaute ich bei Google, welche Seiten zum Thema "Outlook Sicherheit" zu finden sind. Prima, es sind sogar 2 Links zu Artikeln bei Microsoft mit unter den ersten Ergebnissen: 1 und 2. Aber statt zu den erhofften Artikeln fuehren sie lediglich zu dem Hinweis "Um die optimale Anzeige Ihrer Webseiten zu unterstuetzen, wird ueberprueft, ob ein Programm von 2007 Microsoft Office installiert ist." Wow. Dass man neuerdings zur Ansicht von Webseiten nicht mehr einen Browser wie Firefox oder auch Microsofts Internet Explorer benoetigt, sondern ein Microsoft Office der neuesten Generation, ist allerdings ein starkes Stueck. Natuerlich surfe ich mit abgeschaltetem JavaScript, das hat jene Seite richtig erkannt, und so klicke ich selbst auf den angebotenen Link, endlich zum erwarteten Artikel zu gelangen. Aber weit gefehlt! Die folgende Seite bestaetigt, -- nun auf bunte Weise --, dass die Informationen nur fuer Besitzer von MS Office zugaenglich sind, und man wird gleich aufgefordert, die neueste Version zu kaufen. Ist doch interessant, dass man jedoch diese Webseite ohne JavaScript und auch gänzlich ohne Microsoft-Programme lesen kann! (Hervorhebung im Bild von mir.) Was haben wir gelernt?

Die Aussage "Um die optimale Anzeige Ihrer Webseiten zu unterstuetzen", dient nur dazu, einen Laien zum Kauf von MS Office zu noetigen. Was ist an der Anzeige der bunten Webseite nicht optimal, ganz ohne Microsoft Programme? -- Abgesehen davon, dass sie nicht den versprochenen Inhalt hat.
Informationen zu Office bekommt man von Microsoft offenbar nur, wenn man schon Office gekauft hat. Sich vorab zu informieren oder als unabhaengiger Consultant sich zu informieren, ist ausgeschlossen.
Nicht nur wird man zum Kauf von MS Office genoetigt, man muss auch Microsoft erlauben, den eigenen Computer zu durchsuchen! Unser Bundestrojaner wird vor Neid erblassen!

So, WTF? Eigentlich wollte ich mich nur ueber MS Outlook informieren, um meine Kunden gut unterstuetzen zu koennen, aber was passiert? 2 Stunden verschwendet fuer nichts. Danke, Microsoft.

Parenchym

nospam@example.com () — Mon, 25 Aug 2008 08:56:47 +0200

The long planned documentation for my web application framework Parenchym is online. Finally I found a way to describe and present it that suits me right. Since the pages have just started, most of them are in German, but translation into English is under way. http://parenchym.com

My new photoalbum is online

nospam@example.com () — Sun, 03 Aug 2008 13:00:35 +0200

I am happy to announce that Pixy, my new photoalbum is now online. Visit Pixy here

Convert your photo collections into an online album

nospam@example.com () — Sat, 02 Aug 2008 20:26:21 +0200

It has been a long time since I lastly uploaded a new gallery to DigiPics, my old gallery in The Annwn. Now, this does not mean, I had not taken any pictures in the meantime. Au contraire, mon capitain! But how could I handle all those photographs more efficiently? For some picture sets I had made special pages to display them, as you can see in DigiPics. But I only want to repeat this for special occasions. For now I want a method to process the images, like converting them to suitable sizes for displaying on screen, and show them in some decent album, where the visitior can browse through sets and all of that. Additionally, the EXIF data of the photographs should be displayed, so that one could get an impression how they were taken. Of course, you could argue, there already are plenty of picture sharing sites. Yeah, what about being techie if I'd use someone else's program? What I finally have produced are (1) some scripts to process the images, and (2) a small application to display the albums. First, the scripts All scripts (two, in fact) can be installed into Gnome's file browser Nautilus, so you only have to select the images to process, right-click and select the script from the context menu. create_thumbs is a small shell script that resamples the selected images into various sizes, 120px for thumbnails, 640px and 1024px. It also rotates them according to the orientation flag in the EXIF data. Of course, the script relies on external programs to accomplish this, namely convert from ImageMagick, jhead and jpegtran. get_exif is a Perl script that fetches all EXIF data from the original image and stores in in a separate file. I use Image::ExifTool for this. In a first attempt I used PHP's EXIF functions, but as it turned out, the Perl lib is better, especially because it interprets the MakerNotes quite nicely. Download the scripts Second, the application With the scripts we produced a nice gallery and now we need some way to display it. Pixy does right this. It is (almost) a single-file PHP script, which borrows some code and ideas from my Web Application Framework Parenchym. Look inside pixy.php for more information how to set the galleries up. In the next days I will upload my galleries, so that you can see Pixy live. Maybe I'll also publish the sources.

PHP Madness: Floats, type casts and locales

nospam@example.com () — Thu, 26 Jul 2007 10:17:40 +0200

While working on my framework Parenchym, I encountered the following situation: In a form a user has to input a floating point value, which will then get processed. After that, the form with the latest input is re-displayed. Now, Parenchym is very thorough when it comes to checking the user's input. Not only does it check whether the value is valid or not, it also casts PHP's internal type to the right one. As you will remember, all input is recieved by PHP as string. So, the user's float comes in as a string as well, but Parechym converts that to float again. So far, so good. Enter the user's locale. For this examination, we'll suppose it is set to `de_DE.utf8'. The first input poses no problem, because the user (which was me, at that time) is used to input floats with a decimal point. The input of 8.3 enters the script as "8.3" and Parenchym returns it as 8.3. Great. But the shit hits the fan when the form, and so the float, is displayed again. This time the locale settings strike! The value in the form field is suddenly "8,3". Mind the decimal comma here! Submitted again, PHP's type cast -- completely ignorant of the locale settings -- truncates the float, rendering it as 8! WTF! Lesson learned today: PHP respects locale settings when printing floats, it DOES NOT respect locale settings when doing type casts!